Network Pulse

← All notes

The journey label moves at the moment the bus arrives

Status: measured and corrected. Affects 25–47% of journeys depending on operator. The recovery is live in avl_poller.py from 12 September 2026. Look here first if terminus capture on a service falls.

A bus finishes the 120 into Fulwood and its journey reference changes from 1217C to 1331B — the next journey on its block. Nothing about that is wrong. The reference moves when the bus reaches the place the next journey starts from, which is where the last one ended, and that is how a running board works.

The problem is that it moves at the same instant the bus arrives, and arriving is the thing we are trying to record.

Why that costs us the arrival

An arrival is timed between two fixes. The bus is at 424 metres on one fix and 10 metres on the next, the stop sits between them, so the crossing is interpolated and recorded. That needs both fixes to carry the same journey, because progress along a journey is only meaningful within it.

At a terminus the second fix is already badged to the next journey. So the final stretch falls in a gap: the journey that is ending has no more fixes, and the journey that is starting has not departed and owns nothing yet. Everything crossed in that last window goes unrecorded — the terminus, and any stop before it in the same window.

Vehicle FSYO-37488 on 12 September, from the raw SIRI-VM archive:

13:18:53   1217C   424 m from the terminus
13:19:35   1217C   159 m
13:20:14   1217C    50 m
13:20:35   1331B    10 m      <- the label moves, at the stop
13:20:35+  1331B    10 m      standing there for the next six minutes

Our last recorded stop for 1217C was at 13:18:58, when the bus was still 424 metres out. The two stops it covered after that were lost.

Both feeds behave the same way, and neither is wrong

This was worth checking, because "BODS mangles it in conversion" would be a different problem with a different owner. It doesn't.

Sampling the GTFS-RT feed — the one the poller reads — over 34 trip changes across South Yorkshire, measuring how far the bus was from the old journey's last stop at the moment its trip id changed:

Distance from the old journey's last stop Changes
At the stop (under 100 m) 28
100–300 m 4
300 m – 1 km 1
Over 1 km 1

The same as SIRI-VM: the label moves on arrival. There is nothing to raise with the operator, with Vix, or with BODS. The feeds describe what the bus is doing. What was wrong was our assumption that a journey's last fix would still carry that journey's label.

Two earlier readings of this, both wrong and both recorded here so the same wrong turnings are not taken twice: that the label changed "two stops out and four minutes early", which was inferred from our own records rather than measured against the bus's position; and that GTFS-RT switched earlier than SIRI, which the sampling above disproves.

Why Vix has these arrivals and we did not

On the 120 on 10 September, against the timetabled final stop of each of the 67 journeys both systems hold:

Journeys
Both have the terminus 32
Only Vix has it 24
Neither has it 11
Only we have it 0

Vix 56 of 67, this platform 32, and not one journey the other way round. Vix does not reconstruct journey membership from a position stream: the equipment knows which journey the driver signed on to and records the arrival against it. The label moving is not a problem for a system that is not inferring anything from the label.

That Vix misses 11 is its own question, and not explained by anything here.

How widespread

Every operator, on 11 September, counting journeys that ended short of their last stop and ended at the stop where that vehicle's next journey began:

Operator Journeys Complete Recoverable
Go North East 5,417 47.2% 2,218 40.9%
First South Yorkshire 3,126 43.1% 773 24.7%
Stagecoach Yorkshire 2,655 43.7% 951 35.8%
Stagecoach East Midlands 1,437 31.8% 418 29.1%
TM Travel 350 39.7% 164 46.9%
Hulleys of Baslow 231 36.8% 68 29.4%

Network-wide that day: 78,153 journeys, 38,911 ended short of their last stop, and 21,289 of those — 54.7% — ended where the vehicle's next journey began. So this one cause accounts for more than half of all truncated journeys and 27% of every journey recorded.

The other half is a different problem: buses that go quiet well before the end, a mean of 7.6 stops short, and nothing here touches those.

What the fix does

Poller._finish_previous in avl_poller.py. When the feed moves a bus to a new journey, the journey it is leaving gets one last reading: the bus's current position is located on that journey's geometry, and the ordinary observation code runs over the stretch from where it was last seen to where it is now. Whatever stops it crossed in that window are recorded against the journey it was running.

It fires once, at the switch. It does not keep the old journey open.

Only where the journey ending and the journey starting meet at the same stop, which is the case this is about and which keeps a bus that vanished mid-route and reappeared elsewhere from being credited with an arrival it never made. Plus the existing guards: the bus must locate on that geometry without being off route, and the gap must be short enough and the implied speed possible.

Nothing is invented and there is no special-cased terminus row — _observations is the same function the normal path uses every poll, so only recordable stops are kept, the terminus is judged on its arrival, and implausible delays are discarded. stop_observations is uniquely keyed on (service_date, trip_id, stop_sequence), so a stop already recorded cannot be recorded twice.

The poller's log counts them: "N recovered from journeys the feed re-badged early".

Does it work

Like-for-like, the same 14:13–15:45 window, the three South Yorkshire operators, about 600 journeys a day:

Day Journeys Final stop Last two stops
9 Sep 636 40.6% 80.8%
10 Sep 614 22.8% 50.0%
11 Sep 640 40.3% 80.2%
12 Sep (fix live) 592 83.4% 86.0%

The second column is the tell. Before the fix, 80% of journeys reached the penultimate stop while only 40% recorded the last — half of them stopping one stop from the end. Today those are 86.0% and 83.4%. The gap between "nearly finished" and "finished" has closed from 40 points to 2.6.

Network-wide the gain is smaller, +4.9 points, because the fix only reaches the half of journeys that end where their successor begins, and outside South Yorkshire only timing points are recorded at all.

The caveat to carry

A recovered arrival is timed by interpolating between the last fix on the old journey and the first fix after the switch, rather than read from a fix taken at the stop. In the first hour, terminus arrivals recorded this way had a median of +53 seconds against −13 seconds for the rest — later, which is the direction the method would bias if a bus arrives and then stands before the next fix lands.

That matters because "early at the terminus" is a reported measure, and a late-biased arrival understates earliness. The sample was small and mixed, so this is a thing to watch rather than a measured bias: if terminus earliness falls on the services this recovers most, suspect this before suspecting the buses.

What to look at if this changes

  • terminus_capture.py, run nightly at 04:25 by terminus-capture.timer. Both measures, split by whether a journey ends where its successor begins against those that do not — the second is the control that says whether a change was this or just the day.
  • The poller's recovery count in its log, which should be non-zero in any hour of daytime running.
  • The raw archive in data/bods-archive, where a vehicle's journey reference and position can be read poll by poll. That is what settled this, twice, after two wrong explanations drawn from derived tables.

Still open: the switch is not always at the stop

Added 13 September 2026, and it qualifies the reading above rather than replacing it.

Capture reached 87.5% on the morning of 13 September — the first measurement taken wholly under both fixes, on 224 Sunday-morning journeys, against 46–55% on the two Sundays before. Of the 28 journeys that finished without a terminus:

  • 6 are not recoverable by anything. The vehicle stopped transmitting before it arrived.
  • 12 of the remaining 18 provably arrived: the next observation of that bus is at the very stop we recorded it as never reaching.

One of those traced in the archive, FSYO-37473 on the 97:

05:54:47   97  0557B   205 m from the terminus
05:55:01   98  0705    174 m      <- the label moves, well short of the stop
05:55:26   98  0705     14 m
05:55:37   97  0557B    22 m      <- and moves back
05:55:58   98  0705     22 m      <- and forward again

Two departures from what this note says above. The label moved at 174 metres, not on arrival, so closing the journey out at the switch located the bus short of the arrival mark and recorded nothing. And it flickered — returned to the old journey once the bus was at the stop, by which time the state had been closed.

The flicker is not the general cause. Over three hours on 13 September, 806 vehicles produced 407 label switches and 5 A→B→A flickers, 1.2%. The likelier explanation for the rest is the tail of the distribution measured on 12 September: 28 of 34 switches inside 100 m left roughly a sixth further out, and those are exactly the ones that lose the arrival.

The next step is to measure the switch-distance distribution over a full weekday before changing any code. If most switches are 150–200 m out then the close-out logic is wrong in general and should not be patched case by case. The candidate fix, if it is a tail rather than the rule: the next journey starts where the last one ended, so a journey closed out short whose vehicle opens its next journey at that terminus within a few minutes has evidenced its arrival, and should be credited using the last fix that still carried the old label — 22 metres in the trace above, comfortably inside the tolerance.

Nothing has been changed on the strength of this.

When the label moves mid-journey, not at the terminus

Status: recovered for one journey, 15 September 2026. Everything above is about the label moving as the bus arrives. This is the same mechanism going wrong in the middle of a journey, and it costs the whole journey rather than its last two stops.

FSYO-36303, the 21:50 inbound on the 120, Sunday 13 September, from the raw archive:

21:51:48  2150E  inbound  370020717 -> 370010115  aimed 21:50  53.37001,-1.55082
21:52:11  2150   inbound  120       -> 120        aimed  -     53.37001,-1.55080
21:52:42  2150   inbound  370020717 -> 370026725  aimed  -     53.37001,-1.55080

Twenty-three seconds apart, the same spot to two metres, the same block. Three things change at once and each one breaks a different key:

  • journey_code goes to the 0259 sentinel, so matching on the code fails.
  • OriginAimedDepartureTime disappears, so the fallback on departure time fails.
  • DestinationRef changes from Sheffield Interchange to Eckington Way — the 120's other terminus, on the 93-stop working rather than this 34-stop one.

The middle record is the giveaway that this is a garbled publish rather than a reassignment: both stop fields contain 120, the line number.

The identity is not lost, it moves one field over

dated_ref on the sentinel run holds 2150 — the journey code of the run before it. So a run whose dated_ref equals an earlier run's journey_code, same vehicle, is that journey continuing. That is the fourth key in replay_archive.resolve(), guarded twice: the earlier run must be unambiguous, and the trip must not already be held by another vehicle.

It fires rarely — 23 runs across the network on a weekday, 3 on the Sunday, and the second guard caught one wrong carry on 14 September.

This is not the rule measured in feed-coverage-blind-spot.md and found to be a no-op. That one credited a sentinel run to whatever the vehicle worked immediately before, to fill the archive-match column, and added nothing because the preceding run had already matched on its own code. This one keys on dated_ref and feeds the replay, where the preceding run matching is exactly the point: it is what supplies the trip id the continuation inherits.

It was the 21:50, not the working the record claimed

Worth proving rather than assuming, because the re-badged record names a real destination. Every Eckington Way journey from Barncliffe Road that evening was worked end to end by a Stagecoach bus:

Journey Destination Stops seen Vehicle
21:05 Eckington Way 93 of 93 SYRK-11716
21:20 Sheffield Interchange 34 of 34 FSYO-37479
21:50 Sheffield Interchange 0 of 34 FSYO-36303
22:05 Eckington Way 92 of 93 SYRK-11709

A First vehicle cannot have been any of them, and the record named the 21:50 itself a minute before the switch.

What the recovery is worth, and what it is not

Replaying 13 September with the carry key recovers 17 of the 34 stops, sequence 0-16, running two to three minutes late. First's Sunday 120 goes from 39 of 40 journeys with evidence to 40 of 40, and stop coverage from 98% to 99%.

The terminus figure does not move and should not: 39 of 40 before and after. The journey has no arrival to record, because the record froze at 22:00:15 by Tapton Park Road — the same timestamp and position republished in every payload until midnight. That is also where the bogus polls = 384 on that run comes from: about 25 real records and 359 repeats. At 17 of 34 the journey is 50% tracked, below the 80% bar, so it stays out of the terminus-capture denominator either way.

Whether the bus completed the last seventeen stops, the archive cannot say.

The 17 stops were promoted into stop_observations with promote_replay.py, carrying source = 'replay' (migration d5a17c3fb420). Everything the poller wrote is source = 'live'. The journey explorer marks a recovered count with a dagger, and every such row is one query away:

SELECT * FROM stop_observations WHERE source <> 'live';

first_last_daily and lost_mileage_daily for that date were then recomputed. One row changed in each, and every other row for the day came back byte-identical — which is the check worth repeating before any future recompute of a past day, given both tables are meant to be written once:

first_last_daily   FSYO 120 last inbound:  observed f -> t, stops 0 -> 17
lost_mileage_daily FSYO 120:  operated 39 -> 40 journeys, lost 5.8 -> 3.4 mi