The journey label moves at the moment the bus arrives
Status: measured and corrected. Affects 25–47% of journeys depending on
operator. The recovery is live in avl_poller.py from 12 September 2026. Look
here first if terminus capture on a service falls.
A bus finishes the 120 into Fulwood and its journey reference changes from 1217C to 1331B — the next journey on its block. Nothing about that is wrong. The reference moves when the bus reaches the place the next journey starts from, which is where the last one ended, and that is how a running board works.
The problem is that it moves at the same instant the bus arrives, and arriving is the thing we are trying to record.
Why that costs us the arrival
An arrival is timed between two fixes. The bus is at 424 metres on one fix and 10 metres on the next, the stop sits between them, so the crossing is interpolated and recorded. That needs both fixes to carry the same journey, because progress along a journey is only meaningful within it.
At a terminus the second fix is already badged to the next journey. So the final stretch falls in a gap: the journey that is ending has no more fixes, and the journey that is starting has not departed and owns nothing yet. Everything crossed in that last window goes unrecorded — the terminus, and any stop before it in the same window.
Vehicle FSYO-37488 on 12 September, from the raw SIRI-VM archive:
13:18:53 1217C 424 m from the terminus
13:19:35 1217C 159 m
13:20:14 1217C 50 m
13:20:35 1331B 10 m <- the label moves, at the stop
13:20:35+ 1331B 10 m standing there for the next six minutes
Our last recorded stop for 1217C was at 13:18:58, when the bus was still 424 metres out. The two stops it covered after that were lost.
Both feeds behave the same way, and neither is wrong
This was worth checking, because "BODS mangles it in conversion" would be a different problem with a different owner. It doesn't.
Sampling the GTFS-RT feed — the one the poller reads — over 34 trip changes across South Yorkshire, measuring how far the bus was from the old journey's last stop at the moment its trip id changed:
| Distance from the old journey's last stop | Changes |
|---|---|
| At the stop (under 100 m) | 28 |
| 100–300 m | 4 |
| 300 m – 1 km | 1 |
| Over 1 km | 1 |
The same as SIRI-VM: the label moves on arrival. There is nothing to raise with the operator, with Vix, or with BODS. The feeds describe what the bus is doing. What was wrong was our assumption that a journey's last fix would still carry that journey's label.
Two earlier readings of this, both wrong and both recorded here so the same wrong turnings are not taken twice: that the label changed "two stops out and four minutes early", which was inferred from our own records rather than measured against the bus's position; and that GTFS-RT switched earlier than SIRI, which the sampling above disproves.
Why Vix has these arrivals and we did not
On the 120 on 10 September, against the timetabled final stop of each of the 67 journeys both systems hold:
| Journeys | |
|---|---|
| Both have the terminus | 32 |
| Only Vix has it | 24 |
| Neither has it | 11 |
| Only we have it | 0 |
Vix 56 of 67, this platform 32, and not one journey the other way round. Vix does not reconstruct journey membership from a position stream: the equipment knows which journey the driver signed on to and records the arrival against it. The label moving is not a problem for a system that is not inferring anything from the label.
That Vix misses 11 is its own question, and not explained by anything here.
How widespread
Every operator, on 11 September, counting journeys that ended short of their last stop and ended at the stop where that vehicle's next journey began:
| Operator | Journeys | Complete | Recoverable | |
|---|---|---|---|---|
| Go North East | 5,417 | 47.2% | 2,218 | 40.9% |
| First South Yorkshire | 3,126 | 43.1% | 773 | 24.7% |
| Stagecoach Yorkshire | 2,655 | 43.7% | 951 | 35.8% |
| Stagecoach East Midlands | 1,437 | 31.8% | 418 | 29.1% |
| TM Travel | 350 | 39.7% | 164 | 46.9% |
| Hulleys of Baslow | 231 | 36.8% | 68 | 29.4% |
Network-wide that day: 78,153 journeys, 38,911 ended short of their last stop, and 21,289 of those — 54.7% — ended where the vehicle's next journey began. So this one cause accounts for more than half of all truncated journeys and 27% of every journey recorded.
The other half is a different problem: buses that go quiet well before the end, a mean of 7.6 stops short, and nothing here touches those.
What the fix does
Poller._finish_previous in avl_poller.py. When the feed moves a bus to a new
journey, the journey it is leaving gets one last reading: the bus's current
position is located on that journey's geometry, and the ordinary observation code
runs over the stretch from where it was last seen to where it is now. Whatever
stops it crossed in that window are recorded against the journey it was running.
It fires once, at the switch. It does not keep the old journey open.
Only where the journey ending and the journey starting meet at the same stop, which is the case this is about and which keeps a bus that vanished mid-route and reappeared elsewhere from being credited with an arrival it never made. Plus the existing guards: the bus must locate on that geometry without being off route, and the gap must be short enough and the implied speed possible.
Nothing is invented and there is no special-cased terminus row — _observations
is the same function the normal path uses every poll, so only recordable stops
are kept, the terminus is judged on its arrival, and implausible delays are
discarded. stop_observations is uniquely keyed on (service_date, trip_id,
stop_sequence), so a stop already recorded cannot be recorded twice.
The poller's log counts them: "N recovered from journeys the feed re-badged early".
Does it work
Like-for-like, the same 14:13–15:45 window, the three South Yorkshire operators, about 600 journeys a day:
| Day | Journeys | Final stop | Last two stops |
|---|---|---|---|
| 9 Sep | 636 | 40.6% | 80.8% |
| 10 Sep | 614 | 22.8% | 50.0% |
| 11 Sep | 640 | 40.3% | 80.2% |
| 12 Sep (fix live) | 592 | 83.4% | 86.0% |
The second column is the tell. Before the fix, 80% of journeys reached the penultimate stop while only 40% recorded the last — half of them stopping one stop from the end. Today those are 86.0% and 83.4%. The gap between "nearly finished" and "finished" has closed from 40 points to 2.6.
Network-wide the gain is smaller, +4.9 points, because the fix only reaches the half of journeys that end where their successor begins, and outside South Yorkshire only timing points are recorded at all.
The caveat to carry
A recovered arrival is timed by interpolating between the last fix on the old journey and the first fix after the switch, rather than read from a fix taken at the stop. In the first hour, terminus arrivals recorded this way had a median of +53 seconds against −13 seconds for the rest — later, which is the direction the method would bias if a bus arrives and then stands before the next fix lands.
That matters because "early at the terminus" is a reported measure, and a late-biased arrival understates earliness. The sample was small and mixed, so this is a thing to watch rather than a measured bias: if terminus earliness falls on the services this recovers most, suspect this before suspecting the buses.
What to look at if this changes
terminus_capture.py, run nightly at 04:25 byterminus-capture.timer. Both measures, split by whether a journey ends where its successor begins against those that do not — the second is the control that says whether a change was this or just the day.- The poller's recovery count in its log, which should be non-zero in any hour of daytime running.
- The raw archive in
data/bods-archive, where a vehicle's journey reference and position can be read poll by poll. That is what settled this, twice, after two wrong explanations drawn from derived tables.
Still open: the switch is not always at the stop
Added 13 September 2026, and it qualifies the reading above rather than replacing it.
Capture reached 87.5% on the morning of 13 September — the first measurement taken wholly under both fixes, on 224 Sunday-morning journeys, against 46–55% on the two Sundays before. Of the 28 journeys that finished without a terminus:
- 6 are not recoverable by anything. The vehicle stopped transmitting before it arrived.
- 12 of the remaining 18 provably arrived: the next observation of that bus is at the very stop we recorded it as never reaching.
One of those traced in the archive, FSYO-37473 on the 97:
05:54:47 97 0557B 205 m from the terminus
05:55:01 98 0705 174 m <- the label moves, well short of the stop
05:55:26 98 0705 14 m
05:55:37 97 0557B 22 m <- and moves back
05:55:58 98 0705 22 m <- and forward again
Two departures from what this note says above. The label moved at 174 metres, not on arrival, so closing the journey out at the switch located the bus short of the arrival mark and recorded nothing. And it flickered — returned to the old journey once the bus was at the stop, by which time the state had been closed.
The flicker is not the general cause. Over three hours on 13 September, 806 vehicles produced 407 label switches and 5 A→B→A flickers, 1.2%. The likelier explanation for the rest is the tail of the distribution measured on 12 September: 28 of 34 switches inside 100 m left roughly a sixth further out, and those are exactly the ones that lose the arrival.
The next step is to measure the switch-distance distribution over a full weekday before changing any code. If most switches are 150–200 m out then the close-out logic is wrong in general and should not be patched case by case. The candidate fix, if it is a tail rather than the rule: the next journey starts where the last one ended, so a journey closed out short whose vehicle opens its next journey at that terminus within a few minutes has evidenced its arrival, and should be credited using the last fix that still carried the old label — 22 metres in the trace above, comfortably inside the tolerance.
Nothing has been changed on the strength of this.
When the label moves mid-journey, not at the terminus
Status: recovered for one journey, 15 September 2026. Everything above is about the label moving as the bus arrives. This is the same mechanism going wrong in the middle of a journey, and it costs the whole journey rather than its last two stops.
FSYO-36303, the 21:50 inbound on the 120, Sunday 13 September, from the raw
archive:
21:51:48 2150E inbound 370020717 -> 370010115 aimed 21:50 53.37001,-1.55082
21:52:11 2150 inbound 120 -> 120 aimed - 53.37001,-1.55080
21:52:42 2150 inbound 370020717 -> 370026725 aimed - 53.37001,-1.55080
Twenty-three seconds apart, the same spot to two metres, the same block. Three things change at once and each one breaks a different key:
journey_codegoes to the0259sentinel, so matching on the code fails.OriginAimedDepartureTimedisappears, so the fallback on departure time fails.DestinationRefchanges from Sheffield Interchange to Eckington Way — the 120's other terminus, on the 93-stop working rather than this 34-stop one.
The middle record is the giveaway that this is a garbled publish rather than a
reassignment: both stop fields contain 120, the line number.
The identity is not lost, it moves one field over
dated_ref on the sentinel run holds 2150 — the journey code of the run
before it. So a run whose dated_ref equals an earlier run's journey_code,
same vehicle, is that journey continuing. That is the fourth key in
replay_archive.resolve(), guarded twice: the earlier run must be unambiguous,
and the trip must not already be held by another vehicle.
It fires rarely — 23 runs across the network on a weekday, 3 on the Sunday, and the second guard caught one wrong carry on 14 September.
This is not the rule measured in feed-coverage-blind-spot.md and found to be
a no-op. That one credited a sentinel run to whatever the vehicle worked
immediately before, to fill the archive-match column, and added nothing because
the preceding run had already matched on its own code. This one keys on
dated_ref and feeds the replay, where the preceding run matching is exactly
the point: it is what supplies the trip id the continuation inherits.
It was the 21:50, not the working the record claimed
Worth proving rather than assuming, because the re-badged record names a real destination. Every Eckington Way journey from Barncliffe Road that evening was worked end to end by a Stagecoach bus:
| Journey | Destination | Stops seen | Vehicle |
|---|---|---|---|
| 21:05 | Eckington Way | 93 of 93 | SYRK-11716 |
| 21:20 | Sheffield Interchange | 34 of 34 | FSYO-37479 |
| 21:50 | Sheffield Interchange | 0 of 34 | FSYO-36303 |
| 22:05 | Eckington Way | 92 of 93 | SYRK-11709 |
A First vehicle cannot have been any of them, and the record named the 21:50 itself a minute before the switch.
What the recovery is worth, and what it is not
Replaying 13 September with the carry key recovers 17 of the 34 stops, sequence 0-16, running two to three minutes late. First's Sunday 120 goes from 39 of 40 journeys with evidence to 40 of 40, and stop coverage from 98% to 99%.
The terminus figure does not move and should not: 39 of 40 before and after. The
journey has no arrival to record, because the record froze at 22:00:15 by
Tapton Park Road — the same timestamp and position republished in every payload
until midnight. That is also where the bogus polls = 384 on that run comes
from: about 25 real records and 359 repeats. At 17 of 34 the journey is 50%
tracked, below the 80% bar, so it stays out of the terminus-capture denominator
either way.
Whether the bus completed the last seventeen stops, the archive cannot say.
Promoted, and marked
The 17 stops were promoted into stop_observations with promote_replay.py,
carrying source = 'replay' (migration d5a17c3fb420). Everything the poller
wrote is source = 'live'. The journey explorer marks a recovered count with a
dagger, and every such row is one query away:
SELECT * FROM stop_observations WHERE source <> 'live';
first_last_daily and lost_mileage_daily for that date were then recomputed.
One row changed in each, and every other row for the day came back
byte-identical — which is the check worth repeating before any future
recompute of a past day, given both tables are meant to be written once:
first_last_daily FSYO 120 last inbound: observed f -> t, stops 0 -> 17
lost_mileage_daily FSYO 120: operated 39 -> 40 journeys, lost 5.8 -> 3.4 mi