Network Pulse

← All notes

SYMCA's reporting requirements, answered against what we already hold

Status: measured. Every figure below comes from this system on 10 September 2026, not from a specification. The numbers are reproducible from the database and the pages they feed.

SYMCA has a list of issues with reporting from Vix Op Reports and Mosaiq. This note goes through it in the same order and says, for each, whether we can answer it today, and how. It is deliberately about capability rather than comparison: several of these are things any system built on the open feed gets for free, and a few are things no system built on the open feed can ever do.

The short version: sections 1, 2, 4a, 4b and 5 we already answer. What is left for a supplier is passenger numbers, revenue, raw ETM extracts, and any history before 4 August 2026.

1. Elements missing from the data

Service date, 04:00 to 03:59

Answered, and for the reason the request itself gives. A journey leaving at 23:50 and running past midnight keeps the previous service date, with times past 24:00:00 — so a 01:00 departure is 25:00 on the day before, exactly as described. avl_match.resolve_service_date considers both candidate days and takes the one whose scheduled window actually brackets the observation, rather than assuming the clock date.

The full scheduled timestamp is stored alongside, so a strict 04:00 boundary is a reporting choice rather than a change to the pipeline. Both conventions can be shown side by side if a figure needs reconciling against an old one.

Journey sequence: first, last and intermediate across the service day

Answered, and it is a contract measure here rather than an add-on. first_last_daily holds 34,682 rows from 4 August onwards — per service, per direction, per day, whether the first and last journey ran. Every journey in the delivery extract also carries is_first and is_last, so the same question can be asked of a single day or of a period.

"Per service" means per route from 17 September 2026, not per line number. A number is not a service: Stagecoach run two services called 21, one of them 04:48 to 23:45 and the other 06:05 to 18:52, and keyed on the number the earlier one took both ends of the day while the other had no first or last bus recorded at all. Seven numbers in South Yorkshire are in that position. Rows written before that date keep the per-number answer and cannot be rebuilt — the rollup records what was true on the night — so a period spanning 17 September holds both, and the export says which rather than leaving a blank that would read as a bus that did not run.

Any journey with any stop in South Yorkshire

Answered, and this is the sharpest of the three. The request notes that the SY filter appears to select on destination, losing services that start here and run out of the county.

Measured today: 345 lines call at a South Yorkshire stop, and 92 of them — 26.7% — also call outside it. A destination-based filter loses a quarter of the network's cross-boundary services.

Two mechanisms answer it, independently:

  • Punctuality is attributed per stop, not per journey. Every ATCO stop code begins with its administrative area, so punctuality_daily is keyed on the area the departure happened in. A cross-boundary journey contributes rows to both areas, and neither is credited with the other's performance.
  • The timetable side records every area a service calls at. Andrews of Tideswell's 257A reads 100,370 — eighty stops in Derbyshire and a handful in Sheffield — and is in scope. Their 173 reads 100 and is not, though the same dataset contains both.

2. How the data is accessed

Five separate problems are listed: two browser versions with different capabilities, one of them requiring IE and unusable from SYMCA devices; incomplete downloads; a limit imposed by having to render a dataset on screen before downloading it, which failed between 200,000 and 300,000 records; missing column headings for fields added as rows; and columns packed with semicolons that need processing before they can be databased.

All five are properties of a reporting tool sitting between the user and the data. There is no such tool here. Figures are produced by a scheduled job and written as files:

  • No browser, so no browser memory limit. August alone holds 18.7 million observations for the whole record and 5.3 million for South Yorkshire. The question of whether 300,000 rows will render does not arise.
  • The journey-times report emits Vix's own CSV layout — semicolon delimited, 79 columns, unquoted — deliberately, so existing Power Query workbooks can be pointed at our file without being rebuilt. That is the one place semicolons are correct: they are what the receiving spreadsheet expects.
  • Everything else is ordinary well-formed CSV, one value per column, with headings, written by csv.writer rather than assembled by hand.
  • Nothing requires a particular browser or device. The files are on a shared folder and on an HTTP endpoint; the work laptop fetches them over a network that blocks most things.

3. SQL access to the same data

Answered: it is a PostgreSQL database SYMCA owns, on SYMCA's server. No extract step, no overnight FTP, no wait for a file. The reply to the original request — that the CSVs could be dropped to an FTP overnight and imported into a SYMCA database, as WYCA's research team do — describes the arrangement this replaces.

Worth being straight about the trade: their database contains more than the Yorkshire system, including data we have no route to. This is not the same data with better access; it is different data, from the open feed, with direct access.

4. Granularity in Mosaiq

4a. A South Yorkshire view

Answered by the same stop-level rule as above. Every page and extract can be cut to the county, and "in South Yorkshire" means calls at a stop here rather than terminates here.

4b. Three different services numbered "Stagecoach 1"

Distinguishable, and TransXChange is where the answer lives. Each registered service carries its own registration in ServiceCode — for example Andrews of Tideswell's three variants share PC0003422/21, while genuinely separate services have separate registrations. We store it per journey, so operator plus registration plus direction separates services that share a line number.

The reply's point about GTFS is correct and worth keeping in view: GTFS has no depot or garage concept, so a depot split cannot be expressed in it and would need either an extension or a different source. Registration is not the same thing as depot, but it does separate the three services.

5. Punctuality

5a. The specified time bands

Answered, with nothing configured to do it. Punctuality is stored as one row per line per hour, so a band definition is a query rather than a system setting. SYMCA's five bands, South Yorkshire, weekdays 1 to 8 September 2026:

Band Timepoint observations On time
Early morning 04:00–06:59 16,053 91.8%
Morning peak 07:00–08:59 26,981 84.1%
Daytime 09:00–14:59 86,867 81.3%
Afternoon peak 15:00–17:59 40,267 74.8%
Evening 18:00–23:59 40,569 86.1%

Because the bands are a query, Mosaiq's current split — 00:00, 07:00, 09:30, 15:00, 18:00 — can be produced from the same data at the same time, which is what makes a change of definition reconcilable rather than a break in the series.

5b. Weekend and special-day treatment

Answered. Monday to Friday banded as above; Saturday and Sunday each as a single daily measure, with no further subdivision:

Days Timepoint observations On time
Mon–Fri (banded) 20 681,972 83.1%
Saturday (single measure) 4 112,298 83.7%
Sunday (single measure) 4 57,331 84.4%

Targets are not applied here yet. The measure is available at the granularity the targets need, which is the part that has to exist first.

6. Journeys counted twice, and what it does to % tracked

Not in the original list, but raised alongside it and more consequential than anything in it. The observation: where a school-day variation and its holiday counterpart both show as scheduled, there are two scheduled journeys where one bus runs. At most one can ever be tracked, so % tracked and % operated come out low for that operator through no fault of theirs. Andrews of Tideswell do not track at all so it is invisible for them; for an operator that does, it is a contract figure moving for a data reason.

That is exactly right as a mechanism, and it is measurable. Two things came out of testing it that were not expected.

TM Travel is not affected, in this copy of the data

The suspicion was that TM Travel's disagreement with the % tracked they were given might be caused by untracked journeys that should not have been scheduled. On the published data we hold, it is not:

Operator Duplicated departures Scheduled departures Share
First South Yorkshire 20 3,238 0.6%
Stagecoach Yorkshire 16 2,744 0.6%
Arriva Yorkshire 10 3,279 0.3%
TM Travel 0 384 —
High Peak, Team Pennine, Globe 0 —

Measured on 10 September 2026, a Thursday in term time. Twenty-six duplicated departures across South Yorkshire in total, which matches the sense that the overall scale is low while being concentrated enough to move one operator's figure.

School variations are one cause of it, not the main one

The three worst cases in South Yorkshire are three different faults with the same symptom:

  • First's X3, 05:00 is scheduled on a Monday-to-Thursday calendar and a Thursday-only calendar, both live on a Thursday. None of that line's 592 journeys are school-linked at all.
  • First's 18, 09:56 has two trips on the same calendar. That is a straight duplicate, with no variation involved.
  • FlixBus UK003, 08:30 has six trips on six calendars.

So a check that looks for school/holiday twins would miss most of it. Ours counts the symptom -- same route, direction and departure minute scheduled more than once on the same day -- and reports how many distinct calendars are involved, which separates a duplicate trip from an overlapping pattern without assuming the reason. It runs daily and appears on the Data Quality page.

"School day" is never abstract in TransXChange, and First proves it

The related concern: the four South Yorkshire authorities have historically had different term dates, so a variation labelled merely "school day" would be meaningless without knowing whose school year is meant.

TransXChange does not label; it names an organisation and lists its dates. First South Yorkshire publishes four, each with its own calendar:

Calendar Dates run to
Sheffield Schools 23 July 2027
Rotherham Schools 21 July 2027
Doncaster Schools 21 July 2027
Derbyshire Schools 26 July 2027

Stagecoach splits SYMCA Schools from DCC Schools; Centrebus goes further and publishes one calendar per school, twenty-one of them. We now hold 5,000 date ranges across 97 organisations, so if the four authorities diverge again the data can express it and the checks will see it. What cannot be expressed is a variation that says "school day" and leaves the dates to the reader -- and that is a property of the standard, in SYMCA's favour.

The channel that decides who has to fix it

Everything above describes the BODS copy of each operator's TransXChange. There is more than one channel, and they do not agree:

  • BODS is what we read, and what a journey planner reads.
  • TNDS, the Traveline national dataset, is a separate distribution. SYMCA itself prepares and supplies TransXChange to TNDS for several smaller operators, Andrews of Tideswell among them. If Vix takes its timetables from there, then for those operators SYMCA is upstream of its own reporting, and a fault seen in Vix may originate in a file SYMCA supplied rather than one the operator published.

That was testable rather than arguable, and it has now been tested. The TNDS copies of five services were compared against the same services as published on BODS:

File Operator Line Journeys School-noted
SVRYSFO257 Andrews of Tideswell 257 36 0
SVRYSBO257A Andrews of Tideswell 257a 6 4
SVRYSBO257B Andrews of Tideswell 257b 13 8
SVRYSGO096 Globe 96 47 10
SVRYSDO096A Globe 96a 35 2

All five are schema 2.1, revision 0, Modification="new", and were exported within three seconds of each other on 9 September 2026 — one batch from one system, covering two operators.

The school distinction is present, but as a footnote rather than a calendar

The 14:03 Wakefield–Barnsley journey on Globe's 96a, the one at the centre of the duplication, is marked in the TNDS file. Both copies of it are there and each carries a note:

Journey Pattern Note code Note text
122219_568654_5_1440 122219 SCHH School Holidays Only
122220_568654_6_1635 122220 SCHD School Days Only

That is not an isolated marker: 24 journeys across four of the five files are noted the same way, in matched SCHD/SCHH pairs. The intent was recorded, and recorded consistently.

What the files do not contain is any means of evaluating it. Across all five: zero ServicedOrganisation, zero ServicedOrganisationRef, zero ServicedOrganisationDayType. The two 14:03 journeys' OperatingProfile blocks are identical — MondayToFriday, all bank holidays excluded — and that profile is the only machine-readable statement of when a journey runs. They differ in exactly two respects: the note, and one stop. The school-day pattern calls at 370056043, which the operator's file names Darton Academy and NaPTAN names Highfields Road/Ballfield Lane; the holiday pattern does not, giving 66 stops against 65.

So a consumer reading these files correctly schedules both journeys on every weekday in term time and in the holidays alike. A reporting system that then counts two scheduled departures at 14:03 and tracks one is not misreading the file. It is reading the only part of the file that carries dates.

It is not a limitation of schema 2.1

The obvious explanation — that the older schema cannot express a school calendar — is wrong, and worth disposing of before it becomes the answer. The TransXChange 2.1 common schema defines ServicedOrganisation, ServicedOrganisationDayType, WorkingDays, DaysOfOperation and DaysOfNonOperation, and its documentation for a serviced organisation names the exact use case: "an example parent would be all the schools in a Local Education Authority; this would define the default term days (working days) and holidays for the schools in the area."

NoteCode, by contrast, is typed xsd:string with a maximum length of five and no enumeration anywhere in the standard. SCHD and SCHH are a private convention of whichever system wrote these files. No consumer is obliged to know them, and none could act on them if it did, because a note carries no dates.

Globe's own BODS publication of the same service proves the construct is available and in use: schema 2.4, revision 39, two serviced organisations, three sets of working days, one ServicedOrganisationDayType — and no notes at all. Same operator, same line, same 14:03 journey, split into a school-days variant and a holidays variant that a machine can separate.

The gap is therefore a conversion step, not missing information and not a standards limit. The term dates exist upstream of these files, or the SCHD/ SCHH split could not have been written. Turning each note into a ServicedOrganisationDayType — DaysOfOperation for SCHD, DaysOfNonOperation for SCHH, both referencing a WorkingDays organisation with the term dates — would remove the duplication at source for all 24 noted journeys.

A separate finding: the TNDS files expire in December

Every journey in all five files carries the same non-operation range:

2026-12-09 to 2099-01-01
Note: "Data Expires Three Months From Export Date"

Exported 9 September, expiring 9 December. This is deliberate and it is self-documenting, but its effect is that from 9 December 2026 every journey in these files is scheduled to not operate, indefinitely. If the export is refreshed monthly that is harmless. If it is not, these services disappear from any consumer of this channel on a fixed date, and the failure will look like the Andrews expiry already described in School day journeys — which was found the same way, and is the reason our checks now watch for it.

Who fixes what

The extract supplied alongside these files makes the scale checkable. It covers a single day, Monday 13 July 2026, a term-time weekday: 30,041 rows, 30,069 scheduled journeys, 26,391 tracked — 87.8%.

It is worth being clear that this extract is not South Yorkshire. Its 62 operator codes include First York, East Yorkshire, Arriva Yorkshire and Arriva North East, whose X93 runs between Middlesbrough and Whitby. The figures below are therefore Yorkshire-wide and beyond; South Yorkshire is a minority of them. The 26 duplicated departures counted earlier in this section are ours, measured here, and genuinely South Yorkshire. The 28 below are the supplier's, over a wider area. They are not the same measurement and should not be added.

Rows scheduled twice at the same minute 28 (0.09% of journeys)
Operators affected 9, most of them one or two rows
Globe's tracked share, as given 73.1%
Globe's tracked share, duplicates removed 74.3%

So the mechanism is real and the arithmetic works, but on this day it is worth about a single percentage point to the operator worst affected, and essentially nothing network-wide.

Sitting next to it in the same extract is something larger: 34 of the 62 operators tracked nothing at all — not a low share, zero, for every journey. Between them they account for 1,589 scheduled journeys, 5.3% of the day.

That number needs one deduction before it is used. The second largest of the 34 is SYFT, with 439 journeys — South Yorkshire Future Tram, whose lines in the extract are BLUE, PURPLE, YELLOW and TT. Supertram does not appear in a bus AVL feed and never will, so those journeys are a denominator that should not include them rather than an operator failing to report. That is 28% of the zero-tracked total accounted for before anyone investigates.

What remains is still worth asking about, and the largest single case is SPC with 463 journeys. Whether the rest are absent from the feed, reporting under a different operator code, or matched and then dropped is not something this extract can answer.

Three findings, three owners:

  • The duplication is a conversion fault in the batch that produced these files. Not the operator's published data — Globe's BODS copy is correct — and not the standard. For the operators SYMCA supplies to TNDS, it is SYMCA's own to correct, and the fix is to emit serviced organisations instead of notes.
  • A short-term dedupe is available to the reporting supplier today, without waiting for anyone to re-issue TransXChange: on a term weekday drop the SCHH journey, in the holidays drop the SCHD one. Crude, and it leans on a private note code, but it is deterministic and it recovers all 28 rows.
  • Whole operators reporting no vehicle data is the larger question, and it is not a timetable fault at all. Whether those 34 are absent from the AVL feed, present under a different operator code, or matched and then dropped is worth establishing before any % tracked figure is disputed again.

What this cannot answer, at any price

Three limits, stated plainly because they decide what is worth asking a supplier for:

  • No passenger data. No boardings, no revenue, no ticketing, no occupancy. Measured across the whole country rather than assumed: the open feed cannot express a passenger count. Everything about who is on a bus comes from the ticketing system and only from there.
  • No history before 4 August 2026, ever. The feed serves only the present moment and nothing archives it nationally. Any figure asking to compare with last year is unanswerable from this source until a year has passed. That history exists in the supplier's system and cannot be recreated in ours.
  • No raw ETM extracts. Same reason.

So the requirements worth putting to a supplier are those four things — passengers, revenue, pre-August history, and raw ETM data — rather than the reporting mechanics in sections 1, 2, 4 and 5, which are already answered.

One caveat on our own figures

Punctuality here is departure-timed at timing points; Vix appears to time arrivals, so the two will not tie exactly on the same journeys — Vix reads earlier by roughly the dwell at each stop. Neither is wrong. Any comparison between the two systems needs that stated first, or the difference gets attributed to the buses instead of to the method.

See also how a school-day journey is marked for a worked example of a fault in published data that neither system's validation objects to, and what it takes to find it.