Network Pulse

Export parser

← Network Pulse

A per-stop punctuality export arrives as semicolon-delimited text with no header row: seventy-two fields and nothing saying what any of them holds. This page reads one, puts names on the columns, and hands it back as a CSV a spreadsheet can open. Nothing is loaded into the platform — the file is read, described, and given back.

These exports run to hundreds of megabytes. The file is sent in slices rather than in one piece, so a dropped connection costs one slice rather than the lot, and it is read in the background — a million rows is a little over a minute of work, and the progress above is the real count. The CSV comes back split into parts of 1,000,000 rows, which is what a spreadsheet will open; each part carries its own header row.

The names were worked out from the values, so they are evidence rather than fact. The file does arithmetic between its own columns — a scheduled time that is a departure plus a running time, a deviation that falls inside the band beside it, a headway squared and then halved — and every one of those relationships is re-run against whatever is uploaded. If the names are right for your file, the checks below hold. If a check breaks, the header line is wrong for that file and the column table says which columns it was resting on.

Semicolon delimited, one line per stop observation, no header row. Tabs and commas are read too. Nothing here is written to the database.
Or paste a few rows

What each column is

certain the values say so, or the file's own arithmetic proves it. likely one reading fits every row and nothing else does. guess a reading that fits, offered as a starting point. unknown not identified — the column keeps its position and its number so nothing downstream shifts.

#NameWhat it holds FilledValuesExample
1 service_date Date of service, dd/mm/yyyy. certain — —
2 year Calendar year. certain — —
3 month_name Month, short name. certain — —
4 month_number Month, 1-12. certain — —
5 day_of_month Day of the month. certain — —
6 week_number Calendar week number (1 Jun 2023 is week 22). likely — —
7 period_name Calendar period, matching the month. likely — —
8 week_number_alt A second week number. Equal to column 6 in the sample, so the two schemes cannot be told apart from it. guess — —
9 period_name_alt A second period name, equal to column 7 in the sample. guess — —
10 financial_week Week of the financial year: week 9 of a year of 4-week periods starting early April. likely — —
11 financial_period Financial period, 1-13. June 2023 is period 3. likely — —
12 financial_year Financial year. likely — —
13 unknown_13 A date-derived number, 8 for 1 Jun 2023. Not identified -- every row in the sample is the same day, so nothing varies to identify it by. unknown — —
14 day_type Timetable day type: Mon to Fri, Saturday, Sunday. certain — —
15 day_name Day of the week. certain — —
16 school_period School term or school holiday. certain — —
17 scheduled_time Scheduled time at this stop, hh:mm. Equals column 37 plus column 57, and it is the time both period bandings below are worked out from. certain — —
18 daypart_start Start of the daypart column 20 names. certain — —
19 daypart_end End of that daypart. certain — —
20 daypart Daypart: Morning, AMPeak, Intermediate, PMPeak, Evening. certain — —
21 measure_band_start Start of the reporting band column 23 names. certain — —
22 measure_band_end End of that band. certain — —
23 measure_band Reporting band in words: 8am to 10.30am, and so on. certain — —
24 measure_flag Whether that band is one performance is measured in. certain — —
25 operator_code Operating company code. certain — —
26 operator Operating company. certain — —
27 depot_code Depot code. certain — —
28 depot Depot. certain — —
29 stop_code Stop code as the operator holds it (8 digits, not a 12-character NaPTAN ATCO code). certain — —
30 stop_name Stop name. certain — —
31 stop_name_alt Stop name again -- a second name field, identical throughout the sample. likely — —
32 stop_label Stop name with its code in brackets. certain — —
33 unknown_33 Zero in every row. Not identified. unknown — —
34 locality Town or locality the stop is in. certain — —
35 running_board Running board (duty). Fixes the vehicle: each board carries one value of columns 41 and 42 all day. likely — —
36 direction Inbound or Outbound. certain — —
37 journey_start Scheduled departure of the journey from its first stop, hh:mm. certain — —
38 line Service number. likely — —
39 journey_number Journey number. Rises with the departure time through the day. likely — —
40 frequency_class Frequent or non-frequent service. certain — —
41 vehicle Fleet number. Constant for a running board across the day. likely — —
42 vehicle_ref A second vehicle identifier, one-to-one with column 41 -- an internal asset or ticket-machine number. guess — —
43 driver An identifier that changes within a running board and repeats across boards: it travels with the driver, not the bus. Treat it as personal data. guess — —
44 timing_point Timing point or not. certain — —
45 stop_role Where the stop sits on the route: origin, intermediate, destination. certain — —
46 stop_group The reporting group that role falls in. certain — —
47 stop_sequence Position of this stop along the journey. likely — —
48 report_group Which set of measures the row belongs to. 'Exceptions' throughout the sample. guess — —
49 punctuality_band Early, On Time or Late. certain — —
50 band_min_minutes Lower bound of that band, minutes. On Time runs -1 to 5.99. certain — —
51 band_max_minutes Upper bound of that band, minutes. certain — —
52 journey_status Whether the journey ran. certain — —
53 stop_status The same status at stop level. likely — —
54 scheduled_time_full Scheduled time at this stop with seconds. Column 17 again. certain — —
55 unused_55 Empty in every row of the sample. unknown — —
56 schedule_deviation_minutes Deviation at this stop in minutes, positive late. The number to measure from. certain — —
57 scheduled_run_time_minutes Scheduled running time from the first stop to this one. certain — —
58 actual_run_time_minutes Actual running time over the same stretch. certain — —
59 unknown_59_minutes A short duration, 17 to 40 seconds in the sample. Dwell at the stop would fit; nothing in the file proves it. guess — —
60 unknown_60_minutes A second short duration, sometimes zero. guess — —
61 unused_61 Empty in every row of the sample. unknown — —
62 headway_minutes Gap in minutes to the bus before this one at this stop. likely — —
63 headway_squared Column 62 squared. certain — —
64 headway_squared_halved Column 63 halved -- the excess-waiting-time term. certain — —
65 unused_65 Empty in every row of the sample. unknown — —
66 unused_66 Empty in every row of the sample. unknown — —
67 difference_band A banding of a difference: '<15 min difference' throughout the sample. guess — —
68 deviation_words The deviation in whole minutes, in words. Truncated, not rounded. certain — —
69 unknown_69_count A count. Never smaller than column 70 in any row of the sample. unknown — —
70 unknown_70_count A second count, never larger than column 69. unknown — —
71 unused_71 Empty in every row of the sample. unknown — —
72 unused_72 Empty in every row of the sample. unknown — —