Skip to content

Admin

Jump to bottom Edit New page
sheldonabrown edited this page Mar 15, 2013 · 9 revisions

The Admin component provides web interfaces for performing administrative tasks such as transit bundle building, vehicle status tracking, user management etc. The application can be accessed only by "admin" users having valid credentials in the system.

Design

The admin component exists in a single module onebusaway-nyc-admin-webapp. It contains classes and services for various admin utilities such as User Management, Vehicle Status Monitoring, Bundle Building etc.

Below are some of the key classes of the onebusaway-nyc-admin-webapp module:

  • BarcodeService - Gets QR Code images from the remote server. Used by QR Code generation utility.
  • BundleRequestService - Provides methods related to bundle building such as validating a bundle, building a bundle etc. Makes request to a remote web resource to validate and build a bundle. Used by bundle building utility.
  • EmailService - Sends email to configured email address after bundle building process is complete. Used by bundle building utility.
  • ParametersService - Saves configuration parameters on TDM server. Retrieves configuration parameters to display on Configuration Parameters application. Used by Configuration Parameters utility.
  • BarcodeService - Gets QR Code images from the remote server. Used by QR Code generation utility.
  • UserManagementService - Performs read, write and update operations on users of the system. Used by User Management utility.
  • VehicleStatusService - Builds vehicle status data by retrieving vehicle pullout data from TDM and last known location data from archiver. Used by Vehicle Status Display utility.
  • BarcodeResource - Server resource used by QR code generation UI to generate QR codes.
  • ConfigResource - Server resource used by configuration parameters UI to persist and retrieve configuration parameters.
  • UserResource - Server resource used by user management UI to get user details.
  • BundleBuildingService - Provides methods to download and build bundles from Amazon S3.
  • BundleValidationService - Provides methods to verify bundle before it is built.
  • BuildResource - Server resource used by bundle building UI to request bundle building.
  • BundleResource - Server resource used by bundle building UI to request bundle deployment on the TDM.
  • ValidateResource - Server resource used by bundle building UI to request bundle validation before building.
  • BuildRemoteResource - Server resource used by bundle building UI to request bundle building. This is a remote resource which is instantiated when remote AWS instance is fired up. Local bundle resource makes a build request to this remote resource for cost savings.
  • PingRemoteResource - Ping endpoint on remote server. Confirms a host is available.
  • ValidateRemoteResource - Server resource used by bundle building UI to request bundle validation. This is a remote resource which is instantiated when remote AWS instance is fired up. Local bundle resource makes a validate request to this remote resource for cost savings.
  • ModifyStartDateTask - Bundle building task run to manipulate calendar service dates to ease bundle change transitions. See Bundle Change Strategy below for more details.

Transit Data Bundle Utility: S3 Bucket Layout

The Transit Data Bundle Utility uses S3 as a file server, and expects the following conventions to be observed:

/ - Root of the bucket
    <pick_name>/
        gtfs_latest/
            google_acta.zip - GTFS for A City Transit Agency
            GTFS_another_cta.zip - GTFS for another City Transit Agency
        stif_latest/ - (optional: see note below)
            stifs_acta.zip - STIF files for A City Transit Agency
        config/
            tripToDSCMap.txt - the file assigning DSCs to subway shuttle runs, which have trips already (optional)
        builds/
            <pick_name>_r<data_release>_b<build_num>/ 
                <pick_name>_r<data_release>_b<build_num>.tar.gz - Built and packaged bundles placed here
                outputs/
                    All output logs and CSV statistics will be placed at this level.
                inputs/ - The input data as downloaded as the first step in the bundle building process, archived for posterity

A word on STIF files: STIF is a proprietary file format used by the MTA that contains additional schedule information not present in the GTFS. It can be safely ignored.

Bundle Change Strategy

Issue

Bundle changes (schedule data) occur at midnight of a pick change, but do not flawlessly handle buses that are still on the road making service from the previous pick (i.e. buses that pulled out on Saturday but are still on the road when the clock ticks over to Sunday).

Strategy

The solution to this is to treat those buses according to the Saturday-into-Sunday schedules defined by the new pick. ModifyStartDateTask adjusts the start date of relevant service ids in the GTFS calendar.txt to make the previous days' trips applicable.

Here is the logic in ModifyStartDateTask:

Leave all entries in calendar.txt alone, except where all of the following are true:

saturday = 1
AND
monday, tuesday, wednesday, thursday, friday, sunday all = 0,
AND
start_date != end_date

for those calendar.txt entries that match all of those conditions:

start_date = start_date - 15 days

Implications

The implications of this are as follows. For convenience of exposition, let us refer to buses that are servicing trips/runs from the previous pick even after the new pick has started as “hangover” buses/trips.

  1. When neither run number nor DSC’s have changed for those hangover trips from one pick to the next, the system will perform as it does during a pick. 2.When run numbers for those hangover trips have changed from one pick to the next but the DSC’s have not, the inference for those buses will be “informal”. The implications of this are:

    We won’t calculate or report schedule deviation Once the UI’s are telling people about buses that are still inbound to the terminal (i.e. going the opposite direction), which they currently do not, that information would be hidden until the bus reaches the terminal.

  2. If the run number and DSC’s for those hangover trips have changed from one pick to the next, we will fail to infer anything about it, and the buses on those hangover trips will be considered not in service.

  3. If the route for those hangover trips does not exist in the new pick or does not have the late night Saturday service in the new pick, we will fail to infer anything about it, and the buses on those hangover trips will be considered not in service.

Again, the above apply only to that first early Sunday morning when the new pick has started (i.e. 4 times/year). There are definitely a meaningful amount of hangover trips, but the large majority will fall under condition (1) and (2) above, which means that the system have degraded but acceptable performance from the customer and internal management perspective.

Transit Data Bundle Utility: Output Files

Summary Output File

Filename: summary.txt

Contents:

Number of lines for each file below, except the header line
    For DSC Statistics file, number of lines with distinct route_ids greater than one.
Task Status: success or failure

Example:

Task Status
===========
Trips With Null DSCs: OK
GTFS trips not matched to STIF: OK
STIF trips not matched to GTFS: OK
STIF trip layer entries with no DSC or no Sign Code Route: FAIL
DSC Statistics: OK
STIF trips without pullout: OK
Non-pullin stif movement with no next movement/trip "cliffhangers": OK
Two trips on same run with same start time: OK
Route IDs (from GTFS routes.txt) with no trips: OK
STIF Task log: OK

Status
======
Trips With Null DSCs: 0
Total DSCs: 45
DSCs with more than one route ID: 0
GTFS trips not matched to STIF: 0
STIF trips not matched to GTFS: 0
STIF trip layer entries with no DSC or no Sign Code Route: 0
STIF trips without pullout: 0
Non-pullin stif movement with no next movement/trip "cliffhangers": 0
Two trips on same run with same start time: 0
Route IDs (from GTFS routes.txt) with no trips: 0

Trips with null DSCs

Filename: trips_with_null_dscs.csv

Columns:

Trip ID From GTFS
ID info for trip layer in STIF

Example:

gtfs_trip_id, stif_trip, stif_filename, stif_trip_record_line_num
MTA NYCT_20120108EE_079800_S78_0379_S7484_705, "TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22

DSC Statistics

Filename: dsc_statistics.csv

Columns:

Number trips in STIF with DSC
Number distinct Route IDs in GTFS trips associated with DSC

Example:

dsc, number_of_trips_in_stif, number_of_distinct_route_ids_in_gtfs
4630, 44561, 1
4631, 44561, 1

GTFS trips not matched to STIF

Filename: gtfs_trips_with_no_stif_match.csv

Columns:

GTFS Trip id
What tuple was searched for in STIF?

Example:

gtfs_trip_id, stif_trip
MTA NYCT_20120108EE_079800_S78_0379_S7484_705, "TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)"

STIF trips not matched to GTFS

Filename: stif_trips_with_no_gtfs_match.csv

Columns:

ID info for trip layer in STIF 
What id/tuple was searched for in GTFS

Example:

stif_trip, stif_filename, stif_trip_record_line_num
"TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22

STIF trip layer entries with no DSC or no Sign Code Route

Filename: stif_trip_layers_with_missing_route.csv

Columns:

ID info for trip layer in STIF 
What is missing - DSC or sign code route

Example:

stif_trip, stif_filename, stif_trip_record_line_num, missing_field
"TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22, DSC
"TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22, SIGN CODE ROUTE

STIF trips without pullout

Filename: stif_trips_without_pullout.csv

Columns:

ID info for trip layer in STIF 
corresponding GTFS trip id
synthesized block id (note example synthesized block ID below is not necessarily representative of what we're actually using.

Example:

stif_trip, stif_filename, stif_trip_record_line_num, gtfs_trip_id, synthesized_block_id
"TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22, MTA NYCT_20120108EE_079800_S78_0379_S7484_705, MTA NYCT_20110904EE_01_333333_MISC_3333_orphan

Route IDs (from GTFS routes.txt) with no trips

Filename: route_ids_with_no_trips.csv

Columns:

Agency ID
Route ID

Example:

agency_id, route_id
MTA NYCT, B63

Two trips on same run with same start time

Filename: trips_with_duplicate_run_and_start_time.csv

Columns:

(for both trips) ID info for trip layer in STIF

Example:

stif_trip1, stif_filename1, stif_trip_record_line_num1, stif_trip2, stif_filename2, stif_trip_record_line_num2
"TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22, "TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22

Non-pullin STIF movement with no next movement/trip--"cliffhangers"

Filename: non_pullin_without_next_movement.csv

Columns:

ID info for trip layer in STIF

Example:

stif_trip, stif_filename, stif_trip_record_line_num
"TripIdentifier(MISC,05:15:00 or 31500,06:39:00 or 39900,905019,MISC-419)", non_holiday/stif.m_0001__.212001.sat, 22

GTFS-STIF Matched Trips Log

Filename: matched_trips_gtfs_stif.csv

Columns:

agency_id
service ID
block ID
trip ID
DSC
first stop ID
first stop time
last stop ID
last stop time
run ID (run route - run number)
relief run ID (run route - run number)
recovery time
trip is last in block?
trip is first in block?
sign code route from STIF
route ID from GTFS

Example:

blockId,tripId,dsc,firstStop,firstStopTime,lastStop,lastStopTime,runId,reliefRunId,recoveryTime,firstInSeq,lastInSeq,signCodeRoute,gtfsRouteId
MTA NYCT_20120408CC_CH_53700_X0109-712_1,20120408CC_091500_X17_0312_X0109_712,7170,202910,54900,904045,62400,X0109-712,null,13,false,false,X17C,X17C

Configurable Values

Onebusaway-NYC system provides various configuration points that can be set with custom values. These configuration points exist in external configuration files, application contexts as well as Java code. The [Configurable Values](https://github.com/camsys/onebusaway-nyc/wiki/Configurable Values) provides a list of configuration values along with a brief description about them.

Add a custom footer
Add a custom sidebar
Clone this wiki locally