Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce RealtimeUpdateContext #6029

Merged

Conversation

vpaturet
Copy link
Contributor

@vpaturet vpaturet commented Aug 28, 2024

Summary

Real-time updaters use instances of TransitService to look up transit data. Currently these instances are created at construction time.
This is problematic since TransitService instances maintain a reference to a frozen version of the TimetableSnapshot that is not updated after construction.
As of today this does not cause any issue, since the real-time updaters update directly the collections in TransitModelIndex, and the TransitService looks up data in TransitModelIndex.

However #6007 changes this behavior by storing the real-time updates in indexes in TimetableSnapshot, making previous real-time updates invisible from the real-time updaters.
Among other regressions, this prevents the EntityResolver from identifying already created extra journeys, and leads to the creation of duplicated trips.

This PR refactors the real-time updaters so that they use TransitService instances that refer to the TimetableSnapshot buffer, thus giving access to all applied real-time updates (including uncommitted ones).

This PR is a prerequisite for #6007
This PR supersedes #6018

Note: the refactoring consists in:

  • Modifying the GraphWriterRunnable interface so that it accepts a RealTimeUpdateContext as a parameter.
    RealTimeUpdateContext gives access to a transit service that points to the TimetableSnapshot buffer. Other entities exposed in this interface (EntityResolver, SiriFuzzyTripMatcher, GtfsRealtimeFuzzyTripMatcher) are also aware of the latest applied real-time update.
  • Refactoring the real-time updaters in order to make use of the instances of TransitService, EntityResolver, SiriFuzzyTripMatcher, GtfsRealtimeFuzzyTripMatcher provided by the RealTimeUpdateContext, exclusively in the context of a GraphWriterRunnable (where it is safe to do so).

Note: the SiriFuzzyTripMatcher is now instantiated only once in the RealTimeUpdateContext, making it possible to remove the static singleton logic that made it difficult to test.

Note: The SIRI-SX and Alert updaters still make use of a TransitModel instance. In practice they just need access to the StopModel, which is (currently) immutable. These updaters could be refactored in a follow-up PR to use only the StopModel instead of the TransitModel, making the intent clearer.

Issue

No

Unit tests

Added unit tests,
Updated unit tests.

Documentation

No

@vpaturet vpaturet added Technical Debt Real-Time Update The issue/PR is related to RealTime updates Skip Changelog labels Aug 28, 2024
@vpaturet vpaturet self-assigned this Aug 28, 2024
@vpaturet vpaturet force-pushed the introduce_real_time_update_context branch from a248d23 to b5afac1 Compare August 28, 2024 14:15
Copy link

codecov bot commented Aug 28, 2024

Codecov Report

Attention: Patch coverage is 36.79245% with 67 lines in your changes missing coverage. Please review.

Project coverage is 69.76%. Comparing base (99998b8) to head (0789d24).
Report is 60 commits behind head on dev-2.x.

Files with missing lines Patch % Lines
...hicle_position/VehiclePositionUpdaterRunnable.java 0.00% 12 Missing ⚠️
...planner/updater/configure/UpdaterConfigurator.java 16.66% 8 Missing and 2 partials ⚠️
...opentripplanner/ext/siri/SiriFuzzyTripMatcher.java 61.11% 4 Missing and 3 partials ⚠️
...ehicle_position/PollingVehiclePositionUpdater.java 0.00% 5 Missing ⚠️
...ner/ext/siri/updater/azure/SiriAzureETUpdater.java 0.00% 4 Missing ⚠️
...pentripplanner/ext/siri/updater/SiriETUpdater.java 0.00% 3 Missing ⚠️
...pentripplanner/ext/siri/updater/SiriSXUpdater.java 0.00% 3 Missing ⚠️
...t/siri/updater/azure/AbstractAzureSiriUpdater.java 0.00% 3 Missing ⚠️
...ner/ext/siri/updater/azure/SiriAzureSXUpdater.java 0.00% 3 Missing ⚠️
...anner/updater/alert/GtfsRealtimeAlertsUpdater.java 0.00% 3 Missing ⚠️
... and 9 more
Additional details and impacted files
@@              Coverage Diff              @@
##             dev-2.x    #6029      +/-   ##
=============================================
+ Coverage      69.73%   69.76%   +0.03%     
- Complexity     17315    17327      +12     
=============================================
  Files           1960     1961       +1     
  Lines          74267    74267              
  Branches        7603     7603              
=============================================
+ Hits           51789    51812      +23     
+ Misses         19834    19813      -21     
+ Partials        2644     2642       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@vpaturet vpaturet force-pushed the introduce_real_time_update_context branch 2 times, most recently from 11a0967 to 03c9c23 Compare August 28, 2024 14:28
@vpaturet vpaturet force-pushed the introduce_real_time_update_context branch from 03c9c23 to e693e50 Compare August 28, 2024 14:43
@vpaturet vpaturet marked this pull request as ready for review August 28, 2024 14:49
@vpaturet vpaturet requested a review from a team as a code owner August 28, 2024 14:49
@vpaturet vpaturet requested review from t2gran and abyrd August 28, 2024 14:49
@vpaturet vpaturet added the Entur Test This is currently being tested at Entur label Aug 29, 2024
abyrd
abyrd previously approved these changes Sep 3, 2024
Copy link
Member

@abyrd abyrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the very clear PR description and Javadoc on the interface. This looks like a good step in the right direction.

If I am seeing correctly, the same DefaultRealTimeUpdateContext instance is used for all successive updates, across all successive snapshots (a new instance is not made for each snapshot). This instance is in some sense the outer unifying layer of the persistent buffer from which we take snapshots. The lifecycle (specifically the long duration) of these instances might be worth mentioning in the Javadoc of the implementation classes.

@abyrd
Copy link
Member

abyrd commented Sep 3, 2024

For future reference: As discussed in a recent meeting, it's not clear that in the long term we actually want the updaters to consult changes introduced by previously applied update messages, including changes introduced within the same batch of updates between two snapshot publications. We may want updates to be idempotent and interpreted independent of one another, to the greatest extent possible. For example, the choice between creating a realtime-added trip or pattern and reusing an already created one could be encapsulated within a method exposed to the updaters by a deeper layer of the system, so the updaters don't need to know whether another update already created that trip or pattern. But the current system relies on updaters performing their own lookups to function, and these PRs serve to clean up that existing system in advance of any more extensive overhaul to the design.

t2gran
t2gran previously approved these changes Sep 10, 2024
Copy link
Member

@t2gran t2gran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor formatting of a comment...

…otSource.java

Co-authored-by: Thomas Gran <t2gran@gmail.com>
@vpaturet vpaturet dismissed stale reviews from t2gran and abyrd via 0789d24 September 10, 2024 13:28
@t2gran
Copy link
Member

t2gran commented Sep 10, 2024

@abyrd already approved this and the only change after that was a small change to a JavaDoc format, so I will merge this without waiting for a formal last approval from @abyrd.

@t2gran t2gran merged commit 5ba8d94 into opentripplanner:dev-2.x Sep 10, 2024
5 checks passed
@t2gran t2gran deleted the introduce_real_time_update_context branch September 10, 2024 17:33
@t2gran t2gran added this to the 2.6 milestone Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Entur Test This is currently being tested at Entur Real-Time Update The issue/PR is related to RealTime updates Skip Changelog Technical Debt
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants