Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update event types to be discreet #321

Merged
merged 2 commits into from
Jun 28, 2023
Merged

Conversation

kylehuntsman
Copy link
Contributor

@kylehuntsman kylehuntsman commented Jun 16, 2023

In an effort to cut down on the unneccessary bloat some of our events started to have, this update removes the idea of event phases in favor of making the events discreet from each other. Events should not depend on each other and should not require data that is not relative to them.

Notable Changes

  • Phase and phase start time have been removed from all events
  • The started event has been extracted into three separate events singalling:
    1. the start of the entire fetch
    2. the start of the finding candidates
    3. the start of a retrieval from a specific provider
      Note: fetch in this case is the beginning of Lassie.Fetch() and retrieval is when we start talking to a specific provider.
  • The failed event has been extracted into two separate events:
    1. general fails
    2. failing a retrieval
  • A number of interfaces have been made for determining if an event has a given property, the most prominent one being if the event has an SP ID with EventWithSPID. Checks have been added in places to ensure the event has a given property before using.
  • The events have been extracted into their own files due to the one events file becoming very large and hard to parse

I wonder whether we even want RetrievalEvent.PayloadCid() -- does every event actually have a real payload CID? If not, let's delete it. @hannahhoward

I attempted to extract payloadCid out of the base event, but got caught on some assignable candidate finder tests that use CID keyed maps to assert that specific candidates were found for a given CID. The entire test set would have needed to be rewritten. I've made those changes to the feat/remove-cid-from-events branch if we wanted to move forward with this work. I'm going to leave the payloadCid in the base event for this PR.

Copy link
Collaborator

@hannahhoward hannahhoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the broadest sense, loooking good -- direction is strong.

The only suggestion is to switch to a type switch in aggregateeventrecorder.

I'm also not convinced about payloadcid, unless we really have a value everywhere.

Otherwise, I like these changes a lot.

cmd/lassie/fetch.go Outdated Show resolved Hide resolved
pkg/types/types.go Show resolved Hide resolved
@kylehuntsman kylehuntsman force-pushed the feat/discreet-events branch 4 times, most recently from 441d39b to 79307ac Compare June 21, 2023 22:00
@kylehuntsman kylehuntsman marked this pull request as ready for review June 21, 2023 22:05
@codecov-commenter
Copy link

codecov-commenter commented Jun 21, 2023

Codecov Report

Merging #321 (299289d) into main (da0ac6b) will increase coverage by 0.94%.
The diff coverage is 91.89%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #321      +/-   ##
==========================================
+ Coverage   75.65%   76.60%   +0.94%     
==========================================
  Files          70       85      +15     
  Lines        6310     6403      +93     
==========================================
+ Hits         4774     4905     +131     
+ Misses       1262     1226      -36     
+ Partials      274      272       -2     
Impacted Files Coverage Δ
cmd/lassie/fetch.go 32.71% <0.00%> (+1.90%) ⬆️
pkg/types/types.go 82.89% <ø> (-0.64%) ⬇️
...kg/internal/testutil/collectingeventlsubscriber.go 58.41% <88.00%> (+7.01%) ⬆️
pkg/events/candidatesfiltered.go 88.88% <88.88%> (ø)
pkg/retriever/parallelpeerretriever.go 95.28% <94.11%> (-0.05%) ⬇️
...g/aggregateeventrecorder/aggregateeventrecorder.go 89.59% <100.00%> (-0.25%) ⬇️
pkg/events/base.go 100.00% <100.00%> (ø)
pkg/events/candidatesfound.go 100.00% <100.00%> (ø)
pkg/events/connectedtoprovider.go 100.00% <100.00%> (ø)
pkg/events/events.go 100.00% <100.00%> (+2.04%) ⬆️
... and 17 more

... and 7 files with indirect coverage changes

@kylehuntsman kylehuntsman force-pushed the feat/discreet-events branch 3 times, most recently from cdc0bf2 to 48d4d43 Compare June 21, 2023 22:37
Copy link
Collaborator

@hannahhoward hannahhoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only concerning thing to me is we appear to be qualitatively changing the behavior for server timings in the IPFS handler and I want to make sure @willscott is ok with that.

Otherwise, I'm good. A few small nits. Otherwise, feel free to merge tomorrow once @rvagg has reviewed.

pkg/events/candidatesfiltered.go Show resolved Hide resolved
pkg/retriever/parallelpeerretriever.go Outdated Show resolved Hide resolved
pkg/retriever/parallelpeerretriever.go Show resolved Hide resolved
pkg/server/http/ipfs.go Show resolved Hide resolved
pkg/server/http/ipfs.go Outdated Show resolved Hide resolved
@hannahhoward
Copy link
Collaborator

Hi! I'm still holding @willscott 's review of the changes to serving timings cause those are used elsewhere in the stack. I've ping him on slack for visibility.

pkg/types/types.go Outdated Show resolved Hide resolved
pkg/events/base.go Outdated Show resolved Hide resolved
@kylehuntsman
Copy link
Contributor Author

Everything has been resolved or moved to another issue. Waiting on PR #332 before merging this.

In an effort to cut down on the unneccessary bloat some of our events started to have, this update removes the idea of event phases in favor of making the events discreet from each other. Events should not depend on each other and should not require data that is not relative to them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants