Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dedup fork requests #1465

Open
kariy opened this issue Jan 20, 2024 · 7 comments · May be fixed by #2001
Open

Dedup fork requests #1465

kariy opened this issue Jan 20, 2024 · 7 comments · May be fixed by #2001
Assignees
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed katana This issue is related to Katana

Comments

@kariy
Copy link
Member

kariy commented Jan 20, 2024

When running katana in forking mode, there's a potential inefficiency that occurs when the forked backend processes identical requests simultaneously. This leads to unnecessary duplication in data fetching, as the same data is retrieved multiple times. To optimize this, we should implement a mechanism to deduplicate incoming requests, allowing the reuse of data from the first identical request.

Example scenario for context:

Consider a situation where a specific storage slot is not present in the local storage. katana then attempts to fetch it from the forked network. If, while this request is still processing, another request for the same storage slot is made, katana will again try to fetch from the forked provider. As a result, two identical requests are sent to the forked network, which is inefficient.

Proposed solution:

Implement a request deduplication process. When a request is made, katana should first check if an identical request is already in progress. If so, it should wait for the initial request to complete and then use that data, rather than initiating a new request to the forked network.

We can use the requests params (eg. contract address, storage key) as the request id, and perform the dedup process on an incoming request that has identical id.

@kariy kariy added enhancement New feature or request help wanted Extra attention is needed katana This issue is related to Katana good first issue Good for newcomers labels Jan 20, 2024
@rkdud007
Copy link
Contributor

rkdud007 commented Feb 6, 2024

can i take this?

@kariy
Copy link
Member Author

kariy commented Feb 6, 2024

Assigned. Let me know if you have questions or need more context on this issue.

@btirth
Copy link
Contributor

btirth commented Apr 13, 2024

@rkdud007: If you haven't started working on this, can I take this one?

FYI: @kariy

@rkdud007
Copy link
Contributor

Hey, sure you can pick up this issue 😄

@glihm
Copy link
Collaborator

glihm commented Apr 14, 2024

@rkdud007: If you haven't started working on this, can I take this one?

FYI: @kariy

@Tirth1 if you're ok with tackling this and #1409 just assigned to you, let's do it! Let us know if we should also assign this one to you. 👍

@cwkang1998
Copy link

Interested in taking this if it's not being worked on

@kariy
Copy link
Member Author

kariy commented Apr 21, 2024

@btirth im gonna be assigning this to @cwkang1998 considering you already have an issue assigned to you.

@glihm glihm added the ODHack OnlyDust Hack fest label Apr 22, 2024
@cwkang1998 cwkang1998 linked a pull request May 25, 2024 that will close this issue
10 tasks
@glihm glihm removed the ODHack OnlyDust Hack fest label Jun 19, 2024
@glihm glihm added this to the 1.0.0 milestone Jun 19, 2024
@glihm glihm changed the title [Katana] Dedup fork requests Dedup fork requests Jun 19, 2024
@glihm glihm removed this from the 1.0.0 milestone Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed katana This issue is related to Katana
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

5 participants