Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Paraswap Ethereum trades #6581

Merged
merged 51 commits into from
Oct 21, 2024

Conversation

PablloZz
Copy link
Contributor

@PablloZz PablloZz commented Aug 20, 2024

Thank you for contributing to Spellbook 🪄

Update!

Please build spells in the proper subproject directory. For more information, please see the main readme, which also links to a GH discussion with the option to ask questions.

Contribution type

Please check the type of contribution this pull request is for:

  • New spell(s)
  • Adding to existing spell lineage
  • Bug fix

For bug fixes

If you are fixing a bug, please provide the following information:

  • Description:
  1. Transaction Data Source Update:
    Previously, data for calls, i.e. values of tokens, who called, received, etc., were taken from the beginning and the end of a transaction. I modified it so that the data is now taken from the actual caller of the Augustus contract.

  2. Fix for Filtering ETH Transactions:
    For all calls from the ethereum.transactions table, the is_eth value was taken based on whether the value was sent in the call. In swap_detail_out, there were two filters with is_eth = false. This meant that all transactions that sent ETH were filtered out, even if the Augustus call itself didn't involve ETH. This bug caused the most missed transactions. The check was removed and the code was refactored without it.
    Examples: 1, 2, 3, 4, 5.

  3. Use of union all for Grouping Calls:
    Previously, calls were grouped using union, and only one call was selected for transactions with multiple calls. This was replaced with union all for accuracy and performance.

  4. CTE Restructure for Better Data Extraction:
    To more accurately join the events and traces tables and extract all the calls in one transaction, one CTE was split into three (for UniswapV2, Uniswap, and ZeroX).

  5. Remove join with the traces table for UniswapV2 and Uniswap:
    For these calls, I used only the events table and removed traces, which allowed handling of positive slippage and performing an inner join.

  6. Handling Identical Calls:
    Some transactions had identical calls except for the trace_address, like this one, I handled this by using swap_in_row_number and swap_out_row_number for the call, and evt_row_number for the event.

  7. Duplicate Filtering:
    To avoid possible duplicates, I compared the number of initial calls in a transaction with the final rows and filtered out any mismatches.

Result:
The total number of calls increased by 350,807, transactions by 345,737, and the total USD volume increased by $1,073,360,636.

Here is a table for more detailed info:
Statistics.xlsx

  1. Fantom Trades Update:
    Additional events tables were added.

Thank you for your contribution!

PablloZz and others added 11 commits August 1, 2024 15:10
Split tables for trades by version and project to get more precise
trade data using data specific for each table;

Replace inner join to left join and remove is_eth condition: for
ETH/ERC20 is_eth for token_in was true and for token_out was false.
Later inner join just filtered such transactions. Also token_out has
had two is_eth = false conditions which filtered all transactions that
included ETH.
@PablloZz PablloZz marked this pull request as draft August 20, 2024 09:49
@dune-eng
Copy link

Workflow run id 10469285857 approved.

@dune-eng
Copy link

Workflow run id 10469286205 approved.

@dune-eng
Copy link

Workflow run id 10469318035 approved.

@dune-eng
Copy link

Workflow run id 10469318314 approved.

@jeff-dude jeff-dude added the WIP work in progress label Aug 20, 2024
@dune-eng
Copy link

Workflow run id 10529589658 approved.

@dune-eng
Copy link

Workflow run id 10529589305 approved.

@dune-eng
Copy link

Workflow run id 10529607018 approved.

@dune-eng
Copy link

Workflow run id 10529607335 approved.

@dune-eng
Copy link

Workflow run id 10560412413 approved.

@dune-eng
Copy link

Workflow run id 10560412206 approved.

Add checking for TokenTransferProxy because "event from" is not always
Augustus caller
@PablloZz PablloZz force-pushed the fix-paraswap-ethereum-trades branch from b4f6300 to e2da218 Compare August 26, 2024 15:27
@dune-eng
Copy link

Workflow run id 10562598893 approved.

@dune-eng
Copy link

Workflow run id 10562599195 approved.

@dune-eng
Copy link

dune-eng commented Oct 4, 2024

Workflow run id 11180626853 approved.

@dune-eng
Copy link

dune-eng commented Oct 4, 2024

Workflow run id 11180627049 approved.

@dune-eng
Copy link

dune-eng commented Oct 4, 2024

Workflow run id 11183084720 approved.

@dune-eng
Copy link

dune-eng commented Oct 4, 2024

Workflow run id 11183084822 approved.

@dune-eng
Copy link

dune-eng commented Oct 4, 2024

Workflow run id 11183102092 approved.

@dune-eng
Copy link

dune-eng commented Oct 4, 2024

Workflow run id 11183102233 approved.

@PablloZz PablloZz marked this pull request as ready for review October 15, 2024 07:24
@0xRobin 0xRobin added ready-for-review this PR development is complete, please review and removed WIP work in progress labels Oct 15, 2024
@jeff-dude jeff-dude requested a review from Hosuke October 16, 2024 19:05
@jeff-dude jeff-dude added the dbt: dex covers the DEX dbt subproject label Oct 16, 2024
Copy link
Collaborator

@Hosuke Hosuke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall lgtm.✅
Thank you @PablloZz

c.swap_in_pair_token_in,
c.swap_out_pair_token_out,
t."from" AS caller,
concat(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part seems a bit complated.
Do you want to use || syntax to concat string?

Copy link
Collaborator

@Hosuke Hosuke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

Updated trades:
v5-fantom: https://dune.com/queries/4176714
v5-ethereum: https://dune.com/queries/4176725

@Hosuke Hosuke added ready-for-final-review and removed ready-for-review this PR development is complete, please review labels Oct 18, 2024
@Hosuke Hosuke assigned jeff-dude and unassigned Hosuke Oct 18, 2024
Copy link
Member

@jeff-dude jeff-dude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@jeff-dude jeff-dude merged commit f29e3e6 into duneanalytics:main Oct 21, 2024
2 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
dbt: dex covers the DEX dbt subproject ready-for-merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants