Sync helius branch to main branch #127
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix GetAssetsByCreator query
Handle and log http errors separately.
Partially addresses metaplex-foundation#50.
Enable configurable retries for the bgtask.
* acc forwarder * version to .0.1
Cleaning out unused imports.
Account forwarder
fix(das-api): fix getAssetProof
Handle HTTP errors
Fixing unused imports
Configurable bgtask runner
* add dto parameter * add dao condition * upd asset_data fetching to ensure single presence of asset_data in query * use asset.asset_data key for linking between asset/asset_data * fix: breaking arg order for array queries * chore: consistent arg order in `SearchAssets` destructuring * fix: breaking arg count for array params in searchAssets
* Hotfix for asset panic * Fixes some minor details * Update digital_asset_types/src/dapi/common/asset.rs Co-authored-by: Kirill Fomichev <fanatid@ya.ru> * Adding needed parameter. * Updating to kirill suggested change. * Hotfixes for files bugs Hotfixes to attempt to address some files bugs. * Remove commented code Clean out commented code. --------- Co-authored-by: Kirill Fomichev <fanatid@ya.ru> Co-authored-by: juanito87 <juancruz.f87@gmail.com>
* REname tests to tools Rename tests to tools for better clarity. * Fix cargo
This moves the queues the acc forwarder and txn forwarder uses to be backfill queues. This allows us to backfill without disturbing ingestion of new data coming in. This can be quite important considering the need for latest data for proofs to work. It aligns well with the changes introduced via out of order processing allowing us to introduce past history while still maintaining proof status. We have been running this in our infrastructure: It allows us to ingest a tree with 1mn/2mn transactions (which can take a little time) while staying on the tip with all other trees because they aren ot competing for queue space. This means that the "on tip queue" never has more than a few seconds worth of transactions in it, while the backfill queue can have a lot more and can be used to fill in trees that have been missed in ingestion.
Fetches a list of trees from an RPC endpoint.
* Updating test workflow * Adding some annotations * Fixing path issue. * Fixing fetch_trees name * Adding push condition * Main (#104) * Fix raw name build (#122) * Added fetch trees tool (#123) Fetches a list of trees from an RPC endpoint. --------- Co-authored-by: Linus Kendall <git@linuskendall.com> --------- Co-authored-by: Linus Kendall <git@linuskendall.com>
* Apply cl audits patch * Make cl audits configurable * Fixing missing variable. --------- Co-authored-by: juanito87 <juancruz.f87@gmail.com>
* Change path for load generation * Update to Solana 1.16 versions * Update to Solana 1.16.16 (version that does not have tokio pinned and solana-transaction-status supports Borsh 0.10.3). * Update blockbuster to latest branch that supports Bubblegum Update Metadata instruction parsing (still a commit ID). * Update to Borsh 0.10.3. * Update to Plerkle Messenger 1.6.0. * Update to Bubblegum 1.0.1-beta.2 (Rust client version that supports Update Metadata and limits to Borsh < 1.0.0). * Update to SPL Account Compression 0.2.0. * Update to Anchor 0.28.0. * Update Candy Machine and Candy Guard to 2.0.x. * spl-associated-token-account and spl-token use same version ranges as Bubblegum program crate. * Update SPL No-op. * MPL Token Metadata exact version: 2.0.0-beta.1. * Bubblegum Rust client changes * Update plerkle serialization version * Update Rust docker image * Default to no backfiller stream workers (for now)
…ndation#131) * Update to use blockbuster beta release for Bubblegum Update Metadata * Update test validator image in docker * Update docker prep script to download from mainnet
…#139) Update the raw name and raw symbol from the onchain data for existing NFTs when reingesting. This allows correction of incorrect values during reprocessing on an existing index.
* Add code to index Bubblegum Update Metadata * Update rust toolchain file * Fix moved variable after merge * Add code from mintV1 that allows for empty URI * Ordering using asset.seq initially applied to update_metadata * Add simple check for whether asset was decompressed to Bubblegum transformers * Don't prevent sequence number update when already decompressed * Add sequence number to downloading metadata background task * Add sequence number migration (Sea ORM not regenerated yet) * Regenerate Sea-ORM types * Use new sequence numbers for Bubblegum Update Metadata * Extra condition to protect out of order creator verification * Remove base_info_seq for each creator and add creators_added_seq to asset table * Regenerate Sea-ORM types * Change creator metadata updates to use new creators_added_seq * Factor out common creator update code to helper function * Update to latest blockbuster beta * Use less than or equal for download metadata seq check * Index verified for token metadata collection * Add slot_updated to initial asset upsert, and removed duplicate items * Remove asset_was_decompressed Replaced with WHERE clauses on each upsert. Move remaining upserts from mint_v1 to db.rs. Remove upsert to asset_v1_account_attachments from mint_V1. Combine upserts for asset base info and royalty amount. * Rename royalty_amount_seq to base_info_seq * Fix typo in WHERE clause * Do not delete existing creators in mint_v1 * Update comments around database txns * Use transaction in mint_V1 and update_metadata * Use transaction for other Bubblegum instructions asset table updates * Fix tree_id key index in update_metadata * Remove use of was_decompressed flag on asset table * Add migration to remove was_decompressed and regenerate SeaORM types * Combine upsert_asset_base_info and upsert_creators and add lock * Remove unneeded creators_added_seq * Switch to EXCLUSIVE mode lock * Add NULL condition check on asset.seq * Refactored creator indexing * Use new Blockbuster that always updates all creators and verification status. * Remove deleting creators with lower sequence numbers as it would not work due to race conditions. * Add concept of "empty" creator value to support Bubblegum empty creator arrays. * Add filtering out of old creators or having no creators to DAS code. * Also get authority and tree_id accounts from Bubblegum during mint and update_metadata. * Add conditions to creator upsert, add another check at DAS API level * Rename asset_creators.verified_seq back to just regular seq * Remove unneeded condition on asset_authority upsert * Apply stale creator filtering to all DAS API queries * Use latest blockbuster beta release * Remove download_metadata_seq and add URI match check instead * Fix task URI initial query * Regenerate Sea ORM types without download_metadata_seq * asset_grouping.verified option remove * Fix filtering for getAssetsByCreator * Update to blockbuster 0.9.0-beta.5 and mpl-bubblegum 1.0.1-beta.4
* Make workers configurable Make workers fully configurab le and remove reference to the plerkle plugin. * fix lifetime --------- Co-authored-by: Kirill Fomichev <fanatid@ya.ru>
… single. Logic in token account updates would change owner when any token account had amount > 0 would triggers spam updates of the owner of the asset with any transfer. (metaplex-foundation#151)
* Add script to forward transactions an check database results * Fix ordering and add debug info * Add remaining non-creator/non-collection tests * Require asset and cl_items files to exist * Add asset_creators and asset_grouping tests * Add verify_creator and verify_collection tests * Add more collection verification tests * Move test data to subirectory * Move repeated code to functions * Add support for running sequences in reverse * Add instructions to README for running test script * Minor README update
…able (metaplex-foundation#155) * add migration files for cl_audits_v2 * add types * ingester * add getSignaturesForAsset endpoint * refactor to resolve merge conflict related bugs * address clippy error * rename to get_asset_signatures * add instruction type update_metadata * add error log if instruction is unknown * add sort order changes --------- Co-authored-by: Nicolas Pennie <nicolas.pennie@gmail.com>
* fix ORM circular dependency * PR comments
* Remove asset_creator_unique as we now allow duplicate rows * fix creators re-ordering bug * pr comments * fix crgo file --------- Co-authored-by: Michael Danenberg <56533526+danenbm@users.noreply.github.com>
* Fixing docker files to work with workspace * Comment out cargo chef cook for ingester and api
…etadata` test scenarios using devnet txs (metaplex-foundation#165)
* Fix cNFT update metadata indexing (metaplex-foundation#167) * Update to use Token Metadata (and latest Bubblegum) Rust clients (metaplex-foundation#168) * Use Token Metadata and latest Bubblegum Rust clients, new blockbuster * Remove candy machine and candy guard * Update lock file * Improve Master Edition V1 and V2 indexing * Update to use blockbuster published crate * Update lock file * Update Token Metadata and Blockbuster deps (metaplex-foundation#170) * Update to latest token metadata and blockbuster crates * Update Cargo lock file * Add DAS integration tests (metaplex-foundation#169) * Add das integration tests * Delete unused snapshots * Delete unnecessary test data * Clean up dependencies * Nit * Address comments * Add README.md * Fix warnings * feat(ops): add ops crate for bundling misc tools. add a tree backfiller based on cl_audits_v2. * Fix NFT race condition (metaplex-foundation#172) * Add das integration tests * Delete unused snapshots * Delete unnecessary test data * Clean up dependencies * Nit * Address comments * Add README.md * Fix warnings * Fix asset race condition * Add missing files * Fix warnings * Fix missing slot updated * Address comments * Nit * Update Rust version and missing dirs in docker build (metaplex-foundation#173) --------- Co-authored-by: Nicolas Pennie <Nicolas.Pennie@gmail.com> Co-authored-by: Michael Danenberg <56533526+danenbm@users.noreply.github.com> Co-authored-by: pmantica11 <151664502+pmantica11@users.noreply.github.com> Co-authored-by: Kyle Espinola <kyle.s.espinola@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thank you for taking the time to make an enhancement or bug fix to the digital asset validator rpc repo.
To ensure a quality PR experience this call may be recorded. Just Kidding, but we do expect a few things.