Skip to content

Commit

Permalink
Keep data in fails cases in sync service (#2361)
Browse files Browse the repository at this point in the history
## Linked Issues/PRs
Closes #2357

## Description

This pull request introduces a caching mechanism to the sync service to
avoid redundant data fetching from the network. The most important
changes include adding a cache module, modifying the `Import` struct to
include a cache, and updating related methods to utilize this cache.

### Caching Mechanism:

*
[`crates/services/sync/src/import.rs`](diffhunk://#diff-08dae2906fc83b4bb6a03ea474b9e751ef7e290343f66859a669a101c0844a9fR5-R8):
Added a new `cache` module and integrated it into the `Import` struct.
Updated methods to use the cache for fetching and storing headers and
blocks.
* Cache mechanism allow use to retrieve a stream of batches of either
cached headers, cached full blocks, or range to fetch data.

### Test Updates:

* Update the P2P port in mocks to use async to simulate more complex
tests needed for this feature.


This PR contains 50% of changes in the tests and addition of tests in
the cache.

## Checklist
- [x] Breaking changes are clearly marked as such in the PR description
and changelog
- [x] New behavior is reflected in tests
- [x] [The specification](https://github.com/FuelLabs/fuel-specs/)
matches the implemented behavior (link update PR if changes are needed)

### Before requesting review
- [x] I have reviewed the code myself
- [x] I have created follow-up issues caused by this PR and linked them
here

---------

Co-authored-by: Rafał Chabowski <88321181+rafal-ch@users.noreply.github.com>
Co-authored-by: green <xgreenx9999@gmail.com>
  • Loading branch information
3 people authored Dec 2, 2024
1 parent c78b8ba commit 28c4ae3
Show file tree
Hide file tree
Showing 10 changed files with 1,307 additions and 287 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- [2362](https://github.com/FuelLabs/fuel-core/pull/2362): Added a new request_response protocol version `/fuel/req_res/0.0.2`. In comparison with `/fuel/req/0.0.1`, which returns an empty response when a request cannot be fulfilled, this version returns more meaningful error codes. Nodes still support the version `0.0.1` of the protocol to guarantee backward compatibility with fuel-core nodes. Empty responses received from nodes using the old protocol `/fuel/req/0.0.1` are automatically converted into an error `ProtocolV1EmptyResponse` with error code 0, which is also the only error code implemented. More specific error codes will be added in the future.
- [2386](https://github.com/FuelLabs/fuel-core/pull/2386): Add a flag to define the maximum number of file descriptors that RocksDB can use. By default it's half of the OS limit.
- [2376](https://github.com/FuelLabs/fuel-core/pull/2376): Add a way to fetch transactions in P2P without specifying a peer.
- [2361](https://github.com/FuelLabs/fuel-core/pull/2361): Add caches to the sync service to not reask for data it already fetched from the network.
- [2327](https://github.com/FuelLabs/fuel-core/pull/2327): Add more services tests and more checks of the pool. Also add an high level documentation for users of the pool and contributors.
- [2416](https://github.com/FuelLabs/fuel-core/issues/2416): Define the `GasPriceServiceV1` task.
- [2033](https://github.com/FuelLabs/fuel-core/pull/2033): Remove `Option<BlockHeight>` in favor of `BlockHeightQuery` where applicable.
Expand Down
2 changes: 1 addition & 1 deletion benches/benches/import.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ use fuel_core_sync::state::State;
use std::time::Duration;
use tokio::runtime::Runtime;

async fn execute_import(import: PressureImport, shutdown: &mut StateWatcher) {
async fn execute_import(mut import: PressureImport, shutdown: &mut StateWatcher) {
import.import(shutdown).await.unwrap();
}

Expand Down
19 changes: 19 additions & 0 deletions crates/fuel-core/src/service/adapters/sync.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,25 @@ impl PeerToPeerPort for P2PAdapter {
}

async fn get_transactions(
&self,
block_ids: Range<u32>,
) -> anyhow::Result<SourcePeer<Option<Vec<Transactions>>>> {
let result = if let Some(service) = &self.service {
service.get_transactions(block_ids).await
} else {
Err(anyhow::anyhow!("No P2P service available"))
};
match result {
Ok((peer_id, transactions)) => {
let peer_id: PeerId = peer_id.into();
let transactions = peer_id.bind(transactions);
Ok(transactions)
}
Err(err) => Err(err),
}
}

async fn get_transactions_from_peer(
&self,
range: SourcePeer<Range<u32>>,
) -> anyhow::Result<Option<Vec<Transactions>>> {
Expand Down
Loading

0 comments on commit 28c4ae3

Please sign in to comment.