Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sequencer): make ABCI response for account balances deterministic #1574

Merged
merged 9 commits into from
Oct 21, 2024
87 changes: 87 additions & 0 deletions crates/astria-core/src/primitive/v1/asset/denom.rs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the Ord impl here, can you provide exhaustive tests that demonstrate and document the order of the denoms?

Naively, I would expect a lexical ordering among trace-prefixed and ibc-prefixed denoms, so it would be great to have tests that also demonstrate that ordering.

I think I should also walk back my suggestion to order ibc-prefixed, trace-prefixed later: a naive implementation that keeps ibc denominations in string form (without parsing) would just use lexical ordering of the strings. I think the parsed variant here should also reflect that.

In addition to unit tests for IbcPrefixed and TracePrefixed, do a more complex example:

  1. create a "string list" of ibc denomination strings (mixing both trace-prefixed and ibc-prefixed).
  2. parse this list as a Denom list.
  3. sort the string list (this will give lexical order).
  4. sort the denom list by its Ord implementation and then use to_string to convert all elements to strings again
  5. assert that results of steps 3 and 4 are the same.

Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,29 @@ impl FromStr for Denom {
}
}

impl PartialOrd for Denom {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}

impl Ord for Denom {
/// If the denoms are the same type, returns their comparison. Otherwise, returns IBC prefixed
/// denoms as less than trace prefixed denoms.
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
match self {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't nest matches but instead match on tuples:

    use Self::{TracePrefixed, IbcPrefixed};
    match (self, other) {
        (TracePrefixed(lhs), TracePrefixed(rhs)) => lhs.cmp(rhs),
        (TracePrefixed(_), IbcPrefixed(_)) => Ordering::Less,
        (IbcPrefixed(lhs), IbcPrefixed(rhs)) => lhs.cmp(rhs),
        (IbcPrefixed(_), TracePrefixed(_)) => Prdering::Greater,
    }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was unable to import Self due to Rust's restrictions, but implemented the tuples in 535c1c4.

Self::TracePrefixed(lhs) => match other {
Self::TracePrefixed(rhs) => lhs.cmp(rhs),
Self::IbcPrefixed(_) => std::cmp::Ordering::Greater,
},
Self::IbcPrefixed(lhs) => match other {
Self::IbcPrefixed(rhs) => lhs.cmp(rhs),
Self::TracePrefixed(_) => std::cmp::Ordering::Less,
},
}
}
}

#[derive(Debug, thiserror::Error)]
#[error(transparent)]
pub struct ParseDenomError(ParseDenomErrorKind);
Expand Down Expand Up @@ -284,6 +307,21 @@ impl TracePrefixed {
}
}

impl PartialOrd for TracePrefixed {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}

impl Ord for TracePrefixed {
/// Returns trace comparison if not equal, otherwise returns base denom comparison.
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.trace
.cmp(&other.trace)
.then_with(|| self.base_denom.cmp(&other.base_denom))
}
}

#[derive(Debug, Clone, Hash, PartialEq, Eq)]
struct TraceSegments {
inner: VecDeque<PortAndChannel>,
Expand Down Expand Up @@ -354,6 +392,28 @@ impl FromStr for TraceSegments {
Ok(parsed_segments)
}
}

impl PartialOrd for TraceSegments {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}

impl Ord for TraceSegments {
/// Returns the first non-equal comparison between the two trace segments. If one doesn't exist,
/// returns the shortest, and if they are equal, returns equal.
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.inner
.iter()
.zip(other.inner.iter())
.find_map(|(self_segment, other_segment)| {
Some(self_segment.cmp(other_segment))
.filter(|&cmp| cmp != std::cmp::Ordering::Equal)
})
.unwrap_or(self.inner.len().cmp(&other.inner.len()))
}
}

#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct PortAndChannel {
port: String,
Expand All @@ -372,6 +432,21 @@ impl PortAndChannel {
}
}

impl PartialOrd for PortAndChannel {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}

impl Ord for PortAndChannel {
/// Returns port comparison if not equal, otherwise returns channel comparison.
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.port
.cmp(&other.port)
.then_with(|| self.channel.cmp(&other.channel))
}
}

impl std::fmt::Display for TracePrefixed {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
for segment in &self.trace.inner {
Expand Down Expand Up @@ -549,6 +624,18 @@ impl FromStr for IbcPrefixed {
}
}

impl PartialOrd for IbcPrefixed {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}

impl Ord for IbcPrefixed {
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.id.cmp(&other.id)
}
}

#[cfg(feature = "serde")]
mod serde_impl {
use serde::{
Expand Down
91 changes: 90 additions & 1 deletion crates/astria-sequencer/src/accounts/query.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ async fn get_trace_prefixed_account_balances<S: StateRead>(
stream.try_collect::<Vec<_>>().await
}

/// Returns a list of `AssetBalance`s for the provided address. `AssetBalance`s are sorted first in
/// descending order by balance, then in ascending order by denom. IBC prefixed denoms are treated
/// as less than trace prefixed denoms.
pub(crate) async fn balance_request(
storage: Storage,
request: request::Query,
Expand All @@ -88,7 +91,7 @@ pub(crate) async fn balance_request(
Err(err_rsp) => return err_rsp,
ethanoroshiba marked this conversation as resolved.
Show resolved Hide resolved
};

let balances = match get_trace_prefixed_account_balances(&snapshot, address).await {
let mut balances = match get_trace_prefixed_account_balances(&snapshot, address).await {
Ok(balance) => balance,
Err(err) => {
return response::Query {
Expand All @@ -100,6 +103,11 @@ pub(crate) async fn balance_request(
};
}
};

// Unstable sort does not allocate auxillary memory and is typically faster. Custom sorting
// function is deterministic and only allocates in the case that two values are equal.
ethanoroshiba marked this conversation as resolved.
Show resolved Hide resolved
balances.sort_unstable_by(compare_asset_balances);

let payload = BalanceResponse {
height: height.value(),
balances,
Expand Down Expand Up @@ -215,3 +223,84 @@ async fn preprocess_request(
};
Ok((address, snapshot, height))
}

/// Custom deterministic sorting function for `AssetBalance` that sorts by balance in descending
/// order and then by denom in ascending order. This function should never return `Ordering::Equal`,
/// as calling `sort_unstable_by` on equal values has a non-deterministic result.
fn compare_asset_balances(lhs: &AssetBalance, rhs: &AssetBalance) -> std::cmp::Ordering {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rethinking this: these should just be sorted by lexical order of the denominations.

You cannot compare the magnitude of different quantities, so we shouldn't order by them.

use std::cmp::Ordering;

match lhs.balance.cmp(&rhs.balance) {
Ordering::Equal => lhs.denom.cmp(&rhs.denom),
Ordering::Less => Ordering::Greater,
Ordering::Greater => Ordering::Less,
}
}

#[cfg(test)]
mod test {
ethanoroshiba marked this conversation as resolved.
Show resolved Hide resolved
#[test]
fn compare_asset_balances_sorts_asset_balances_as_expected() {
use astria_core::{
primitive::v1::asset::Denom,
protocol::account::v1alpha1::AssetBalance,
};

let trace_denom_1 = "first/new/asset".parse::<Denom>().unwrap();
let trace_denom_2 = "second/new/asset".parse::<Denom>().unwrap();
let ibc_denom_1 = format!("ibc/{}", hex::encode([0u8; 32]))
.parse::<Denom>()
.unwrap();
let ibc_denom_2 = format!("ibc/{}", hex::encode([1u8; 32]))
.parse::<Denom>()
.unwrap();

let mut balances = vec![
AssetBalance {
denom: trace_denom_1.clone(),
balance: 1,
},
AssetBalance {
denom: trace_denom_1.clone(),
balance: 2,
},
AssetBalance {
denom: trace_denom_2.clone(),
balance: 2,
},
AssetBalance {
denom: trace_denom_2.clone(),
balance: 3,
},
AssetBalance {
denom: ibc_denom_1.clone(),
balance: 3,
},
AssetBalance {
denom: ibc_denom_1.clone(),
balance: 4,
},
AssetBalance {
denom: ibc_denom_2.clone(),
balance: 4,
},
AssetBalance {
denom: ibc_denom_2.clone(),
balance: 5,
},
];

let original_balances = balances.clone();

balances.sort_unstable_by(super::compare_asset_balances);

assert_eq!(balances[0], original_balances[7]);
assert_eq!(balances[1], original_balances[5]);
assert_eq!(balances[2], original_balances[6]);
assert_eq!(balances[3], original_balances[4]);
assert_eq!(balances[4], original_balances[3]);
assert_eq!(balances[5], original_balances[1]);
assert_eq!(balances[6], original_balances[2]);
assert_eq!(balances[7], original_balances[0]);
}
}
Loading