Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(docs): updates to data verification and inclusion proof specs #1689

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

chore(docs): updates to data verification and inclusion proof specs #1689

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cd901eb
updates to data verification and inclusion proof specs
noot Oct 18, 2024
a83eff5
address comment
noot Nov 6, 2024
8c5a42a
Merge branch 'main' into noot/specs
noot Nov 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
195 changes: 108 additions & 87 deletions specs/data-flow-and-verification.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,32 +11,31 @@ the exit point is execution by a rollup node.

## Entry point

The entry point for rollup data is via a `sequence::Action`, which can become
part of a sequencer transaction. The data types are as follows:
The entry point for rollup data is via a `RollupDataSubmission` action, which can
become part of a sequencer transaction. The data types are as follows:

```rust
// sequence::Action
pub struct Action {
pub(crate) chain_id: Vec<u8>,
pub(crate) data: Vec<u8>,
pub struct RollupDataSubmission {
rollup_id: Vec<u8>,
data: Vec<u8>,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks more like this now:

pub struct RollupDataSubmission {
    rollup_id: RollupId,
    data: Bytes,
    fee_asset: asset::Denom,
}

There probably should be a short explanation of the fee_asset field below where we already describe rollup_id and data?

```

```rust
// an unsigned transaction
pub struct Unsigned {
pub(crate) nonce: Nonce,
pub(crate) actions: Vec<Action>,
// an unsigned transaction body
pub struct TransactionBody {
actions: Actions, // vector of actions
params: TransactionParams, // chain id, nonce
}
```

The `data` field inside the `sequence::Action` is arbitrary bytes, which should
The `data` field inside the `RollupDataSubmission` is arbitrary bytes, which should
be an encoded rollup transaction. The sequencer is agnostic to the transaction
format of the rollups using it. The `chain_id` field is an identifier for the
format of the rollups using it. The `rollup_id` field is an identifier for the
rollup the data is destined for.

To submit rollup data to the system, the user creates a transaction with a
`sequence::Action` within it and signs and submits it to the sequencer. The
`RollupDataSubmission` within it and signs and submits it to the sequencer. The
sequencer will then include it in a block, thus finalizing its ordering.

## Sequencer to data availability
Expand All @@ -47,56 +46,81 @@ the block data is published via a data availability layer.
The block data published is as follows:

```rust
pub struct SequencerBlockData {
block_hash: Hash,
header: Header,
/// chain ID -> rollup transactions
rollup_data: BTreeMap<ChainId, Vec<Vec<u8>>>,
/// The root of the action tree for this block.
action_tree_root: [u8; 32],
/// The inclusion proof that the action tree root is included
/// in `Header::data_hash`.
action_tree_root_inclusion_proof: merkle::Proof,
/// The commitment to the chain IDs of the rollup data.
/// The merkle root of the tree where the leaves are the chain IDs.
chain_ids_commitment: [u8; 32],
/// The inclusion proof that the chain IDs commitment is included
/// in `Header::data_hash`.
chain_ids_commitment_inclusion_proof: merkle::Proof,
pub struct SequencerBlockHeader {
chain_id: tendermint::chain::Id,
height: tendermint::block::Height,
time: Time,
// the merkle root of all the rollup data in the block
rollup_transactions_root: [u8; 32],
// the merkle root of all transactions in the block
data_hash: [u8; 32],
proposer_address: account::Id,
}

pub struct SequencerBlock {
/// the cometbft block hash for this block
block_hash: [u8; 32],
/// the block header, which contains cometbft header info and additional sequencer-specific
/// commitments.
header: SequencerBlockHeader,
/// The collection of rollup transactions that were included in this block.
rollup_transactions: IndexMap<RollupId, RollupTransactions>,
/// The inclusion proof that the rollup transactions merkle root is included
/// in `header.data_hash`.
rollup_transactions_proof: merkle::Proof,
/// The inclusion proof that the rollup IDs commitment is included
/// in `header.data_hash`.
rollup_ids_proof: merkle::Proof,
}
```

When this data is actually published, it's split into multiple structures.
When the `SequencerBlock` is actually published, it's split into multiple structures.
Specifically, the data for each rollup is written independently, while a "base"
data type which contains the rollup chain IDs included in the block is also
written. This allows each rollup to only require the `SequencerNamespaceData`
for the block and the `RollupNamespaceData` for its own rollup transactions. For
each block, if there are N rollup chain IDs included, 1 + N structures are
written to DA.
data type which contains all the other `SequencerBlock` info, plus the list of
rollup IDs in the block, is written. This allows each rollup to only require
the `SequencerNamespaceData` for the block and the `RollupNamespaceData` for
its own rollup transactions. For each block, if there are N rollup chain IDs
included, 1 + N structures are written to DA.

```rust
/// SequencerNamespaceData represents the data written to the "base"
/// sequencer namespace. It contains all the other chain IDs (and thus,
/// SubmittedMetadata represents the data written to the "base"
/// sequencer namespace. It contains all the other rollup IDs (and thus,
/// namespaces) that were also written to in the same block.
#[derive(Serialize, Deserialize, Debug)]
pub struct SequencerNamespaceData {
pub block_hash: Hash,
pub header: Header,
pub rollup_chain_ids: Vec<ChainId>,
pub action_tree_root: [u8; 32],
pub action_tree_root_inclusion_proof: InclusionProof,
pub chain_ids_commitment: [u8; 32],
pub struct SubmittedMetadata {
/// The block hash obtained from hashing `.header`.
block_hash: [u8; 32],
/// The sequencer block header.
header: SequencerBlockHeader,
/// The rollup IDs for which `SubmittedRollupData`s were submitted to celestia.
/// Corresponds to the `astria.sequencer.v1.RollupTransactions.id` field
/// and is extracted from `astria.SequencerBlock.rollup_transactions`.
rollup_ids: Vec<RollupId>,
/// The proof that the rollup transactions are included in sequencer block.
/// Corresponds to `astria.SequencerBlock.rollup_transactions_proof`.
rollup_transactions_proof: merkle::Proof,
/// The proof that this sequencer blob includes all rollup IDs of
/// the original sequencer block it was derived from.
/// This proof together with `Sha256(MTH(rollup_ids))` (Sha256
/// applied to the Merkle Tree Hash of the rollup ID sequence) must be
/// equal to `header.data_hash` which itself must match
/// `astria.SequencerBlock.header.data_hash`. This field corresponds to
/// `astria.SequencerBlock.rollup_ids_proof`.
rollup_ids_proof: merkle::Proof,
}
```

```rust
/// RollupNamespaceData represents the data written to a rollup namespace.
#[derive(Serialize, Deserialize, Debug)]
pub struct RollupNamespaceData {
pub(crate) block_hash: Hash,
pub(crate) chain_id: ChainId,
pub rollup_txs: Vec<Vec<u8>>,
pub(crate) inclusion_proof: InclusionProof,
/// SubmittedRollupData represents the data written to a rollup namespace.
pub struct SubmittedRollupData {
/// The hash of the sequencer block. Must be 32 bytes.
sequencer_block_hash: [u8; 32],
/// The 32 bytes identifying the rollup this blob belongs to. Matches
/// `astria.sequencerblock.v1.RollupTransactions.rollup_id`
rollup_id: RollupId,
/// A list of opaque bytes that are serialized rollup transactions.
transactions: Vec<Bytes>,
/// The proof that these rollup transactions are included in sequencer block.
proof: merkle::Proof,
}
```

Expand All @@ -108,9 +132,6 @@ properties as ordering, completeness, and correctness respectively. It is able
to do this *without* requiring the full transaction data of the block, as is
explained below.

Note that the `Header` field in `SequencerNamespaceData` is a [Tendermint
header](https://github.com/informalsystems/tendermint-rs/blob/4d81b67c28510db7d2d99ed62ebfa9fdf0e02141/tendermint/src/block/header.rs#L25).

## Data availability to rollup node

For a rollup node to verify the ordering, completeness, and correctness of the
Expand All @@ -120,13 +141,13 @@ block data it receives, it must verify the following:
2. the block hash was in fact committed by the sequencer (ie. >2/3 stake voted
to commit this block hash to the chain)
3. the block header correctly hashes to the block hash
4. the `data_hash` inside the header contains the `action_tree_root` of the
4. the `data_hash` inside the header contains the `rollup_transactions_root` of the
block (see [sequencer inclusion proofs](sequencer-inclusion-proofs.md) for
details), which is a commitment to the `sequence:Action`s in the block
5. the `rollup_txs` inside `RollupNamespaceData` is contained within the
`action_tree_root`
6. the `chain_ids_commitment` is a valid commitment to `rollup_chain_ids`
7. the `data_hash` inside the header contains the `chain_ids_commitment`
details), which is a commitment to the `RollupDataSubmission`s in the block
5. the `transactions` inside `SubmittedRollupData` is contained within the
`rollup_transactions_root`
6. the `rollup_ids_commitment` is a valid commitment to `rollup_ids`
7. the `data_hash` inside the header contains the `rollup_ids_commitment`
for the block.

Let's go through these one-by-one.
Expand Down Expand Up @@ -159,45 +180,45 @@ The block hash is a commitment to the block header (specifically, the merkle
root of the tree where the leaves are each header field). We then verify that
the block header merkleizes to the block hash correctly.

### 4. `action_tree_root`
### 4. `rollup_transactions_root`

The block's data (transactions) contain the `action_tree_root` of the block (see
[sequencer inclusion proofs](sequencer-inclusion-proofs.md) for details), which
is a commitment to the `sequence:Action`s in the block. Specifically, the
`action_tree_root` is the root of a merkle tree where each leaf is a commitment
to the rollup data for one spceific rollup. The block header contains the field
`data_hash` which is the merkle root of all the transactions in a block. Since
`action_tree_root` is a transaction, we can prove its inclusion inside
`data_hash` (the `action_tree_root_inclusion_proof` field inside
`SequencerNamespaceData`). Then, in the next step, we can verify that the rollup
data we received was included inside `action_tree_root`.
The block's data (transactions) contain the `rollup_transactions_root` of the
block (see [sequencer inclusion proofs](sequencer-inclusion-proofs.md) for details),
which is a commitment to the `RollupDataSubmission`s in the block. Specifically,
the `rollup_transactions_root` is the root of a merkle tree where each leaf is a
commitment to the rollup data for one spceific rollup. The block header contains
the field `data_hash` which is the merkle root of all the transactions in a block.
Since `rollup_transactions_root` is a transaction, we can prove its inclusion inside
`data_hash` (the `rollup_transactions_proof` field inside
`SubmittedMetadata`). Then, in the next step, we can verify that the rollup
data we received was included inside `rollup_transactions_root`.

### 5. `rollup_txs`

We calculate a commitment of the rollup data we receive (`rollup_txs` inside
`RollupNamespaceData`). We then verify that this data is included inside
`action_tree_root` (via the `inclusion_proof` field inside
`RollupNamespaceData`). At this point, we are now certain that the rollup data
`SubmittedRollupMetadata`). We then verify that this data is included inside
`rollup_transactions_root` (via the `proof` field inside
`SubmittedRollupMetadata`). At this point, we are now certain that the rollup data
we received, which is a subset of the entire block's data, was in fact committed
by the majority of the sequencer chain's validators.

### 6. `chain_ids_commitment`
### 6. `rollup_ids_root`

The `SequencerNamespaceData` contains a list of the `rollup_chain_ids` that were
included in the block. However, to ensure that chain IDs are not omitted when
The `SubmittedMetadata` contains a list of the `rollup_ids` that were
included in the block. However, to ensure that rollup IDs are not omitted when
publishing the data (which would be undetectable to rollup nodes without forcing
them to pull the entire block's data), we also add a commitment to the chain IDs
them to pull the entire block's data), we also add a commitment to the rollup IDs
in the block inside the block's transaction data. We ensure that the
`rollup_chain_ids` inside `SequencerNamespaceData` match the
`chain_ids_commitment`. This proves that no chain IDs were omitted from the
published block, as if any were omitted, then the `chain_ids_commitment` would
not match the commitment generated from `rollup_chain_ids`.
`rollup_ids` inside `SubmittedMetadata` match the
`rollup_ids_root`. This proves that no rollup IDs were omitted from the
published block, as if any were omitted, then the `rollup_ids_root` would
not match the commitment generated from `rollup_ids`.

### 7. `chain_ids_commitment_inclusion_proof`
### 7. `rollup_ids_root_inclusion_proof`

Similarly to verification of `action_tree_root` inside `data_hash`, we also verify
an inclusion proof of `chain_ids_commitment` inside `data_hash` when receiving a
published block.
Similarly to verification of `rollup_transactions_root` inside `data_hash`, we also
verify an inclusion proof of `rollup_ids_root` inside `data_hash` when receiving
a published block.

## Exit point

Expand Down
20 changes: 12 additions & 8 deletions specs/sequencer-inclusion-proofs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,18 @@ A rollup conductor needs to be able to verify its subset of relevant data
without needing to pull all the transaction data for a block. To do this, the
block proposer includes a special "commitment tx" at the start of the block.
This a special transaction type that can only be included by the proposer.
The "commitment tx" is the merkle root of a tree where each leaf is a commitment
to the batch of transactions for one specific rollup.

![image](assets/sequencer_inclusion_proof_0.png)

When building a block, the proposer deconstructs all txs into their contained
`sequence::Action`s and groups them all. Remember that 1 `sequence::Action`
corresponds to 1 rollup transaction. Then, a commitment to the set of all
actions for a chain becomes a leaf in a merkle tree, the root of which becomes
the "commitment tx"
`RollupDataSubmission` actions and groups them all. Remember that 1
`RollupDataSubmission` action corresponds to 1 rollup transaction. Then, a
commitment to the set of all actions for a chain becomes a leaf in a merkle tree,
the root of which becomes the "commitment tx". The other validators reconstruct
this merkle tree when validating the proposal, and only accept the proposal if
the tree is a valid representation of the rollup data in the block.

![image](assets/sequencer_inclusion_proof_2.png)

Expand All @@ -55,8 +59,8 @@ For example:
layer and calculate the commitment which should match the leaf in red (in the
above diagram)
- we use the inclusion proof (in green) to verify the txs for chain C were
included in the action tree
- we verify a proof of inclusion of "commitment tx" (action tree root) inside
included in the rollup data merkle tree
- we verify a proof of inclusion of "commitment tx" (rollup data tree root) inside
the block header's `data_hash`
- we verify that `data_hash` was correctly included in the block's `block_hash`
- verify `block_hash`'s cometbft >2/3 commitment
Expand All @@ -68,8 +72,8 @@ staking power of the sequencer chain.

Additionally, the commitment to the actions for a chain actually also includes a
merkle root. The commitment contains of the merkle root of a tree where where
the leaves are the transactions for that rollup; ie. all the `sequence::Action`s
for that chain. "Commitment to actions for chain X" is implemented as `(chain_id
the leaves are the transactions for that rollup; ie. all the `RollupDataSubmission`
actions for that chain. "Commitment to actions for chain X" is implemented as `(chain_id
|| root of tx tree for rollup)`, allowing for easy verification that a specific
rollup transaction was included in a sequencer block. This isn't required for
any specific conductor logic, but nice for applications building on top of the
Expand Down
Loading