Deneb fork choice tests - take 2 #3463

hwwhww · 2023-07-25T15:41:59Z

Replace #3402

New on_block step fields:

{
    block: string           -- the name of the `block_<32-byte-root>.ssz_snappy` file.
                            To execute `on_block(store, block)` with the given attestation.
    blobs: string or `null` -- optional, the name of the `blobs_<32-byte-root>.ssz_snappy` file.
                            The blobs file content is a `List[Blob, MAX_REQUEST_BLOB_SIDECARS]` SSZ object.
                            If it's `null`, `blobs` is an empty list.
    proofs: array of byte48 hex string(s) -- optional, the proofs of blob commitments.
    valid: bool             -- optional, default to `true`.
                            If it's `false`, this execution step is expected to be invalid.
}

blobs and proofs are new fields from Deneb EIP-4844. These are the expected values from retrieve_blobs_and_proofs() helper inside is_data_available() helper.
example:

- block: block_0xe22b0ebe155b38971620a2d794d55f1ab4e058a85eb5b9e189ba95ded0f161ce
  blobs: blobs_0xef5082d6a11f3a142ac70b8a262b1dc025c3a4fa5f3b510336b5cc7edb6329d4
  proofs: ['0x92d7c0767cb497c594737918d0d76a56f15d36a21cd2285901fffe497ec465a356a40733f43ba73fbacaa4c88092b3e6']
  valid: true

  - block: block_0xe22b0ebe155b38971620a2d794d55f1ab4e058a85eb5b9e189ba95ded0f161ce
    blobs: null
    proofs: []
    valid: false

Clean up is_data_available helper: remove the "# For testing, retrieve_blobs_and_proofs returns ("TEST", "TEST")." stub. The retrieve_blobs_and_proofs helper returns empty lists [] ,[] by default, but we can use with_blob_data testing helper to inject the return values now.
Add new debeb FC test cases:
- test_simple_blob_data: one simple blob
- test_invalid_incorrect_proof: one blob commitment with incorrect proof
- test_invalid_data_unavailable: one blob commitment, but retrieve_blobs_and_proofs returns empty lists.

…ofs` responses.

djrtwo

very nice!

I'm comfortable getting this into the release if no opposition from clients (as seemingly on the call)

djrtwo · 2023-07-27T16:48:04Z

tests/formats/fork_choice/README.md

-                      If it's `false`, this execution step is expected to be invalid.
+    block: string           -- the name of the `block_<32-byte-root>.ssz_snappy` file.
+                              To execute `on_block(store, block)` with the given attestation.
+    blobs: string or `null` -- optional, the name of the `blobs_<32-byte-root>.ssz_snappy` file.


Why null instead of just an empty list in the file? seems like that would reduce the exceptional cases here.

djrtwo · 2023-07-27T16:49:22Z

tests/formats/fork_choice/README.md

+    blobs: string or `null` -- optional, the name of the `blobs_<32-byte-root>.ssz_snappy` file.
+                               The blobs file content is a `List[Blob, MAX_BLOBS_PER_BLOCK]` SSZ object.
+                               If it's `null`, `blobs` is an empty list.
+    proofs: array of byte48 hex string -- optional, the proofs of blob commitments.


What if one of blobs or proofs exists but the other is not included? Is that an invalid test? or does one just consider the missing key as empty list?

I think it makes sense that either both keys should be there or both should be absent. And if so, maybe a note below about the relationship between the option keys

And must these two lists have same length? or should they be able to handle if you have more of one than other?

If the latter, should add a test of mismatched lengths

Thanks for the feedback. I changed it back to allow empty blobs SSZ file.

I think it makes sense that either both keys should be there or both should be absent.

Agreed. This is also the is_blob_data_test variable indicated in fork_choice.py::add_block.

And must these two lists have same length?

So the verify_blob_kzg_proof_batch API in is_data_available would check the length consistency and throw invalid.

If the latter, should add a test of mismatched lengths

👍 added these tests.

djrtwo · 2023-07-27T16:51:30Z

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

+    state_transition_and_sign_block,
+)
+from eth2spec.test.helpers.sharding import (
+    get_sample_opaque_tx


Suggested change

get_sample_opaque_tx

get_sample_opaque_tx,

djrtwo · 2023-07-27T16:52:51Z

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

+    block = build_empty_block_for_next_slot(spec, state)
+    opaque_tx, blobs, blob_kzg_commitments, blob_kzg_proofs = get_sample_opaque_tx(spec, blob_count=1, rng=rng)
+    block.body.execution_payload.transactions = [opaque_tx]
+    # block.body.execution_payload.block_hash = compute_el_block_hash(spec, block.body.execution_payload)


Is this intentionally left here?

No, it should have been uncommented although it's unrelated here.

djrtwo · 2023-07-27T16:56:09Z

tests/core/pyspec/eth2spec/test/helpers/fork_choice.py

+
+def with_blob_data(spec, blob_data, func):
+    """
+    This helper runs the given ``func`` with monkeypatched ``retrieve_blobs_and_proofs``


djrtwo · 2023-07-27T16:59:28Z

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

+    signed_block = state_transition_and_sign_block(spec, state, block)
+    blob_data = BlobData(blobs, blob_kzg_proofs)
+
+    def run_func_1():


maybe run_tick_and_add_block_with_data_1 -- a little long... just a bit hesitant on run_func_1 generic name not explaining what's going on

Or maybe we should hide with_blob_data usage with a helper -- tick_and_add_block_with_data -- that does the meta-programming so you don't have to explicitly do it each time. Seems like the exact same usage is used over and over in this file so I think helper to hide makes sense

good suggestion 👍

djrtwo

looks good!

Preparation for processing new tests from: - ethereum/consensus-specs#3463

etan-status · 2023-09-19T12:50:07Z

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

+    signed_block = state_transition_and_sign_block(spec, state, block)
+    blob_data = BlobData(blobs, blob_kzg_proofs)
+
+    yield from tick_and_add_block_with_data(spec, store, signed_block, test_steps, blob_data)


there is no {tick} entry in the resulting steps.yaml, fork choice is still at time 12 when processing the block.

etan-status · 2023-09-19T12:50:22Z

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

+    assert spec.get_head(store) == signed_block.message.hash_tree_root()
+
+    # On receiving a block of next epoch
+    store.time = current_time + spec.config.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH


shouldn't there be a on_tick_and_append_step to run tick processing in fork choice?

Without the on_tick_and_append_step, the followup checks in steps.yaml will fail as the time doesn't match with the test runner fork choice. test runner fork choice needs to be told manually to advance to the target time, via steps.yaml

hwwhww · 2023-09-19T14:06:38Z

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

+    assert spec.get_head(store) == signed_block.message.hash_tree_root()
+
+    # On receiving a block of next epoch
+    store.time = current_time + spec.config.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH


Ahh nice catch! I think this line should be removed because tick_and_add_block_with_data would call on_tick properly.

Preparation for processing new tests from: - ethereum/consensus-specs#3463

hwwhww added 3 commits July 25, 2023 22:29

Add deneb fc tests and update test format

2210cea

Add test_invalid_data_unavailable

d2d351f

Clean up is_data_available. Remove the stub `retrieve_blobs_and_pro…

e79caff

…ofs` responses.

hwwhww added scope:CI/tests/pyspec scope:fork-choice Deneb was called: eip-4844 labels Jul 25, 2023

hwwhww mentioned this pull request Jul 25, 2023

Enhance Deneb tests #3458

Closed

7 tasks

hwwhww requested review from mkalinin and djrtwo July 25, 2023 16:01

handle len(blobs) == 0 case

85b0ae8

hwwhww mentioned this pull request Jul 27, 2023

release v1.4.0-beta.1 #3465

Merged

5 tasks

djrtwo reviewed Jul 27, 2023

View reviewed changes

hwwhww added 2 commits July 31, 2023 19:23

PR feedback from @djrtwo

32056b2

Change it back to allow empty blobs list file

39134d5

djrtwo approved these changes Aug 1, 2023

View reviewed changes

hwwhww merged commit 56d6d1a into dev Aug 3, 2023
13 checks passed

hwwhww deleted the deneb-fc-tests-take-2 branch August 3, 2023 13:40

This was referenced Aug 3, 2023

Extend fork choice tests for testing is_data_available logic #3402

Closed

Add tests for is_data_available in fork choice #3194

Closed

etan-status added a commit to status-im/nimbus-eth2 that referenced this pull request Sep 18, 2023

add blob validation to test_fixture_fork_choice

2d94478

Preparation for processing new tests from: - ethereum/consensus-specs#3463

etan-status mentioned this pull request Sep 18, 2023

add blob validation to test_fixture_fork_choice status-im/nimbus-eth2#5441

Merged

etan-status reviewed Sep 19, 2023

View reviewed changes

hwwhww commented Sep 19, 2023

View reviewed changes

hwwhww mentioned this pull request Sep 19, 2023

Fix Deneb fork choice baseline test case #3508

Merged

etan-status added a commit to status-im/nimbus-eth2 that referenced this pull request Sep 20, 2023

add blob validation to test_fixture_fork_choice (#5441)

aacb853

Preparation for processing new tests from: - ethereum/consensus-specs#3463

terencechain mentioned this pull request Sep 21, 2023

Add New Deneb Fork Choice Spec Test prysmaticlabs/prysm#12933

Closed

StefanBratanov mentioned this pull request Sep 21, 2023

Support Deneb fork choice reference tests Consensys/teku#7539

Closed

terencechain mentioned this pull request Sep 25, 2023

Align to consensus release v1.4.0-beta.2 prysmaticlabs/prysm#12908

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deneb fork choice tests - take 2 #3463

Deneb fork choice tests - take 2 #3463

hwwhww commented Jul 25, 2023 •

edited

Loading

djrtwo left a comment

djrtwo Jul 27, 2023

djrtwo Jul 27, 2023

djrtwo Jul 27, 2023

hwwhww Jul 31, 2023

djrtwo Jul 27, 2023

djrtwo Jul 27, 2023

hwwhww Jul 31, 2023

djrtwo Jul 27, 2023

djrtwo Jul 27, 2023

djrtwo Jul 27, 2023

hwwhww Jul 31, 2023

djrtwo left a comment

etan-status Sep 19, 2023

etan-status Sep 19, 2023

etan-status Sep 19, 2023

hwwhww Sep 19, 2023

Deneb fork choice tests - take 2 #3463

Deneb fork choice tests - take 2 #3463

Conversation

hwwhww commented Jul 25, 2023 • edited Loading

djrtwo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djrtwo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hwwhww commented Jul 25, 2023 •

edited

Loading