Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: ABCI methods benchmarks #3904

Merged
merged 37 commits into from
Oct 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
91b40ae
chore: add benchmarks for msg send in 8mb
rach-id Sep 25, 2024
5092439
chore: run benchmarks multiple times
rach-id Sep 25, 2024
7335643
chore: two decimals after comma in block size
rach-id Sep 25, 2024
9c0742e
chore: add 8b check tx and deliver tx
rach-id Sep 26, 2024
ff28c12
chore: add PFB benchmarks
rach-id Sep 27, 2024
53177d8
chore: initial ibc client update benchmarks
rach-id Sep 28, 2024
a260476
chore: create the ibc client before updating it
rach-id Sep 30, 2024
3511b4b
chore: working update client transaction
rach-id Sep 30, 2024
9345a2a
chore: working ibc update client benchmarks
rach-id Sep 30, 2024
62729ca
chore: checks on validity of benchmark + not working comment
rach-id Oct 1, 2024
95250f7
chore: ibc update client benchmarks working
rach-id Oct 1, 2024
8b083fa
chore: pfb benchmarks working
rach-id Oct 1, 2024
5a556b7
chore: msg send benchmarks working
rach-id Oct 1, 2024
68002b7
chore: fix ibc update client comment and reports
rach-id Oct 3, 2024
dab12d3
chore: add half second benchmarks
rach-id Oct 3, 2024
a18707d
chore: only report the result
rach-id Oct 10, 2024
7b386be
Merge remote-tracking branch 'origin/main' into 8mb-benchmark
rach-id Oct 17, 2024
ae7fb19
docs: add result docs
rach-id Oct 17, 2024
87c28e2
chore: move to benchmark folder
rach-id Oct 17, 2024
dd3dd6b
chore: revert unnecessary changes
rach-id Oct 17, 2024
3a5e979
chore: build tags
rach-id Oct 17, 2024
ce56501
chore: add readme and cosmetics
rach-id Oct 17, 2024
ecaaeb6
chore: index out of range fix
rach-id Oct 17, 2024
af52646
Update app/benchmarks/benchmark_msg_send_test.go
rach-id Oct 17, 2024
0742f18
Merge remote-tracking branch 'origin/8mb-benchmark' into 8mb-benchmark
rach-id Oct 17, 2024
0fb7b39
chore: check error when simulating transaction
rach-id Oct 17, 2024
12fea1c
chore: set number of iterations
rach-id Oct 17, 2024
2bd31ca
chore: use bench_abci_methods instead
rach-id Oct 17, 2024
ad76ff2
chore: remove unnecessarily duplicate no op logger set
rach-id Oct 17, 2024
3a4e835
chore: remove print statement
rach-id Oct 17, 2024
a76ffcc
Update app/benchmarks/README.md
rach-id Oct 18, 2024
9a45f4c
chore: rename to numberOfTransactions and blobSize
rach-id Oct 18, 2024
d60a1b4
Merge remote-tracking branch 'origin/8mb-benchmark' into 8mb-benchmark
rach-id Oct 18, 2024
1608073
Update app/benchmarks/benchmark_ibc_update_client_test.go
rach-id Oct 18, 2024
1ed45b1
chore: use megabyte const
rach-id Oct 18, 2024
0fc3483
Merge remote-tracking branch 'origin/main' into 8mb-benchmark
rach-id Oct 18, 2024
c6c6c81
chore: rename
rach-id Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions app/benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Benchmarks

This package contains benchmarks for the ABCI methods with the following transaction types:

- Message send
- IBC update client
- PayForBlobs

## How to run

To run the benchmarks, run the following in the root directory:

```shell
go test -tags=bench_abci_methods -bench=<benchmark_name> app/benchmarks/benchmark_*
```

## Results

The results are outlined in the [results](results.md) document.

## Key takeaways

We decided to softly limit the number of messages contained in a block, via introducing the `MaxPFBMessages` and `MaxNonPFBMessages`, and checking against them in prepare proposal.

This way, the default block construction mechanism will only propose blocks that respect these limitations. And if a block that doesn't respect them reached consensus, it will still be accepted since this rule is not consensus breaking.

As specified in [results](results.md) document, those results were generated on 16 core 48GB RAM machine, and gave us certain thresholds. However, when we run the same experiments on the recommended validator setup, 4 cores 16GB RAM, the numbers were lower. These low numbers are what we used in the limits.
Loading
Loading