Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: add sanitizers support for regression tests #672

Merged
merged 1 commit into from
Aug 7, 2024
Merged

ci: add sanitizers support for regression tests #672

merged 1 commit into from
Aug 7, 2024

Conversation

antonbaliasnikov
Copy link
Contributor

@antonbaliasnikov antonbaliasnikov commented Aug 7, 2024 •
edited
Loading

Code Review Checklist

  • Adds sanitizer supports for regression tests with a manual trigger.
  • Removed explicit LLVM_ENABLE_WERROR and CMAKE_EXPORT_COMPILE_COMMANDS as it's now default behavior
  • Fix typo for evmemulator output

Purpose

To be able to run regression tests with a sanitizer in debugging purposes.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024

Benchmark results default:

╔═╡ Size (-%) ╞════════════════╡ All M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════════════╡ All M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════════════╡ All M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞════════════════╡ All MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════════════╡ All MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════════════╡ All MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞═════╡ EVMInterpreter M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                                 NaN ║
╠═╡ Cycles (-%) ╞═══╡ EVMInterpreter M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞═════╡ EVMInterpreter M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs/gas ╞══════╡ EVMInterpreter M3B3 ╞═╣
║ ADD                                42.750 ║
║ MUL                                25.650 ║
║ SUB                                42.750 ║
║ DIV                                30.450 ║
║ SDIV                               46.050 ║
║ MOD                                30.450 ║
║ SMOD                               43.650 ║
║ ADDMOD                             24.406 ║
║ MULMOD                             25.906 ║
║ EXP                                 7.438 ║
║ SIGNEXTEND                         28.050 ║
║ LT                                 46.750 ║
║ GT                                 46.750 ║
║ SLT                                70.750 ║
║ SGT                                68.750 ║
║ EQ                                 46.750 ║
║ ISZERO                             44.417 ║
║ AND                                42.750 ║
║ OR                                 42.750 ║
║ XOR                                42.750 ║
║ NOT                                38.417 ║
║ BYTE                               54.750 ║
║ SHL                                46.750 ║
║ SHR                                48.750 ║
║ SAR                                68.750 ║
║ SGT                                68.750 ║
║ SHA3                               28.243 ║
║ ADDRESS                            53.812 ║
║ BALANCE                            73.625 ║
║ ORIGIN                           1369.375 ║
║ CALLER                             53.812 ║
║ CALLVALUE                          53.812 ║
║ CALLDATALOAD                       40.750 ║
║ CALLDATASIZE                       54.125 ║
║ CALLDATACOPY                       73.123 ║
║ CODESIZE                           54.625 ║
║ CODECOPY                          133.379 ║
║ GASPRICE                         1372.188 ║
║ EXTCODESIZE                         5.128 ║
║ EXTCODECOPY                         5.180 ║
║ RETURNDATASIZE                     52.500 ║
║ RETURNDATACOPY                     45.222 ║
║ EXTCODEHASH                         8.212 ║
║ BLOCKHASH                         241.019 ║
║ COINBASE                         1372.375 ║
║ TIMESTAMP                        1369.375 ║
║ NUMBER                           1369.375 ║
║ PREVRANDAO                       1369.375 ║
║ GASLIMIT                         1375.375 ║
║ CHAINID                          1369.375 ║
║ SELFBALANCE                       647.975 ║
║ BASEFEE                          1369.375 ║
║ POP                                44.625 ║
║ MLOAD                              64.390 ║
║ MSTORE                             62.562 ║
║ MSTORE8                            74.127 ║
║ SLOAD                              26.837 ║
║ SSTORE                              9.059 ║
║ JUMP                               23.111 ║
║ JUMPI                              21.727 ║
║ PC                                 54.312 ║
║ MSIZE                              60.812 ║
║ GAS                                51.312 ║
║ JUMPDEST                           77.625 ║
║ PUSH0                              51.312 ║
║ PUSH1                              45.958 ║
║ PUSH2                              49.375 ║
║ PUSH4                              52.208 ║
║ PUSH5                              53.625 ║
║ PUSH6                              55.042 ║
║ PUSH7                              56.458 ║
║ PUSH8                              57.875 ║
║ PUSH9                              59.292 ║
║ PUSH10                             60.708 ║
║ PUSH11                             62.125 ║
║ PUSH12                             63.542 ║
║ PUSH13                             64.958 ║
║ PUSH14                             66.375 ║
║ PUSH15                             67.792 ║
║ PUSH16                             69.208 ║
║ PUSH17                             70.625 ║
║ PUSH18                             72.042 ║
║ PUSH19                             73.458 ║
║ PUSH20                             74.875 ║
║ PUSH21                             76.292 ║
║ PUSH22                             77.708 ║
║ PUSH23                             79.125 ║
║ PUSH24                             80.542 ║
║ PUSH25                             81.958 ║
║ PUSH26                             83.375 ║
║ PUSH27                             84.792 ║
║ PUSH28                             86.208 ║
║ PUSH29                             87.625 ║
║ PUSH30                             89.042 ║
║ PUSH31                             90.458 ║
║ PUSH32                             89.875 ║
║ DUP1                               36.417 ║
║ DUP2                               44.417 ║
║ DUP3                               42.417 ║
║ DUP4                               44.417 ║
║ DUP5                               44.417 ║
║ DUP6                               44.417 ║
║ DUP7                               44.417 ║
║ DUP8                               44.417 ║
║ DUP9                               44.417 ║
║ DUP10                              44.417 ║
║ DUP11                              44.417 ║
║ DUP12                              44.417 ║
║ DUP13                              44.417 ║
║ DUP14                              44.417 ║
║ DUP15                              44.417 ║
║ DUP16                              44.417 ║
║ SWAP1                              47.083 ║
║ SWAP2                              47.083 ║
║ SWAP3                              47.083 ║
║ SWAP4                              47.083 ║
║ SWAP5                              45.083 ║
║ SWAP6                              47.083 ║
║ SWAP7                              47.083 ║
║ SWAP8                              47.083 ║
║ SWAP9                              47.083 ║
║ SWAP10                             47.083 ║
║ SWAP11                             47.083 ║
║ SWAP12                             47.083 ║
║ SWAP13                             47.083 ║
║ SWAP14                             47.083 ║
║ SWAP15                             47.083 ║
║ SWAP16                             47.083 ║
║ CALL                               60.523 ║
║ STATICCALL                         60.513 ║
║ DELEGATECALL                       59.548 ║
║ CREATE                              5.380 ║
║ CREATE2                             7.576 ║
║ RETURN                              1.000 ║
║ REVERT                              1.000 ║
╠═╡ Ergs/gas (-%) ╞═╡ EVMInterpreter M3B3 ╞═╣
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞═════╡ EVMInterpreter MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                                 NaN ║
╠═╡ Cycles (-%) ╞═══╡ EVMInterpreter MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞═════╡ EVMInterpreter MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞════════╡ Precompiles M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════╡ Precompiles M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════╡ Precompiles M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞════════╡ Precompiles MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════╡ Precompiles MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════╡ Precompiles MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞══════════╡ Real life M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞════════╡ Real life M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞══════════╡ Real life M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞══════════╡ Real life MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞════════╡ Real life MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞══════════╡ Real life MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

@antonbaliasnikov antonbaliasnikov force-pushed the abalias-cpr-1055-llvm-regression-tests-with-sanitizer-github-action branch from c0eaea7 to 125c582 Compare August 7, 2024 14:27
@antonbaliasnikov antonbaliasnikov marked this pull request as ready for review August 7, 2024 14:28
@antonbaliasnikov antonbaliasnikov merged commit 4e4c2ba into main Aug 7, 2024
@antonbaliasnikov antonbaliasnikov deleted the abalias-cpr-1055-llvm-regression-tests-with-sanitizer-github-action branch August 7, 2024 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vladimirradosavljevic vladimirradosavljevic vladimirradosavljevic approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@antonbaliasnikov @vladimirradosavljevic