Benchmarks

[kderme]

These benchmarks use a custom criterion, since the official releases don't allow users to determine the number of executions.

Can be executed with

cabal bench cardano-chain-gen --benchmark-options="+RTS -T"

The -T flag enables more measurements from the runtime, mostly gc time

[erikd]

Two problems with this:

• We should probably avoid referencing people's personal GH accounts and instead move this to the input-output-hk account.
• The cabal.project file does not contain a --sha256: .... line for criterion which messes up the Nix build system. This hash can be generated by the nar command in the repo https://github.com/input-output-hk/nix-archive/

Also, have you submitted a PR to upstream criterion?

[kderme]

I should have marked this as WIP.

Also, have you submitted a PR to upstream criterion

My patch may be good enough for what we want, but not good enough to pr criterion. It simply prints a list of measurements for each benchmark like this

Measurement 23 "cardano-chain/bench/rollback multiassets/1 block" [Measured {measTime = 0.12329917199986085, 
measCpuTime = 8.340625999835538e-3, measCycles = 319615318, measIters = 1, measAllocated = 576832, measNumGcs = 
0, measBytesCopied = 14560, measMutatorWallSeconds = 0.12346538299993881, measMutatorCpuSeconds = 
8.77998899989052e-3, measGcWallSeconds = 0.0, measGcCpuSeconds = 0.0},Measured {measTime = 
0.1232295569998314, measCpuTime = 7.885569000109172e-3, measCycles = 319431904, measIters = 1, measAllocated = 
573304, measNumGcs = 0, measBytesCopied = 13928, measMutatorWallSeconds = 0.12339154899996174, 
measMutatorCpuSeconds = 8.429873999943993e-3, measGcWallSeconds = 0.0, measGcCpuSeconds = 0.0},Measured 
{measTime = 0.1220895249998648, measCpuTime = 8.620198000016899e-3, measCycles = 316470088, measIters = 1, 
measAllocated = 571592, measNumGcs = 0, measBytesCopied = 14152, measMutatorWallSeconds = 0.12224910199984151, 
measMutatorCpuSeconds = 9.123472000055699e-3, measGcWallSeconds = 0.0, measGcCpuSeconds = 0.0}]

The list has one meassurement per iteration, but there is no report created with all the benchmarks together.

I'm not sure if it's worth improving the final report of benchmarks, or if we even want to have these benchmark merged or executed on ci. It's just a tool I used to easily test optimizations.