Skip to content

Commit

Permalink
docs(benchmarks): add CPython vs Cinder performance comparison
Browse files Browse the repository at this point in the history
  • Loading branch information
@MAMV3x3
MAMV3x3 authored Oct 12, 2023
1 parent dc1656b commit 0de2730
Showing 1 changed file with 349 additions and 0 deletions.
349 changes: 349 additions & 0 deletions benchmarks/benchmark_comparison_cpython_vs_cinder.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,349 @@
result_cpython.json
===================

Performance version: 1.0.9
Report on Linux-6.2.0-33-generic-x86_64-with-glibc2.35
Number of logical CPUs: 8
Start date: 2023-10-01 12:25:22.733168
End date: 2023-10-01 14:12:59.712189

result_cinder.json
==================

Performance version: 1.0.9
Report on Linux-6.2.0-34-generic-x86_64-with-glibc2.35
Number of logical CPUs: 8
Start date: 2023-10-09 13:30:18.298923
End date: 2023-10-09 15:06:56.933948

### 2to3 ###
Mean +- std dev: 743 ms +- 10 ms -> 608 ms +- 22 ms: 1.22x faster
Significant (t=42.26)

### async_generators ###
Mean +- std dev: 1.30 sec +- 0.01 sec -> 0.94 sec +- 0.03 sec: 1.39x faster
Significant (t=87.36)

### async_tree_cpu_io_mixed ###
Mean +- std dev: 3.23 sec +- 0.07 sec -> 0.66 sec +- 0.01 sec: 4.89x faster
Significant (t=263.22)

### async_tree_io ###
Mean +- std dev: 7.36 sec +- 0.26 sec -> 9.59 sec +- 0.11 sec: 1.30x slower
Significant (t=-61.08)

### async_tree_memoization ###
Mean +- std dev: 2.99 sec +- 0.11 sec -> 0.48 sec +- 0.01 sec: 6.21x faster
Significant (t=172.41)

### async_tree_none ###
Mean +- std dev: 2.36 sec +- 0.06 sec -> 0.05 sec +- 0.00 sec: 48.26x faster
Significant (t=309.05)

### asyncio_tcp ###
Mean +- std dev: 796 ms +- 15 ms -> 1402 ms +- 15 ms: 1.76x slower
Significant (t=-220.50)

### asyncio_tcp_ssl ###
Mean +- std dev: 2.83 sec +- 0.10 sec -> 4.16 sec +- 0.10 sec: 1.47x slower
Significant (t=-72.74)

### bench_mp_pool ###
Mean +- std dev: 10.1 ms +- 1.2 ms -> 10.0 ms +- 2.7 ms: 1.01x faster
Not significant

### bench_thread_pool ###
Mean +- std dev: 3.61 ms +- 1.29 ms -> 2.68 ms +- 0.38 ms: 1.35x faster
Significant (t=5.34)

### chameleon ###
Mean +- std dev: 25.4 ms +- 1.4 ms -> 19.2 ms +- 0.2 ms: 1.32x faster
Significant (t=34.88)

### chaos ###
Mean +- std dev: 197 ms +- 10 ms -> 180 ms +- 2 ms: 1.09x faster
Significant (t=12.42)

### comprehensions ###
Mean +- std dev: 75.6 us +- 4.8 us -> 57.1 us +- 1.0 us: 1.32x faster
Significant (t=29.15)

### coroutines ###
Mean +- std dev: 104 ms +- 3 ms -> 41 ms +- 1 ms: 2.53x faster
Significant (t=143.49)

### coverage ###
Mean +- std dev: 2.76 sec +- 0.06 sec -> 0.09 sec +- 0.00 sec: 31.84x faster
Significant (t=319.56)

### create_gc_cycles ###
Mean +- std dev: 2.52 ms +- 0.06 ms -> 2.52 ms +- 0.03 ms: 1.00x faster
Not significant

### crypto_pyaes ###
Mean +- std dev: 258 ms +- 8 ms -> 271 ms +- 2 ms: 1.05x slower
Significant (t=-12.05)

### deepcopy ###
Mean +- std dev: 1.26 ms +- 0.05 ms -> 0.96 ms +- 0.01 ms: 1.31x faster
Significant (t=42.75)

### deepcopy_memo ###
Mean +- std dev: 134 us +- 4 us -> 110 us +- 1 us: 1.22x faster
Significant (t=42.29)

### deepcopy_reduce ###
Mean +- std dev: 11.6 us +- 0.1 us -> 8.5 us +- 0.1 us: 1.37x faster
Significant (t=139.16)

### deltablue ###
Mean +- std dev: 13.1 ms +- 0.5 ms -> 11.9 ms +- 0.3 ms: 1.10x faster
Significant (t=15.65)

### docutils ###
Mean +- std dev: 7.96 sec +- 0.24 sec -> 6.81 sec +- 0.11 sec: 1.17x faster
Significant (t=33.55)

### fannkuch ###
Mean +- std dev: 995 ms +- 5 ms -> 886 ms +- 7 ms: 1.12x faster
Significant (t=102.01)

### float ###
Mean +- std dev: 277 ms +- 4 ms -> 208 ms +- 3 ms: 1.33x faster
Significant (t=101.99)

### gc_traversal ###
Mean +- std dev: 7.13 ms +- 0.08 ms -> 5.14 ms +- 0.03 ms: 1.39x faster
Significant (t=173.45)

### generators ###
Mean +- std dev: 173 ms +- 9 ms -> 146 ms +- 6 ms: 1.18x faster
Significant (t=18.79)

### genshi_text ###
Mean +- std dev: 83.9 ms +- 2.9 ms -> 57.4 ms +- 0.5 ms: 1.46x faster
Significant (t=69.61)

### genshi_xml ###
Mean +- std dev: 185 ms +- 4 ms -> 129 ms +- 2 ms: 1.44x faster
Significant (t=90.28)

### go ###
Mean +- std dev: 412 ms +- 8 ms -> 363 ms +- 4 ms: 1.13x faster
Significant (t=44.00)

### hexiom ###
Mean +- std dev: 20.8 ms +- 1.0 ms -> 17.7 ms +- 0.3 ms: 1.17x faster
Significant (t=21.84)

### html5lib ###
Mean +- std dev: 180 ms +- 18 ms -> 145 ms +- 12 ms: 1.24x faster
Significant (t=12.72)

### json_dumps ###
Mean +- std dev: 41.1 ms +- 0.7 ms -> 32.2 ms +- 0.3 ms: 1.28x faster
Significant (t=88.13)

### json_loads ###
Mean +- std dev: 97.8 us +- 2.4 us -> 65.4 us +- 0.9 us: 1.50x faster
Significant (t=97.85)

### logging_format ###
Mean +- std dev: 24.9 us +- 0.5 us -> 15.6 us +- 0.3 us: 1.60x faster
Significant (t=134.95)

### logging_silent ###
Mean +- std dev: 398 ns +- 12 ns -> 379 ns +- 9 ns: 1.05x faster
Significant (t=9.83)

### logging_simple ###
Mean +- std dev: 22.1 us +- 0.4 us -> 14.2 us +- 0.3 us: 1.55x faster
Significant (t=127.13)

### mako ###
Mean +- std dev: 41.7 ms +- 0.6 ms -> 37.0 ms +- 0.5 ms: 1.13x faster
Significant (t=46.97)

### mdp ###
Mean +- std dev: 6.14 sec +- 0.05 sec -> 5.00 sec +- 0.14 sec: 1.23x faster
Significant (t=60.35)

### meteor_contest ###
Mean +- std dev: 221 ms +- 6 ms -> 183 ms +- 5 ms: 1.21x faster
Significant (t=37.23)

### nbody ###
Mean +- std dev: 248 ms +- 5 ms -> 279 ms +- 8 ms: 1.13x slower
Significant (t=-24.41)

### nqueens ###
Mean +- std dev: 284 ms +- 4 ms -> 228 ms +- 8 ms: 1.25x faster
Significant (t=48.31)

### pathlib ###
Mean +- std dev: 61.5 ms +- 2.1 ms -> 37.2 ms +- 0.4 ms: 1.65x faster
Significant (t=87.13)

### pickle ###
Mean +- std dev: 26.6 us +- 0.5 us -> 19.3 us +- 0.2 us: 1.38x faster
Significant (t=107.42)

### pickle_dict ###
Mean +- std dev: 50.1 us +- 0.4 us -> 38.3 us +- 0.2 us: 1.31x faster
Significant (t=196.35)

### pickle_list ###
Mean +- std dev: 8.61 us +- 0.21 us -> 5.96 us +- 0.05 us: 1.45x faster
Significant (t=93.17)

### pickle_pure_python ###
Mean +- std dev: 1.06 ms +- 0.03 ms -> 0.95 ms +- 0.01 ms: 1.12x faster
Significant (t=28.12)

### pidigits ###
Mean +- std dev: 317 ms +- 3 ms -> 286 ms +- 2 ms: 1.11x faster
Significant (t=76.29)

### pprint_pformat ###
Mean +- std dev: 5.44 sec +- 0.05 sec -> 4.02 sec +- 0.05 sec: 1.35x faster
Significant (t=145.92)

### pprint_safe_repr ###
Mean +- std dev: 2.66 sec +- 0.03 sec -> 1.95 sec +- 0.02 sec: 1.36x faster
Significant (t=148.84)

### pyflate ###
Mean +- std dev: 1.42 sec +- 0.03 sec -> 1.37 sec +- 0.01 sec: 1.04x faster
Significant (t=15.44)

### python_startup ###
Mean +- std dev: 20.1 ms +- 0.3 ms -> 17.0 ms +- 0.1 ms: 1.18x faster
Significant (t=117.65)

### python_startup_no_site ###
Mean +- std dev: 13.4 ms +- 0.6 ms -> 11.4 ms +- 0.2 ms: 1.18x faster
Significant (t=50.18)

### raytrace ###
Mean +- std dev: 1.01 sec +- 0.01 sec -> 0.89 sec +- 0.01 sec: 1.13x faster
Significant (t=52.41)

### regex_compile ###
Mean +- std dev: 410 ms +- 6 ms -> 311 ms +- 2 ms: 1.32x faster
Significant (t=115.36)

### regex_dna ###
Mean +- std dev: 195 ms +- 4 ms -> 212 ms +- 1 ms: 1.09x slower
Significant (t=-33.91)

### regex_effbot ###
Mean +- std dev: 7.90 ms +- 0.06 ms -> 5.88 ms +- 0.03 ms: 1.34x faster
Significant (t=232.26)

### regex_v8 ###
Mean +- std dev: 50.1 ms +- 0.3 ms -> 36.6 ms +- 0.3 ms: 1.37x faster
Significant (t=258.81)

### richards ###
Mean +- std dev: 150 ms +- 5 ms -> 124 ms +- 1 ms: 1.20x faster
Significant (t=36.17)

### richards_super ###
Mean +- std dev: 170 ms +- 3 ms -> 141 ms +- 1 ms: 1.20x faster
Significant (t=66.40)

### scimark_fft ###
Mean +- std dev: 1.21 sec +- 0.01 sec -> 0.93 sec +- 0.00 sec: 1.30x faster
Significant (t=209.21)

### scimark_lu ###
Mean +- std dev: 343 ms +- 15 ms -> 307 ms +- 5 ms: 1.12x faster
Significant (t=18.00)

### scimark_monte_carlo ###
Mean +- std dev: 181 ms +- 2 ms -> 171 ms +- 2 ms: 1.06x faster
Significant (t=28.10)

### scimark_sor ###
Mean +- std dev: 476 ms +- 15 ms -> 410 ms +- 8 ms: 1.16x faster
Significant (t=29.77)

### scimark_sparse_mat_mult ###
Mean +- std dev: 19.0 ms +- 0.3 ms -> 14.1 ms +- 0.1 ms: 1.35x faster
Significant (t=133.04)

### spectral_norm ###
Mean +- std dev: 501 ms +- 15 ms -> 398 ms +- 3 ms: 1.26x faster
Significant (t=51.61)

### sqlglot_normalize ###
Mean +- std dev: 384 ms +- 7 ms -> 307 ms +- 4 ms: 1.25x faster
Significant (t=75.45)

### sqlglot_optimize ###
Mean +- std dev: 174 ms +- 2 ms -> 137 ms +- 2 ms: 1.27x faster
Significant (t=105.64)

### sqlglot_parse ###
Mean +- std dev: 3.96 ms +- 0.08 ms -> 3.63 ms +- 0.08 ms: 1.09x faster
Significant (t=23.32)

### sqlglot_transpile ###
Mean +- std dev: 4.74 ms +- 0.07 ms -> 4.32 ms +- 0.08 ms: 1.10x faster
Significant (t=30.30)

### sqlite_synth ###
Mean +- std dev: 5.87 us +- 0.11 us -> 4.71 us +- 0.11 us: 1.25x faster
Significant (t=58.21)

### telco ###
Mean +- std dev: 24.5 ms +- 1.5 ms -> 14.6 ms +- 0.2 ms: 1.68x faster
Significant (t=50.75)

### tomli_loads ###
Mean +- std dev: 7.23 sec +- 0.08 sec -> 6.51 sec +- 0.04 sec: 1.11x faster
Significant (t=68.00)

### tornado_http ###
Mean +- std dev: 259 ms +- 3 ms -> 229 ms +- 3 ms: 1.13x faster
Significant (t=52.94)

### typing_runtime_protocols ###
Mean +- std dev: 478 us +- 8 us -> 1175 us +- 7 us: 2.46x slower
Significant (t=-516.57)

### unpack_sequence ###
Mean +- std dev: 92.3 ns +- 0.7 ns -> 79.7 ns +- 0.2 ns: 1.16x faster
Significant (t=135.31)

### unpickle ###
Mean +- std dev: 55.2 us +- 3.2 us -> 35.9 us +- 0.2 us: 1.54x faster
Significant (t=46.96)

### unpickle_list ###
Mean +- std dev: 15.9 us +- 0.1 us -> 9.7 us +- 0.1 us: 1.64x faster
Significant (t=358.37)

### unpickle_pure_python ###
Mean +- std dev: 708 us +- 21 us -> 667 us +- 7 us: 1.06x faster
Significant (t=14.91)

### xml_etree_generate ###
Mean +- std dev: 292 ms +- 4 ms -> 215 ms +- 2 ms: 1.36x faster
Significant (t=138.45)

### xml_etree_iterparse ###
Mean +- std dev: 290 ms +- 5 ms -> 224 ms +- 3 ms: 1.29x faster
Significant (t=81.79)

### xml_etree_parse ###
Mean +- std dev: 354 ms +- 5 ms -> 342 ms +- 4 ms: 1.04x faster
Significant (t=15.27)

### xml_etree_process ###
Mean +- std dev: 206 ms +- 8 ms -> 158 ms +- 2 ms: 1.30x faster
Significant (t=42.26)

Skipped 1 benchmarks only in result_cpython.json: dulwich_log

Skipped 8 benchmarks only in result_cinder.json: dask, django_template, sqlalchemy_declarative, sqlalchemy_imperative, sympy_expand, sympy_integrate, sympy_str, sympy_sum

0 comments on commit 0de2730

Please sign in to comment.