diff --git a/benchmarks/benchmark_comparison_cpython_vs_cinder.txt b/benchmarks/benchmark_comparison_cpython_vs_cinder.txt new file mode 100644 index 00000000000..75c4ea94a1b --- /dev/null +++ b/benchmarks/benchmark_comparison_cpython_vs_cinder.txt @@ -0,0 +1,349 @@ +result_cpython.json +=================== + +Performance version: 1.0.9 +Report on Linux-6.2.0-33-generic-x86_64-with-glibc2.35 +Number of logical CPUs: 8 +Start date: 2023-10-01 12:25:22.733168 +End date: 2023-10-01 14:12:59.712189 + +result_cinder.json +================== + +Performance version: 1.0.9 +Report on Linux-6.2.0-34-generic-x86_64-with-glibc2.35 +Number of logical CPUs: 8 +Start date: 2023-10-09 13:30:18.298923 +End date: 2023-10-09 15:06:56.933948 + +### 2to3 ### +Mean +- std dev: 743 ms +- 10 ms -> 608 ms +- 22 ms: 1.22x faster +Significant (t=42.26) + +### async_generators ### +Mean +- std dev: 1.30 sec +- 0.01 sec -> 0.94 sec +- 0.03 sec: 1.39x faster +Significant (t=87.36) + +### async_tree_cpu_io_mixed ### +Mean +- std dev: 3.23 sec +- 0.07 sec -> 0.66 sec +- 0.01 sec: 4.89x faster +Significant (t=263.22) + +### async_tree_io ### +Mean +- std dev: 7.36 sec +- 0.26 sec -> 9.59 sec +- 0.11 sec: 1.30x slower +Significant (t=-61.08) + +### async_tree_memoization ### +Mean +- std dev: 2.99 sec +- 0.11 sec -> 0.48 sec +- 0.01 sec: 6.21x faster +Significant (t=172.41) + +### async_tree_none ### +Mean +- std dev: 2.36 sec +- 0.06 sec -> 0.05 sec +- 0.00 sec: 48.26x faster +Significant (t=309.05) + +### asyncio_tcp ### +Mean +- std dev: 796 ms +- 15 ms -> 1402 ms +- 15 ms: 1.76x slower +Significant (t=-220.50) + +### asyncio_tcp_ssl ### +Mean +- std dev: 2.83 sec +- 0.10 sec -> 4.16 sec +- 0.10 sec: 1.47x slower +Significant (t=-72.74) + +### bench_mp_pool ### +Mean +- std dev: 10.1 ms +- 1.2 ms -> 10.0 ms +- 2.7 ms: 1.01x faster +Not significant + +### bench_thread_pool ### +Mean +- std dev: 3.61 ms +- 1.29 ms -> 2.68 ms +- 0.38 ms: 1.35x faster +Significant (t=5.34) + +### chameleon ### +Mean +- std dev: 25.4 ms +- 1.4 ms -> 19.2 ms +- 0.2 ms: 1.32x faster +Significant (t=34.88) + +### chaos ### +Mean +- std dev: 197 ms +- 10 ms -> 180 ms +- 2 ms: 1.09x faster +Significant (t=12.42) + +### comprehensions ### +Mean +- std dev: 75.6 us +- 4.8 us -> 57.1 us +- 1.0 us: 1.32x faster +Significant (t=29.15) + +### coroutines ### +Mean +- std dev: 104 ms +- 3 ms -> 41 ms +- 1 ms: 2.53x faster +Significant (t=143.49) + +### coverage ### +Mean +- std dev: 2.76 sec +- 0.06 sec -> 0.09 sec +- 0.00 sec: 31.84x faster +Significant (t=319.56) + +### create_gc_cycles ### +Mean +- std dev: 2.52 ms +- 0.06 ms -> 2.52 ms +- 0.03 ms: 1.00x faster +Not significant + +### crypto_pyaes ### +Mean +- std dev: 258 ms +- 8 ms -> 271 ms +- 2 ms: 1.05x slower +Significant (t=-12.05) + +### deepcopy ### +Mean +- std dev: 1.26 ms +- 0.05 ms -> 0.96 ms +- 0.01 ms: 1.31x faster +Significant (t=42.75) + +### deepcopy_memo ### +Mean +- std dev: 134 us +- 4 us -> 110 us +- 1 us: 1.22x faster +Significant (t=42.29) + +### deepcopy_reduce ### +Mean +- std dev: 11.6 us +- 0.1 us -> 8.5 us +- 0.1 us: 1.37x faster +Significant (t=139.16) + +### deltablue ### +Mean +- std dev: 13.1 ms +- 0.5 ms -> 11.9 ms +- 0.3 ms: 1.10x faster +Significant (t=15.65) + +### docutils ### +Mean +- std dev: 7.96 sec +- 0.24 sec -> 6.81 sec +- 0.11 sec: 1.17x faster +Significant (t=33.55) + +### fannkuch ### +Mean +- std dev: 995 ms +- 5 ms -> 886 ms +- 7 ms: 1.12x faster +Significant (t=102.01) + +### float ### +Mean +- std dev: 277 ms +- 4 ms -> 208 ms +- 3 ms: 1.33x faster +Significant (t=101.99) + +### gc_traversal ### +Mean +- std dev: 7.13 ms +- 0.08 ms -> 5.14 ms +- 0.03 ms: 1.39x faster +Significant (t=173.45) + +### generators ### +Mean +- std dev: 173 ms +- 9 ms -> 146 ms +- 6 ms: 1.18x faster +Significant (t=18.79) + +### genshi_text ### +Mean +- std dev: 83.9 ms +- 2.9 ms -> 57.4 ms +- 0.5 ms: 1.46x faster +Significant (t=69.61) + +### genshi_xml ### +Mean +- std dev: 185 ms +- 4 ms -> 129 ms +- 2 ms: 1.44x faster +Significant (t=90.28) + +### go ### +Mean +- std dev: 412 ms +- 8 ms -> 363 ms +- 4 ms: 1.13x faster +Significant (t=44.00) + +### hexiom ### +Mean +- std dev: 20.8 ms +- 1.0 ms -> 17.7 ms +- 0.3 ms: 1.17x faster +Significant (t=21.84) + +### html5lib ### +Mean +- std dev: 180 ms +- 18 ms -> 145 ms +- 12 ms: 1.24x faster +Significant (t=12.72) + +### json_dumps ### +Mean +- std dev: 41.1 ms +- 0.7 ms -> 32.2 ms +- 0.3 ms: 1.28x faster +Significant (t=88.13) + +### json_loads ### +Mean +- std dev: 97.8 us +- 2.4 us -> 65.4 us +- 0.9 us: 1.50x faster +Significant (t=97.85) + +### logging_format ### +Mean +- std dev: 24.9 us +- 0.5 us -> 15.6 us +- 0.3 us: 1.60x faster +Significant (t=134.95) + +### logging_silent ### +Mean +- std dev: 398 ns +- 12 ns -> 379 ns +- 9 ns: 1.05x faster +Significant (t=9.83) + +### logging_simple ### +Mean +- std dev: 22.1 us +- 0.4 us -> 14.2 us +- 0.3 us: 1.55x faster +Significant (t=127.13) + +### mako ### +Mean +- std dev: 41.7 ms +- 0.6 ms -> 37.0 ms +- 0.5 ms: 1.13x faster +Significant (t=46.97) + +### mdp ### +Mean +- std dev: 6.14 sec +- 0.05 sec -> 5.00 sec +- 0.14 sec: 1.23x faster +Significant (t=60.35) + +### meteor_contest ### +Mean +- std dev: 221 ms +- 6 ms -> 183 ms +- 5 ms: 1.21x faster +Significant (t=37.23) + +### nbody ### +Mean +- std dev: 248 ms +- 5 ms -> 279 ms +- 8 ms: 1.13x slower +Significant (t=-24.41) + +### nqueens ### +Mean +- std dev: 284 ms +- 4 ms -> 228 ms +- 8 ms: 1.25x faster +Significant (t=48.31) + +### pathlib ### +Mean +- std dev: 61.5 ms +- 2.1 ms -> 37.2 ms +- 0.4 ms: 1.65x faster +Significant (t=87.13) + +### pickle ### +Mean +- std dev: 26.6 us +- 0.5 us -> 19.3 us +- 0.2 us: 1.38x faster +Significant (t=107.42) + +### pickle_dict ### +Mean +- std dev: 50.1 us +- 0.4 us -> 38.3 us +- 0.2 us: 1.31x faster +Significant (t=196.35) + +### pickle_list ### +Mean +- std dev: 8.61 us +- 0.21 us -> 5.96 us +- 0.05 us: 1.45x faster +Significant (t=93.17) + +### pickle_pure_python ### +Mean +- std dev: 1.06 ms +- 0.03 ms -> 0.95 ms +- 0.01 ms: 1.12x faster +Significant (t=28.12) + +### pidigits ### +Mean +- std dev: 317 ms +- 3 ms -> 286 ms +- 2 ms: 1.11x faster +Significant (t=76.29) + +### pprint_pformat ### +Mean +- std dev: 5.44 sec +- 0.05 sec -> 4.02 sec +- 0.05 sec: 1.35x faster +Significant (t=145.92) + +### pprint_safe_repr ### +Mean +- std dev: 2.66 sec +- 0.03 sec -> 1.95 sec +- 0.02 sec: 1.36x faster +Significant (t=148.84) + +### pyflate ### +Mean +- std dev: 1.42 sec +- 0.03 sec -> 1.37 sec +- 0.01 sec: 1.04x faster +Significant (t=15.44) + +### python_startup ### +Mean +- std dev: 20.1 ms +- 0.3 ms -> 17.0 ms +- 0.1 ms: 1.18x faster +Significant (t=117.65) + +### python_startup_no_site ### +Mean +- std dev: 13.4 ms +- 0.6 ms -> 11.4 ms +- 0.2 ms: 1.18x faster +Significant (t=50.18) + +### raytrace ### +Mean +- std dev: 1.01 sec +- 0.01 sec -> 0.89 sec +- 0.01 sec: 1.13x faster +Significant (t=52.41) + +### regex_compile ### +Mean +- std dev: 410 ms +- 6 ms -> 311 ms +- 2 ms: 1.32x faster +Significant (t=115.36) + +### regex_dna ### +Mean +- std dev: 195 ms +- 4 ms -> 212 ms +- 1 ms: 1.09x slower +Significant (t=-33.91) + +### regex_effbot ### +Mean +- std dev: 7.90 ms +- 0.06 ms -> 5.88 ms +- 0.03 ms: 1.34x faster +Significant (t=232.26) + +### regex_v8 ### +Mean +- std dev: 50.1 ms +- 0.3 ms -> 36.6 ms +- 0.3 ms: 1.37x faster +Significant (t=258.81) + +### richards ### +Mean +- std dev: 150 ms +- 5 ms -> 124 ms +- 1 ms: 1.20x faster +Significant (t=36.17) + +### richards_super ### +Mean +- std dev: 170 ms +- 3 ms -> 141 ms +- 1 ms: 1.20x faster +Significant (t=66.40) + +### scimark_fft ### +Mean +- std dev: 1.21 sec +- 0.01 sec -> 0.93 sec +- 0.00 sec: 1.30x faster +Significant (t=209.21) + +### scimark_lu ### +Mean +- std dev: 343 ms +- 15 ms -> 307 ms +- 5 ms: 1.12x faster +Significant (t=18.00) + +### scimark_monte_carlo ### +Mean +- std dev: 181 ms +- 2 ms -> 171 ms +- 2 ms: 1.06x faster +Significant (t=28.10) + +### scimark_sor ### +Mean +- std dev: 476 ms +- 15 ms -> 410 ms +- 8 ms: 1.16x faster +Significant (t=29.77) + +### scimark_sparse_mat_mult ### +Mean +- std dev: 19.0 ms +- 0.3 ms -> 14.1 ms +- 0.1 ms: 1.35x faster +Significant (t=133.04) + +### spectral_norm ### +Mean +- std dev: 501 ms +- 15 ms -> 398 ms +- 3 ms: 1.26x faster +Significant (t=51.61) + +### sqlglot_normalize ### +Mean +- std dev: 384 ms +- 7 ms -> 307 ms +- 4 ms: 1.25x faster +Significant (t=75.45) + +### sqlglot_optimize ### +Mean +- std dev: 174 ms +- 2 ms -> 137 ms +- 2 ms: 1.27x faster +Significant (t=105.64) + +### sqlglot_parse ### +Mean +- std dev: 3.96 ms +- 0.08 ms -> 3.63 ms +- 0.08 ms: 1.09x faster +Significant (t=23.32) + +### sqlglot_transpile ### +Mean +- std dev: 4.74 ms +- 0.07 ms -> 4.32 ms +- 0.08 ms: 1.10x faster +Significant (t=30.30) + +### sqlite_synth ### +Mean +- std dev: 5.87 us +- 0.11 us -> 4.71 us +- 0.11 us: 1.25x faster +Significant (t=58.21) + +### telco ### +Mean +- std dev: 24.5 ms +- 1.5 ms -> 14.6 ms +- 0.2 ms: 1.68x faster +Significant (t=50.75) + +### tomli_loads ### +Mean +- std dev: 7.23 sec +- 0.08 sec -> 6.51 sec +- 0.04 sec: 1.11x faster +Significant (t=68.00) + +### tornado_http ### +Mean +- std dev: 259 ms +- 3 ms -> 229 ms +- 3 ms: 1.13x faster +Significant (t=52.94) + +### typing_runtime_protocols ### +Mean +- std dev: 478 us +- 8 us -> 1175 us +- 7 us: 2.46x slower +Significant (t=-516.57) + +### unpack_sequence ### +Mean +- std dev: 92.3 ns +- 0.7 ns -> 79.7 ns +- 0.2 ns: 1.16x faster +Significant (t=135.31) + +### unpickle ### +Mean +- std dev: 55.2 us +- 3.2 us -> 35.9 us +- 0.2 us: 1.54x faster +Significant (t=46.96) + +### unpickle_list ### +Mean +- std dev: 15.9 us +- 0.1 us -> 9.7 us +- 0.1 us: 1.64x faster +Significant (t=358.37) + +### unpickle_pure_python ### +Mean +- std dev: 708 us +- 21 us -> 667 us +- 7 us: 1.06x faster +Significant (t=14.91) + +### xml_etree_generate ### +Mean +- std dev: 292 ms +- 4 ms -> 215 ms +- 2 ms: 1.36x faster +Significant (t=138.45) + +### xml_etree_iterparse ### +Mean +- std dev: 290 ms +- 5 ms -> 224 ms +- 3 ms: 1.29x faster +Significant (t=81.79) + +### xml_etree_parse ### +Mean +- std dev: 354 ms +- 5 ms -> 342 ms +- 4 ms: 1.04x faster +Significant (t=15.27) + +### xml_etree_process ### +Mean +- std dev: 206 ms +- 8 ms -> 158 ms +- 2 ms: 1.30x faster +Significant (t=42.26) + +Skipped 1 benchmarks only in result_cpython.json: dulwich_log + +Skipped 8 benchmarks only in result_cinder.json: dask, django_template, sqlalchemy_declarative, sqlalchemy_imperative, sympy_expand, sympy_integrate, sympy_str, sympy_sum