Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test against earlier releases #3288

Closed
wants to merge 10 commits into from
Closed

Test against earlier releases #3288

wants to merge 10 commits into from

Conversation

SimonDanisch
Copy link
Member

The sudden WGLMakie CI failure is really weird, since nothing important should have changed, its not throwing an error and can't be reproduced locally.
This pr will test some earlier versions of Makie, to make sure its not just some unfortunate change in Makie itself.

@MakieBot
Copy link
Collaborator

MakieBot commented Oct 12, 2023

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(display(fig))
using create display create display
GLMakie 11.84s (11.80, 11.88) 0.02+- 1.19s (1.18, 1.20) 0.01+- 895.90ms (883.87, 942.57) 20.94+- 10.41ms (10.33, 10.46) 0.05+- 161.83ms (160.83, 163.16) 0.82+-
master 11.84s (11.82, 11.87) 0.01+- 1.24s (1.23, 1.24) 0.00+- 874.77ms (870.55, 878.57) 3.23+- 10.42ms (10.37, 10.48) 0.04+- 161.26ms (160.78, 162.21) 0.50+-
evaluation -0.05%, -0.01s invariant (-0.31d, 0.57p, 0.02std) -3.68%, -0.04s faster ✓ (-8.59d, 0.00p, 0.00std) +2.36%, 21.13ms slower X (1.41d, 0.04p, 12.08std) -0.06%, -0.01ms invariant (-0.15d, 0.78p, 0.04std) +0.35%, 0.57ms invariant (0.83d, 0.15p, 0.66std)
CairoMakie 10.02s (9.97, 10.11) 0.05+- 1.10s (1.09, 1.11) 0.00+- 210.87ms (209.06, 213.58) 1.86+- 9.60ms (9.57, 9.65) 0.02+- 5.74ms (5.62, 5.80) 0.06+-
master 10.01s (9.96, 10.06) 0.04+- 1.13s (1.12, 1.14) 0.01+- 223.24ms (221.47, 225.90) 1.34+- 9.60ms (9.52, 9.70) 0.08+- 5.73ms (5.61, 5.95) 0.12+-
evaluation +0.08%, 0.01s invariant (0.18d, 0.75p, 0.04std) -2.51%, -0.03s faster ✓ (-4.55d, 0.00p, 0.01std) -5.87%, -12.37ms faster✅ (-7.64d, 0.00p, 1.60std) -0.02%, -0.0ms invariant (-0.03d, 0.96p, 0.05std) +0.27%, 0.02ms invariant (0.16d, 0.77p, 0.09std)
WGLMakie 13.94s (13.71, 14.05) 0.12+- 1.13s (1.11, 1.15) 0.01+- 11.97s (11.80, 12.25) 0.17+- 17.47ms (15.31, 25.81) 3.73+- 1.07s (1.00, 1.13) 0.04+-
master 13.97s (13.88, 14.17) 0.10+- 1.17s (1.14, 1.21) 0.02+- 12.05s (11.85, 12.25) 0.14+- 15.64ms (14.85, 17.70) 1.03+- 1.04s (1.00, 1.07) 0.02+-
evaluation -0.16%, -0.02s invariant (-0.20d, 0.71p, 0.11std) -4.03%, -0.05s faster ✓ (-2.39d, 0.00p, 0.02std) -0.60%, -0.07s invariant (-0.45d, 0.41p, 0.16std) +10.49%, 1.83ms noisy🤷‍♀️ (0.67d, 0.25p, 2.38std) +2.52%, 0.03s invariant (0.83d, 0.16p, 0.03std)

@SimonDanisch
Copy link
Member Author

That ain't it...

@SimonDanisch SimonDanisch deleted the sd/test-ci-2 branch October 12, 2023 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants