Global figure time

[Kolaru]

Description

Implement a global figure time that is incremented when the figure is displayed. This allow to have some animation shown when the GLMakie windows shows up for example and to have everything in sync across the figure.

For example with this PR

using GLMakie

begin
    fig = Figure()

    axs = [Axis(fig[i, j]) for i in 1:2, j in 1:2]
    xlims = [[-1.2, 0], [0, 1.2]]
    ylims = [[0, 1.2], [-1.2, 0]]
    wavelengths = [
        5 8
        11 14
    ]
    ω = 0.3
    θs = range(0, 2π ; length = 1000)

    for i in 1:2 
        for j in 1:2
            ax = axs[i, j]
            xlims!(ax, xlims[j])
            ylims!(ax, ylims[i])
            w = wavelengths[i, j]

            ρs = @lift [sin(w * (θ + ω * $(fig.time))) for θ in θs]
            xx = @lift $ρs .* cos.(θs)
            yy = @lift $ρs .* sin.(θs)

            lines!(ax, xx, yy)
        end
    end
    
    fig
end

record(fig, "flower.gif", 1:240) do i
    fig.time[] = i/24
end

Gives (whenever the figure is displayed)

Documentation will be straightforward once we are set on the semantic, but I have no idea if it would be possible to have the feature tested.

Type of change

Delete options that do not apply:

• New feature (non-breaking change which adds functionality)

Checklist

• Added an entry in NEWS.md (for new features and breaking changes)
• Added or changed relevant sections in the documentation
• Added unit tests for new algorithms, conversion methods, etc.
• Added reference image tests for new plotting functions, recipes, visual options, etc.

[ffreyer]

I'd rather have something like this implemented as a (render) tick in Events. In GLMakie that could probably just react to screen.render_tick and synchronize with the renderloop, which is preferable to a clock that runs out of sync. For recording that wouldn't work on master since colorbuffer doesn't trigger render_tick, but would after #3246. With the changes you could have

timer = Observable(0)
on(_ -> timer[] = timer[] + time_per_frame, events(fig).tick)

to do the same kind of thing as your example. (Or, if we have a frame counter as part of the tick just map(tick -> f(tick.frames * frame_time), events(fig).tick). But frames are conceptually a bit awkward because render_tick triggering doesn't mean a new frame is drawn. It triggers every so often to update events, and then only draws a new frame if that is actually necessary.)

[Kolaru]

I can indeed sync that on the screen render_tick update.

Since I have to add the callback when the screen is known, I add it at each display. It is not super pretty to have those dangling connections, but I don't know how I could avoid them.

[jkrumbiegel]

I outlined this somewhere else, not sure where, but I also wanted to have some sort of time accessible to the rendering logic. However, I would not want that to be time() in all cases. The reason is that in record you don't care about real time, but I still want to be able to render out videos as they would have looked if played live. So in that case, the timestamps would need to be set to the exact frame times.

[Kolaru]

The way I see how it could be done is to have a frame observable that trigger at each frame and contain the dt to the previous frame, like game engines do.

Something like the following

struct Frame
    n::Observable  # Frame number because why not
    dt::Observable  # Time from the previous frame
    t::Observable  # Real time from fig beginning
end

Then in record or the render loop, you can ignore dt and t values and have a per-frame callback.

However I don't know how it plays with what @ffreyer said, since AFAIK Makie doesn't have a proper "frame" concept right now.

[ffreyer]

No, this is pretty much what I want too. After thinking about it some more I don't think there is a meaningful difference between "I tried to render a new frame and realized it wasn't necessary so I didn't" and "I rendered a new frame" for this. It's just time/work saving, and a bit misleading if someone uses it to judge performance.

My thoughts on the implementation are that we should add a render_tick or delta_time to Events:

Makie.jl/src/types.jl

Lines 41 to 104 in 0329a70

	struct Events
	"""
	The area of the window in pixels, as a `Rect2`.
	"""
	window_area::Observable{Rect2i}
	"""
	The DPI resolution of the window, as a `Float64`.
	"""
	window_dpi::Observable{Float64}
	"""
	The state of the window (open => true, closed => false).
	"""
	window_open::Observable{Bool}
	"""
	Most recently triggered `MouseButtonEvent`. Contains the relevant
	`event.button` and `event.action` (press/release)
	See also [`ispressed`](@ref).
	"""
	mousebutton::Observable{MouseButtonEvent}
	"""
	A Set of all currently pressed mousebuttons.
	"""
	mousebuttonstate::Set{Mouse.Button}
	"""
	The position of the mouse as a `NTuple{2, Float64}`.
	Updates once per event poll/frame.
	"""
	mouseposition::Observable{NTuple{2, Float64}} # why no Vec2?
	"""
	The direction of scroll
	"""
	scroll::Observable{NTuple{2, Float64}} # why no Vec2?
	"""
	Most recently triggered `KeyEvent`. Contains the relevant `event.key` and
	`event.action` (press/repeat/release)
	See also [`ispressed`](@ref).
	"""
	keyboardbutton::Observable{KeyEvent}
	"""
	Contains all currently pressed keys.
	"""
	keyboardstate::Set{Keyboard.Button}
	"""
	Contains the last typed character.
	"""
	unicode_input::Observable{Char}
	"""
	Contains a list of filepaths to files dragged into the scene.
	"""
	dropped_files::Observable{Vector{String}}
	"""
	Whether the Scene window is in focus or not.
	"""
	hasfocus::Observable{Bool}
	"""
	Whether the mouse is inside the window or not.
	"""
	entered_window::Observable{Bool}
	end

This requires changes to the events interface here:
https://github.com/MakieOrg/Makie.jl/blob/master/src/interaction/events.jl
And for GLMakie here, where you can grab screen._rendertick:
https://github.com/MakieOrg/Makie.jl/blob/master/GLMakie/src/events.jl

Regarding naming I would prefer calling the struct something else because Frame sounds like something visible to me, something related to rendering. Maybe just DeltaTime or RenderTick instead? You may also want a raw last_time in there to calculate time deltas, or maybe change screen.render_tick to hold that.

For CairoMakie this should probably only trigger when an image is produced. To be compatible with an interactive Cairo backend it should come from actual drawing, maybe here:

Makie.jl/CairoMakie/src/infrastructure.jl

Lines 10 to 57 in 0329a70

	function cairo_draw(screen::Screen, scene::Scene)
	Cairo.save(screen.context)
	draw_background(screen, scene)
	allplots = Makie.collect_atomic_plots(scene; is_atomic_plot = is_cairomakie_atomic_plot)
	zvals = Makie.zvalue2d.(allplots)
	permute!(allplots, sortperm(zvals))
	# If the backend is not a vector surface (i.e., PNG/ARGB),
	# then there is no point in rasterizing twice.
	should_rasterize = is_vector_backend(screen.surface)
	last_scene = scene
	Cairo.save(screen.context)
	for p in allplots
	check_parent_plots(p) do plot
	to_value(get(plot, :visible, true))
	end || continue
	# only prepare for scene when it changes
	# this should reduce the number of unnecessary clipping masks etc.
	pparent = Makie.parent_scene(p)
	pparent.visible[] || continue
	if pparent != last_scene
	Cairo.restore(screen.context)
	Cairo.save(screen.context)
	prepare_for_scene(screen, pparent)
	last_scene = pparent
	end
	Cairo.save(screen.context)
	# When a plot is too large to save with a reasonable file size on a vector backend,
	# the user can choose to rasterize it when plotting to vector backends, by using the
	# `rasterize` keyword argument. This can be set to a Bool or an Int which describes
	# the density of rasterization (in terms of a direct scaling factor.)
	# TODO: In future, this can also be set to a Tuple{Module, Int} which describes
	# the backend module which should be used to render the scene, and the pixel density
	# at which it should be rendered.
	if to_value(get(p, :rasterize, false)) != false && should_rasterize
	draw_plot_as_image(pparent, screen, p, p[:rasterize][])
	else # draw vector
	draw_plot(pparent, screen, p)
	end
	Cairo.restore(screen.context)
	end
	Cairo.restore(screen.context)
	return
	end

For WGLMakie we would want the same behavior as GLMakie, but I don't know if we have anything set up for that. Maybe the event pipeline is already synced to rendering and you'd just need to add a notification here:

Makie.jl/WGLMakie/src/events.jl

Lines 51 to 58 in 0329a70

	function connect_scene_events!(scene::Scene, comm::Observable)
	e = events(scene)
	on(comm) do msg
	@async try
	@handle msg.mouseposition begin
	x, y = Float64.((mouseposition...,))
	e.mouseposition[] = (x, size(scene)[2] - y)
	end

[jkrumbiegel]

If it was added to events we could sync on it inside recipes and blocks right?

[ffreyer]

Yes, everything that has access to a scene would have access to to that event.

[ffreyer]

Related issue: #3163

[ffreyer]

Closed via #3948