time axis support

[visr]

For plotting time series, it would be nice to support Date and DateTime axes.

A minimal example:

using Makie
x = DateTime(2020):Hour(1):DateTime(2020, 1, 2)
y = randn(25)
scatter(x, y)  # -> MethodError: no method matching push!(::Annotations{...}, ::Array{String,1}, ::Tuple{Int64,Float64}; rotation=0.0, textsize=0.8251367139816284, align=(:center, :top), color=:black, font="Dejavu Sans")

I currently work around this by making an "hours since" Float64, using hours(times::Vector{DateTime}) = Dates.value.(times - first(times)) / 3_600_000 , but you easily lose track of time.

Using Plots this works:

using Plots: Plots
Plots.gr()
Plots.scatter(x, y)

Resulting in

[mkborregaard]

@daschw is this functionality not all in PlotUtils so it would mostly trivial to add to Makie?

[daschw]

The functionality for calculating ticks for Date(Time) objects is in PlotUtils optimize_datetime_ticks.
Plotting the objects is implemented in Plots via a type recipe. I guess that should be fairly trivial for Makie.

[SimonDanisch]

Just seems to be a bug ;)

[asinghvi17]

Definitely a bug on our end:

ERROR: MethodError: no method matching push!(::Annotations{...}, ::Array{String,1}, ::Tuple{Int64,Float64}; rotation=0.0, textsize=0.7853086853027342, align=(:center, :top), color=:black, font="Dejavu Sans")
Closest candidates are:
  push!(::Any, ::Any, ::Any) at abstractarray.jl:2158 got unsupported keyword arguments "rotation", "textsize", "align", "color", "font"
  push!(::Any, ::Any, ::Any, ::Any...) at abstractarray.jl:2159 got unsupported keyword arguments "rotation", "textsize", "align", "color", "font"
  push!(::OffsetArrays.OffsetArray{T,1,AA} where AA<:AbstractArray where T, ::Any...) at /Users/anshul/.julia/packages/OffsetArrays/fZSaL/src/OffsetArrays.jl:220 got unsupported keyword arguments "rotation", "textsize", "align", "color", "font"
  ...
Stacktrace:
     Function         Module            Signature
     ────────         ──────            ─────────
[1]  draw_ticks       AbstractPlotting  (::Annotations{...}, ::Int64, ::Tuple{Float64,Float64}, ::Base.Iterators.Zip{Tuple{UnitRange{Int64},Tuple{Array{String,1},Array{String,1}}}}, ::Tuple{Int64,Int64}, ::Tuple{Tuple{Symbol,Float64},Tuple{Symbol,Float64}}, ::Tuple{Nothing,Nothing}, ::Tuple{Symbol,Symbol}, ::Tuple{Float64,Float64}, ::Tuple{Float64,Float64}, ::Tuple{Tuple{Symbol,Symbol},Tuple{Symbol,Symbol}}, ::Tuple{String,String})
at: ~/.julia/dev/AbstractPlotting/src/basic_recipes/axis.jl:287
[2]  #660             AbstractPlotting  (::Int64, ::Tuple{Float64,Float64}, ::Base.Iterators.Zip{Tuple{UnitRange{Int64},Tuple{Array{String,1},Array{String,1}}}})
at: ~/.julia/dev/AbstractPlotting/src/basic_recipes/axis.jl:518
[3]  foreach [i]
at: /Applications/Julia-1.4.app/Contents/Resources/julia/bin/../share/julia/base/abstractarray.jl:1920
[4]  draw_axis2d      AbstractPlotting  (::Annotations{...}, ::LineSegments{...}, ::Tuple{LineSegments{...},LineSegments{...}}, ::Tuple{LineSegments{...},LineSegments{...}}, ::StaticArrays.SArray{Tuple{4,4},Float32,2,16}, ::Float64, ::Tuple{Tuple{Float32,Float32},Tuple{Float32,Float32}}, ::Tuple{Tuple{UnitRange{Int64},Array{Float64,1}},Tuple{Tuple{Array{String,1},Array{String,1}},Array{String,1}}}, ::Tuple{Bool,Bool}, ::Tuple{Bool,Bool}, ::Tuple{Bool,Bool}, ::Tuple{Float64,Float64}, ::Tuple{Tuple{Symbol,Float64},Tuple{Symbol,Float64}}, ::Tuple{Nothing,Nothing}, ::Tuple{Int64,Int64}, ::Tuple{Tuple{Symbol,Float64},Tuple{Symbol,Float64}}, ::Tuple{Nothing,Nothing}, ::Tuple{Symbol,Symbol}, ::Tuple{Int64,Int64}, ::Tuple{Float64,Float64}, ::Tuple{Tuple{Symbol,Symbol},Tuple{Symbol,Symbol}}, ::Tuple{String,String}, ::Int64, ::Int64, ::Tuple{Int64,Int64}, ::Tuple{Tuple{Symbol,Float64},Tuple{Symbol,Float64}}, ::Tuple{Nothing,Nothing}, ::Tuple{Float64,Float64}, ::Float64, ::Symbol, ::Nothing, ::Nothing, ::Bool, ::Float64, ::Tuple{Tuple{Bool,Bool},Tuple{Bool,Bool}}, ::Tuple{String,String}, ::Tuple{Symbol,Symbol}, ::Tuple{Int64,Int64}, ::Tuple{Float64,Float64}, ::Tuple{Tuple{Symbol,Symbol},Tuple{Symbol,Symbol}}, ::Tuple{String,String}, ::Nothing)
at: ~/.julia/dev/AbstractPlotting/src/basic_recipes/axis.jl:516
[5]  map_once         AbstractPlotting  (::Function, ::Observables.Observable{Annotations{...}}, ::Observables.Observable{LineSegments{...}}, ::Vararg{Observables.Observable,N} where N)
at: ~/.julia/dev/AbstractPlotting/src/interaction/nodes.jl:83
[6]  plot!            AbstractPlotting  (::Scene, ::Type{Axis2D{...}}, ::Attributes, ::Observables.Observable{GeometryTypes.HyperRectangle{3,Float32}})
at: ~/tmp/MakieSys.so:-1
[7]  #axis2d!#622     AbstractPlotting  (::Base.Iterators.Pairs{Symbol,NamedTuple{(:ranges, :labels),Tuple{Observables.Observable{Any},Observables.Observable{Any}}},Tuple{Symbol},NamedTuple{(:ticks,),Tuple{NamedTuple{(:ranges, :labels),Tuple{Observables.Observable{Any},Observables.Observable{Any}}}}}}, ::typeof(axis2d!), ::Scene, ::Attributes, ::Observables.Observable{GeometryTypes.HyperRectangle{3,Float32}})
at: ~/.julia/dev/AbstractPlotting/src/recipes.jl:38
[8]  add_axis!        AbstractPlotting  (::Scene, ::Attributes)
at: ~/tmp/MakieSys.so:-1
[9]  plot!            AbstractPlotting  (::Scene, ::Type{Scatter{...}}, ::Attributes, ::Tuple{Observables.Observable{StepRange{DateTime,Hour}},Observables.Observable{Array{Float64,1}}}, ::Observables.Observable{Tuple{Array{Point{2,Float32},1}}})
at: ~/.julia/dev/AbstractPlotting/src/interfaces.jl:659
[10] #plot!#206       AbstractPlotting  (::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(plot!), ::Scene, ::Type{Scatter{...}}, ::Attributes, ::StepRange{DateTime,Hour}, ::Vararg{Any,N} where N)
at: ~/.julia/dev/AbstractPlotting/src/interfaces.jl:572
[11] plot!            AbstractPlotting  (::Scene, ::Type{Scatter{...}}, ::Attributes, ::StepRange{DateTime,Hour}, ::Array{Float64,1})
at: ~/.julia/dev/AbstractPlotting/src/interfaces.jl:541
[12] #scatter#147     AbstractPlotting  (::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(scatter), ::StepRange{DateTime,Hour}, ::Vararg{Any,N} where N)
at: ~/.julia/dev/AbstractPlotting/src/recipes.jl:15
[13] scatter          AbstractPlotting  (::StepRange{DateTime,Hour}, ::Array{Float64,1})
at: ~/.julia/dev/AbstractPlotting/src/recipes.jl:13

is the full stacktrace.

[SimonDanisch]

I dont know what I was thinking when I was writing the code handling the tick integration :D We should just throw it away and integrate the MakieLayout axes^^

[mkborregaard]

Yes it looks to me as if the MakieLayout functionality could largely just replace the functionality in AbstractPlotting

[cmey]

Bump, could use this too!
How would you use MakieLayout to workaround this?

[jkrumbiegel]

The problem I see is always the same. What does it mean for an axis to plot time data? In GLMakie, everything boils down to 3D projections of floating point values at the end, so the question is, do we work with floating point values in the background and only show times on the axis (that's a tick formatting question then) or do we somehow really enforce the "time-ness" of the axis. This maps to other problems as well, if you have a categorical axis ["dog", "cat", "mouse"], should you then only be allowed to plot values that adhere to this categorization? Where do you enforce that, on the lowest level in the scene projections?

To me, the non-interactive plotting softwares "hide" this problem from you because they can just prepare a single finished product, which is only built once all parts are known. I think it's much more difficult to say what should happen if you plot something else into a subplot that currently has an x-axis with time scaling.

Currently with MakieLayout you would just make a custom ticks type which can read the float values underlying the date values and then compute good ticks in the date domain (for example with the existing plotutils function), then give out the ticks as float values and finally format with a date-aware tick formatter. This could obviously be done by Makie itself when you plot something with a time axis, it's just that the time axis is not "really" a time axis, it's a normal axis with dates as tick labels.

[AlexisRenchon]

I think if we don't add a time axis support, we should at least have an example in the docs of custom ticks as @jkrumbiegel describes, for users who are not so experienced but want to plot time series easily

[chris-b1]

Just to put a complete example in this issue of what worked for me (may be a better way to accomplish this). I'm only solving the formatting, still using the default LinearTicks(4) for spacing

using CairoMakie
using DataFrames
using Dates

df = DataFrame(dates=Date(2020, 1, 1) : Day(1) : Date(2020, 1, 31), values=1:31);
plt = lines(df.dates, df.values)
# replace "m/d/Y" with desired format
plt.axis.xtickformat = xs -> Dates.format.(df.dates[convert.(Int, xs .+ 1)], "m/d/Y")
plt

[AlexisRenchon]

I didn't know you could plot Date format type data
I have been doing:

using GLMakie
using DataFrames
using Dates
using PlotUtils: optimize_ticks

df = DataFrame(dates=Date(2020, 1, 1) : Day(1) : Date(2020, 1, 31), values=1:31);
dateticks = optimize_ticks(df.dates[1], df.dates[end])[1]

fig = Figure()
ax1 = Axis(fig[1,1])
plt = lines!(ax1, datetime2rata.(df.dates), df.values)
ax1.xticks[] = (datetime2rata.(dateticks) , Dates.format.(dateticks, "mm/dd/yyyy"));

[ValentinKaisermayer]

There is a problem with this since Makie does not handle large integers, as Dates.value(dt::Datees.DateTime) will produce, very well, see #1373.

[AlexisRenchon]

Yeah you should use datetime2unix instead of datetime2rata
A note: I know that Matlab dynamically adapt the date format (e.g., dd/mm/yyyy or HH:MM), depending on the time period of your axis (e.g., if it is less than a day, minutes format, is years, mm/yyyy...) that's useful if you zoom in (change ticks and format as you zoom in), or if you have a slider or menu that change the time period of your axis. Would be neat to implement something similar eventually

[robsmith11]

Is there any reasonable way currently to create plots with a datetime axis?

Plotly does this very well and adapts the labels based on zoom level, so that you'll see years, months, days, HH:MM, or even HH:MM:SS.sss up to nanoseconds. It's useful when zooming in on time series to see exactly when an event occurred without having to mentally convert between a timestamp and a numeric offset.

See example time-series plots here and try zooming in:
https://plotly.com/javascript/time-series/

[ValentinKaisermayer]

I use this:

using Makie
using Dates
using GLMakie
import PlotUtils: optimize_datetime_tic

function timestamp_plot(timestamps, t0::Dates.DateTime)
    return Dates.value.(timestamps) .- Dates.value(t0)
end

struct DateTimeTicks
    t0::Dates.DateTime
end

function Makie.get_ticks(t::DateTimeTicks, any_scale, ::Makie.Automatic, vmin, vmax)
    dateticks, dateticklabels = optimize_datetime_ticks(
        Dates.value(Dates.DateTime(Dates.Millisecond(Int64(vmin)) + t.t0)),
        Dates.value(Dates.DateTime(Dates.Millisecond(Int64(vmax)) + t.t0)),
    )
    return dateticks .- Dates.value(t.t0), dateticklabels
end

timestamps = DateTime(2023):Minute(15):DateTime(2023,01,07)
data = cumsum(randn(length(timestamps))

t0 = timestamps[1]

fig = Figure()
ax1 = Axis(xticks=DateTimeTicks(t0))
lines!(timestamp_plot(timestamps, t0), data)
fig

[robsmith11]

Thanks @ValentinKaisermayer but I think a few things got cut off in the example code. For example there's a missing ) and t0 is undefined in the following line:

ax1 = Axis(xticks=DateTimeTicks(t0)

Most of it was obvious how to fix, but I couldn't figure out that line.

[jkrumbiegel]

I think you choose it yourself, the logic circumvents Makie's problem with loss of precision when you convert normal Dates to Float32s. If you subtract a t0 close to your values, then the precision will often be enough (there are fewer and fewer floats, the further away from 0 you go)

[robsmith11]

Ah thanks I see. Looks like the DateTime situation will be a lot better after #2573 merges.

[jkrumbiegel]

At least for CairoMakie, yes

[ValentinKaisermayer]

I updated my code example.
The idea is to use the internal representation of a DateTime (milliseconds), but offset it by some number in the range of the data.

[rkube]

@ValentinKaisermayer Thanks for posting, there are some errors when running your code example, this code should produce a plot:

using Makie
using Dates
using CairoMakie
import PlotUtils: optimize_datetime_ticks

function timestamp_plot(timestamps, t0::Dates.DateTime)
    return Dates.value.(timestamps) .- Dates.value(t0)
end

struct DateTimeTicks
    t0::Dates.DateTime
end

function Makie.get_ticks(t::DateTimeTicks, any_scale, ::Makie.Automatic, vmin, vmax)
    dateticks, dateticklabels = optimize_datetime_ticks(
        Dates.value(Dates.DateTime(Dates.Millisecond(Int64(vmin)) + t.t0)),
        Dates.value(Dates.DateTime(Dates.Millisecond(Int64(vmax)) + t.t0)),
    )
    return dateticks .- Dates.value(t.t0), dateticklabels
end

timestamps = DateTime(2023, 01, 01):Day(1):DateTime(2023, 03, 01)
data = cumsum(randn(length(timestamps)))

t0 = timestamps[1]

fig = Figure()
ax1 = Axis(fig[1, 1], xticks=DateTimeTicks(timestamps[1]))
lines!(ax1, timestamp_plot(timestamps, t0), data)
fig

[FelipeLema]

the previous example does not seem to be working now (with WGLMakie), I'm getting the following error my bad ... I had just messed up the REPL

[FelipeLema]

I can do a PR using the provided sample code

[rafaqz]

I'm writing plot recipes for DimensionalData.jl and this not working is pretty damaging to the prospect of doing that comprehensively. We also need Unitful.jl valued axes and anything similar to be possible.

Setting the ticks manually means we cant just pass dates as return values of convert_arguments.

[mkborregaard]

@jkrumbiegel why can't we just plug in the functionality in PlotUtils in the same way that Plots does? It's not just a question of formatting but also of having tick positions that make sense in a time context, but PlotUtils does that. Is this inherently more complicated in an interactive setting than any other type of axis?

[SimonDanisch]

It needs to integrate with the whole argument conversion pipeline, and it needs to have access to the axis at the same time, which isn't currently possible, and also introduces some kind of chicken / egg problem, since the axis needs a plot to figure out what axis to create, while the plot needs the axis to do the conversion.
The ticks formatting/placement isn't the problem and already exists in #3226 and ideas for solving the conversion problems also exist.
Hopefully I'll have time soon to get this merged in January!

[mkborregaard]

ah great, thanks for clarifying!

[jkrumbiegel]

The problem is not the logic where to place what ticks. The problem is that typical time data has too high a resolution for Float32 conversion to work. So we need to add dynamic rescaling in the backend, and it's not 100% clear how to do it

Ah sorry didn't see Simon replied already :)

[mkborregaard]

I guess accuracy isn't that crucial though? Could one just recalculate the ticks at redraw and ignore the accumulating errors? Or is that not the issue at all and it's more a question of where in the pipeline to insert the reconvert?

[visr]

Fixed in Makie v0.21. Great effort!

[kosukesando]

Is this actually fixed?
Stealing the examples above, the following fails (in Pluto)

begin
  df = DataFrame(dates=Date(2020, 1, 1) : Day(1) : Date(2020, 1, 31), values=1:31);
  plt = lines(df.dates, df.values)
  plt.axis.xtickformat = values -> ["foo" for value in values]
  plt
end

with

MethodError: Cannot convert an object of type var"#73#75" to an object of type Vector{Float64}

plt.axis.ytickformat = values -> ["bar" for value in values] works as expected.
Am I not understanding how tickformat works, or is this a Pluto issue on my end?

EDIT:
Okay, so ticks for Date and Time are still unsupported, which is another issue.

Now this produces a plot, but the xtick isn't changed at all.

begin
  df = DataFrame(dates=DateTime(2020, 1, 1) : Hour(1) : DateTime(2020, 1, 2), values=1:25);
  plt = lines(df.dates, df.values)
  plt.axis.xtickformat = values -> ["foo" for value in values]
  plt.axis.ytickformat = values -> ["bar" for value in values]
  plt
end