Unit support for Axes & Recipes, a.k.a axis converts (#3226)

* take over most of the work from #1347

* add typed argument conversion (#3565)

* add typed argument conversion

* fix volume

* add function to get available conversions

* make conversion apply more narrowly

* more cleanly separate recursion in convert_arguments

* clean up

* allow to get axis before creating a plot

* clean up

* fix tests

* bring back dim converts (axis_convert)

* update tests

* fix tests and work around conversion problems

* fix WGLMakie

* fix errors

* clean up conversion pipeline

* fix tests

* add changelog entry

* disable project run

* improve performance slightly

* might as well use array

* tmp

* wip

* implement axis convert recursion

* fix tests

* fix datashader

* fix datashader

* move unitful integration

* fix performance regression!?

* fix merge & new date time improvements

* fix scaling test

* remove test false

* clean up

* converts shouldnt be here

* move axis converts to scene

* further refactor [skip ci]

* finish refactor for AxisConversion type

* allow limit setting and ticks

* make tests less noisy

* cleanup

* clean up and fix unitful/date conversion

* make sure all tests work correctly

* remove rand

* rename, clean up and make axis spec work

* clean up and test new conversion pipeline

* undo feature deletion, don't reintroduce Rect2f

* be explicit about Volume Interval types

* minor docstring cleanup

* try to clarify new conversion docstrings

* remove convert_arguments_typed in favor of types_for_plot_arguments

* fix remaining bugs for conversion simplification

* fix ticks not updating

* fix specapi

* fix qqnorm

* clean up types_for_plot_arguments

* fix tuple conversion

* try to fix compile time regression

* try to fix compile time regression

* clean up and introduce expand_dimensions

* fix #3655 and clean up convert_arguments + add tests

* fix #3509 and add tests for

* clean up observables and more docs

* final rename

* fix docs

* cleanup

* small clean up

* small doc improvements

* improve docs

* fix docs

* try relative link

* try without .md

* take out link

* try fix

---------

Co-authored-by: ffreyer <frederic481994@hotmail.de>

.codecov.yml

@@ -1 +1,4 @@
 comment: false
+coverage:
+  status:
+    project: false

CHANGELOG.md

@@ -1,16 +1,17 @@
 # Changelog
 
-## [0.20.9] - 2024-03-29
+## [Unreleased]
 
 - Improved thread safety of rendering with CairoMakie (independent `Scene`s only) by locking FreeType handles [#3777](https://github.com/MakieOrg/Makie.jl/pull/3777).
 
 ## [0.21.0] - 2024-03-0X
 
-- Add `voxels` plot [#3527](https://github.com/MakieOrg/Makie.jl/pull/3527)
+- Add `voxels` plot [#3527](https://github.com/MakieOrg/Makie.jl/pull/3527).
 - Added supported markers hint to unsupported marker warn message [#3666](https://github.com/MakieOrg/Makie.jl/pull/3666).
 - Fixed bug in CairoMakie line drawing when multiple successive points had the same color [#3712](https://github.com/MakieOrg/Makie.jl/pull/3712).
 - Remove StableHashTraits in favor of calculating hashes directly with CRC32c [#3667](https://github.com/MakieOrg/Makie.jl/pull/3667).
-- **Potentially breaking** Added a new `@recipe` variant with a `begin end` block. This variant is considered **internal** to Makie for now and could be refactored in patch releases. If you want to use it already on an experimental basis, do so only with Makie pinned to a patch version to avoid breakage. The new syntax allows documenting attributes directly in the `@recipe` definition and validating that all user-passed attributes are known whenever a plot is created. All of Makie's recipes have been ported over to this new syntax and will therefore throw errors when they receive invalid attributes. This is not breaking in the sense that the API changes, but existing user code is likely to break because of misspelled attribute names etc. that have so far gone unnoticed. A likely culprit is the forwarding of attributes to child plots by splatting the parent attributes into it like `some_plot!(...; attrs...)`. To fix this, make sure that you are only passing along valid attributes to the child plots in your recipes.
+- **Breaking (sort of)** Added a new `@recipe` variant which allows documenting attributes directly where they are defined and validating that all attributes are known whenever a plot is created. This is not breaking in the sense that the API changes, but user code is likely to break because of misspelled attribute names etc. that have so far gone unnoticed.
+- Add axis converts, enabling unit/categorical support and more [#3226](https://github.com/MakieOrg/Makie.jl/pull/3226).
 - **Breaking** Streamlined `data_limits` and `boundingbox` [#3671](https://github.com/MakieOrg/Makie.jl/pull/3671)
   - `data_limits` now only considers plot positions, completely ignoring transformations
   - `boundingbox(p::Text)` is deprecated in favor of `boundingbox(p::Text, p.markerspace[])`. The more internal methods use `string_boundingbox(p)`. [#3723](https://github.com/MakieOrg/Makie.jl/pull/3723)
@@ -39,6 +40,18 @@
 - Fixed an issue with the texture atlas not updating in WGLMakie after display, causing new symbols to not show up [#3737](https://github.com/MakieOrg/Makie.jl/pull/3737)
 - Added `linecap` and `joinstyle` attributes for lines and linesegments. Also normalized `miter_limit` to 60° across all backends. [#3771](https://github.com/MakieOrg/Makie.jl/pull/3771)
 
+## [0.20.9] - 2024-03-29
+
+- Added supported markers hint to unsupported marker warn message [#3666](https://github.com/MakieOrg/Makie.jl/pull/3666).
+- Fixed bug in CairoMakie line drawing when multiple successive points had the same color [#3712](https://github.com/MakieOrg/Makie.jl/pull/3712).
+- Remove StableHashTraits in favor of calculating hashes directly with CRC32c [#3667](https://github.com/MakieOrg/Makie.jl/pull/3667).
+- Fixed `contourf` bug where n levels would sometimes miss the uppermost value, causing gaps [#3713](https://github.com/MakieOrg/Makie.jl/pull/3713).
+- Added `scale` attribute to `violin` [#3352](https://github.com/MakieOrg/Makie.jl/pull/3352).
+- Use label formatter in barplot [#3718](https://github.com/MakieOrg/Makie.jl/pull/3718).
+- Fix the incorrect shading with non uniform markerscale in meshscatter [#3722](https://github.com/MakieOrg/Makie.jl/pull/3722)
+- Add `scale_to=:flip` option to `hist`, which flips the direction of the bars [#3732](https://github.com/MakieOrg/Makie.jl/pull/3732)
+- Fixed an issue with the texture atlas not updating in WGLMakie after display, causing new symbols to not show up [#3737](https://github.com/MakieOrg/Makie.jl/pull/3737)
+
 ## [0.20.8] - 2024-02-22
 
 - Fixed excessive use of space with HTML image outputs [#3642](https://github.com/MakieOrg/Makie.jl/pull/3642).

GLMakie/src/drawing_primitives.jl

@@ -226,7 +226,7 @@ const EXCLUDE_KEYS = Set([:transformation, :tickranges, :ticklabels, :raw, :SSAO
                         :lightposition, :material, :axis_cycler,
                         :inspector_label, :inspector_hover, :inspector_clear, :inspectable,
                         :colorrange, :colormap, :colorscale, :highclip, :lowclip, :nan_color,
-                        :calculated_colors, :space, :markerspace, :model])
+                        :calculated_colors, :space, :markerspace, :model, :dim_conversions])
 
 
 function cached_robj!(robj_func, screen, scene, plot::AbstractPlot)

GLMakie/src/precompiles.jl

@@ -68,3 +68,7 @@ precompile(glTexImage, (GLenum, Int, GLenum, Int, Int, Int, GLenum, GLenum, Ptr{
 precompile(glTexImage, (GLenum, Int, GLenum, Int, Int, Int, GLenum, GLenum, Ptr{RGBA{Float16}}))
 precompile(glTexImage, (GLenum, Int, GLenum, Int, Int, Int, GLenum, GLenum, Ptr{N0f8}))
 precompile(setindex!, (GLMakie.GLAbstraction.Texture{Float16,2}, Matrix{Float32}, Rect2{Int32}))
+precompile(getindex, (Makie.Text{Tuple{Vector{Point{2,Float32}}}}, Symbol))
+precompile(getproperty, (Makie.Text{Tuple{Vector{Point{2,Float32}}}}, Symbol))
+precompile(plot!, (Makie.Text{Tuple{Vector{Point{2,Float32}}}},))
+precompile(Base.getindex, (Attributes, Symbol))

GLMakie/src/screen.jl

@@ -497,7 +497,7 @@ function Base.delete!(screen::Screen, scene::Scene)
 
         # Remap scene IDs to a continuous range by replacing the largest ID
         # with the one that got removed
-        if deleted_id-1 != length(screen.screens)
+        if deleted_id - 1 != length(screen.screens)
             key, max_id = first(screen.screen2scene)
             for p in screen.screen2scene
                 if p[2] > max_id

MakieCore/Project.toml

@@ -4,6 +4,9 @@ authors = ["Simon Danisch"]
 version = "0.8.0"
 
 [deps]
+ColorTypes = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
+GeometryBasics = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
+IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953"
 Observables = "510215fc-4207-5dde-b226-833fc4488ee2"
 REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
 

MakieCore/src/MakieCore.jl

@@ -8,10 +8,9 @@ end
 using Observables
 using Observables: to_value
 using Base: RefValue
-# Needing REPL for Base.Docs.doc on julia
-# https://github.com/MakieOrg/Makie.jl/issues/3276
-using REPL
-
+using GeometryBasics
+using ColorTypes
+using IntervalSets: ClosedInterval, Interval
 
 include("types.jl")
 include("attributes.jl")

MakieCore/src/attributes.jl

@@ -118,17 +118,7 @@ function Base.setindex!(x::Attributes, value, key::Symbol)
 end
 
 function Base.setindex!(x::Attributes, value::Observable, key::Symbol)
-    if haskey(x, key)
-        # error("You're trying to update an attribute Observable with a new Observable. This is not supported right now.
-        # You can do this manually like this:
-        # lift(val-> attributes[$key] = val, Observable::$(typeof(value)))
-        # ")
-        return x.attributes[key] = node_any(value)
-    else
-        #TODO make this error. Attributes should be sort of immutable
-        return x.attributes[key] = node_any(value)
-    end
-    return x
+    return x.attributes[key] = node_any(value)
 end
 
 _indent_attrs(s, n) = join(split(s, '\n'), "\n" * " "^n)
@@ -190,16 +180,16 @@ Base.get(x::AttributeOrPlot, key::Symbol, default) = get(()-> default, x, key)
 # Plot plots break this assumption in some way, but the way to look at it is,
 # that the plots contained in a Plot plot are not subplots, but _are_ actually
 # the plot itself.
-Base.getindex(plot::AbstractPlot, idx::Integer) = plot.converted[idx]
-Base.getindex(plot::AbstractPlot, idx::UnitRange{<:Integer}) = plot.converted[idx]
-Base.setindex!(plot::AbstractPlot, value, idx::Integer) = (plot.args[idx][] = value)
-Base.length(plot::AbstractPlot) = length(plot.converted)
+Base.getindex(plot::Plot, idx::Integer) = plot.converted[idx]
+Base.getindex(plot::Plot, idx::UnitRange{<:Integer}) = plot.converted[idx]
+Base.setindex!(plot::Plot, value, idx::Integer) = (plot.args[idx][] = value)
+Base.length(plot::Plot) = length(plot.converted)
 
-function Base.getindex(x::AbstractPlot, key::Symbol)
-    argnames = argument_names(typeof(x), length(x.converted))
+function Base.getindex(x::T, key::Symbol) where {T <: Plot}
+    argnames = argument_names(T, length(x.converted))
     idx = findfirst(isequal(key), argnames)
     if idx === nothing
-        return x.attributes[key]
+        return attributes(x)[key]
     else
         return x.converted[idx]
     end
@@ -233,15 +223,7 @@ function Base.setindex!(x::AbstractPlot, value::Observable, key::Symbol)
     argnames = argument_names(typeof(x), length(x.converted))
     idx = findfirst(isequal(key), argnames)
     if idx === nothing
-        if haskey(x, key)
-            # error("You're trying to update an attribute Observable with a new Observable. This is not supported right now.
-            # You can do this manually like this:
-            # lift(val-> attributes[$key] = val, Observable::$(typeof(value)))
-            # ")
-            return x.attributes[key] = value
-        else
-            return x.attributes[key] = value
-        end
+        return attributes(x)[key] = value
     else
         return setindex!(x.converted[idx], value)
     end

MakieCore/src/basic_plots.jl

@@ -200,7 +200,7 @@ calculated_attributes!(plot::T) where T = calculated_attributes!(T, plot)
 
 Plots an image on a rectangle bounded by `x` and `y` (defaults to size of image).
 """
-@recipe Image x y image begin
+@recipe Image (x::ClosedInterval{<:FloatType}, y::ClosedInterval{<:FloatType}, image::AbstractMatrix{<:Union{FloatType,Colorant}}) begin
     "Sets whether colors should be interpolated between pixels."
     interpolate = true
     mixin_generic_plot_attributes()...
@@ -238,7 +238,7 @@ If `x` and `y` are omitted with a matrix argument, they default to `x, y = axes(
 
 Note that `heatmap` is slower to render than `image` so `image` should be preferred for large, regularly spaced grids.
 """
-@recipe Heatmap x y values begin
+@recipe Heatmap (x::RealVector, y::RealVector, values::AbstractMatrix{<:Union{FloatType,Colorant}}) begin
     "Sets whether colors should be interpolated"
     interpolate = false
     mixin_generic_plot_attributes()...
@@ -258,7 +258,12 @@ Available algorithms are:
 * `:additive` => AdditiveRGBA
 * `:indexedabsorption` => IndexedAbsorptionRGBA
 """
-@recipe Volume x y z volume begin
+@recipe Volume (
+        x::ClosedInterval,
+        y::ClosedInterval,
+        z::ClosedInterval,
+        volume::AbstractArray{Float32,3}
+    ) begin
     "Sets the volume algorithm that is used."
     algorithm = :mip
     "Sets the range of values picked up by the IsoValue algorithm."
@@ -276,14 +281,16 @@ Available algorithms are:
     mixin_colormap_attributes()...
 end
 
+const VecOrMat{T} = Union{AbstractVector{T}, AbstractMatrix{T}}
+
 """
     surface(x, y, z)
     surface(z)
 
 Plots a surface, where `(x, y)` define a grid whose heights are the entries in `z`.
 `x` and `y` may be `Vectors` which define a regular grid, **or** `Matrices` which define an irregular grid.
 """
-@recipe Surface x y z begin
+@recipe Surface (x::VecOrMat{<:FloatType}, y::VecOrMat{<:FloatType}, z::VecOrMat{<:FloatType}) begin
     "Can be set to an `Matrix{<: Union{Number, Colorant}}` to color surface independent of the `z` component. If `color=nothing`, it defaults to `color=z`."
     color = nothing
     "Inverts the normals generated for the surface. This can be useful to illuminate the other side of the surface."
@@ -302,7 +309,7 @@ Creates a connected line plot for each element in `(x, y, z)`, `(x, y)` or `posi
 
 `NaN` values are displayed as gaps in the line.
 """
-@recipe Lines positions begin
+@recipe Lines (positions,) begin
     "The color of the line."
     color = @inherit linecolor
     "Sets the width of the line in screen units"
@@ -329,22 +336,8 @@ end
     linesegments(x, y, z)
 
 Plots a line for each pair of points in `(x, y, z)`, `(x, y)`, or `positions`.
-
-## Attributes
-
-### Specific to `LineSegments`
-
-- `color=theme(scene, :linecolor)` sets the color of the linesegments. If no color is set, multiple calls to `linesegments!` will cycle through the axis color palette.
-  Otherwise, one can set one color per line point or one color per linesegment by passing a `Vector{<:Colorant}`, or one colorant for the whole line. If color is a vector of numbers, the colormap args are used to map the numbers to colors.
-- `cycle::Vector{Symbol} = [:color]` sets which attributes to cycle when creating multiple plots.
-- `linestyle::Union{Nothing, Symbol, Vector} = nothing` sets the pattern of the line (e.g. `:solid`, `:dot`, `:dashdot`)
-- `linewidth::Union{Real, Vector} = 1.5` sets the width of the line in pixel units.
-
-$(Base.Docs.doc(colormap_attributes!))
-
-$(Base.Docs.doc(MakieCore.generic_plot_attributes!))
 """
-@recipe LineSegments positions begin
+@recipe LineSegments (positions,) begin
     "The color of the line."
     color = @inherit linecolor
     "Sets the width of the line in pixel units"
@@ -369,7 +362,7 @@ end
 
 Plots a 3D or 2D mesh. Supported `mesh_object`s include `Mesh` types from [GeometryBasics.jl](https://github.com/JuliaGeometry/GeometryBasics.jl).
 """
-@recipe Mesh mesh begin
+@recipe Mesh (mesh::Union{AbstractVector{<:GeometryBasics.Mesh},GeometryBasics.Mesh},) begin
     "Sets the color of the mesh. Can be a `Vector{<:Colorant}` for per vertex colors or a single `Colorant`. A `Matrix{<:Colorant}` can be used to color the mesh with a texture, which requires the mesh to contain texture coordinates."
     color = @inherit patchcolor
     "sets whether colors should be interpolated"
@@ -388,7 +381,7 @@ end
 
 Plots a marker for each element in `(x, y, z)`, `(x, y)`, or `positions`.
 """
-@recipe Scatter positions begin
+@recipe Scatter (positions,) begin
     "Sets the color of the marker. If no color is set, multiple calls to `scatter!` will cycle through the axis color palette."
     color = @inherit markercolor
     "Sets the scatter marker."
@@ -438,7 +431,7 @@ end
 Plots a mesh for each element in `(x, y, z)`, `(x, y)`, or `positions` (similar to `scatter`).
 `markersize` is a scaling applied to the primitive passed as `marker`.
 """
-@recipe MeshScatter positions begin
+@recipe MeshScatter (positions,) begin
     "Sets the color of the marker."
     color = @inherit markercolor
     "Sets the scattered mesh."
@@ -467,7 +460,7 @@ end
 Plots one or multiple texts passed via the `text` keyword.
 `Text` uses the `PointBased` conversion trait.
 """
-@recipe Text positions begin
+@recipe Text (positions,) begin
     "Specifies one piece of text or a vector of texts to show, where the number has to match the number of positions given. Makie supports `String` which is used for all normal text and `LaTeXString` which layouts mathematical expressions using `MathTeXEngine.jl`."
     text = ""
     "Sets the color of the text. One can set one color per glyph by passing a `Vector{<:Colorant}`, or one colorant for the whole text. If color is a vector of numbers, the colormap args are used to map the numbers to colors."
@@ -613,7 +606,7 @@ Draws a wireframe, either interpreted as a surface or as a mesh.
     depth_shift = -1f-5
 end
 
-@recipe Arrows points directions begin
+@recipe Arrows (points, directions) begin
     "Sets the color of arrowheads and lines. Can be overridden separately using `linecolor` and `arrowcolor`."
     color = :black
     """Scales the size of the arrow head. This defaults to

MakieCore/src/conversion.jl

@@ -1,5 +1,4 @@
 
-function convert_arguments end
 
 """
     convert_attribute(value, attribute::Key[, plottype::Key])
@@ -31,8 +30,6 @@ struct NoConversion <: ConversionTrait end
 conversion_trait(::Type) = NoConversion()
 conversion_trait(T::Type, args...) = conversion_trait(T)
 
-convert_arguments(::NoConversion, args...) = args
-
 """
     PointBased() <: ConversionTrait
 
@@ -41,7 +38,6 @@ Plots with the `PointBased` trait convert their input data to a
 """
 struct PointBased <: ConversionTrait end
 conversion_trait(::Type{<: XYBased}) = PointBased()
-conversion_trait(::Type{<: Text}) = PointBased()
 
 """
     GridBased <: ConversionTrait
@@ -100,3 +96,57 @@ conversion_trait(::Type{<: Image}) = ImageLike()
 
 struct VolumeLike <: ConversionTrait end
 conversion_trait(::Type{<: Volume}) = VolumeLike()
+
+function convert_arguments end
+
+convert_arguments(::NoConversion, args...; kw...) = args
+
+get_element_type(::T) where {T} = T
+function get_element_type(arr::AbstractArray{T}) where {T}
+    if T == Any
+        return mapreduce(typeof, promote_type, arr)
+    else
+        return T
+    end
+end
+
+types_for_plot_arguments(trait) = nothing
+function types_for_plot_arguments(P::Type{<:Plot}, Trait::ConversionTrait)
+    p = types_for_plot_arguments(P)
+    isnothing(p) || return p
+    return types_for_plot_arguments(Trait)
+end
+
+function types_for_plot_arguments(::PointBased)
+    return Tuple{AbstractVector{<:Union{Point2, Point3}}}
+end
+
+should_dim_convert(::Type) = false
+
+"""
+    MakieCore.should_dim_convert(::Type{<: Plot}, args)::Bool
+    MakieCore.should_dim_convert(eltype::DataType)::Bool
+
+Returns `true` if the plot type should convert its arguments via DimConversions.
+Needs to be overloaded for recipes that want to use DimConversions. Also needs
+to be overloaded for DimConversions, e.g. for CategoricalConversion:
+
+```julia
+    MakieCore.should_dim_convert(::Type{Categorical}) = true
+```
+
+`should_dim_convert(::Type{<: Plot}, args)` falls back on checking if
+`has_typed_convert(plot_or_trait)` and `should_dim_convert(get_element_type(args))`
+ are true. The former is defined as true by `@convert_target`, i.e. when
+`convert_arguments_typed` is defined for the given plot type or conversion trait.
+The latter marks specific types as convertable.
+
+If a recipe wants to use dim conversions, it should overload this function:
+```julia
+    MakieCore.should_dim_convert(::Type{<:MyPlotType}, args) = should_dim_convert(get_element_type(args))
+``
+"""
+function should_dim_convert(P, arg)
+    isnothing(types_for_plot_arguments(P)) && return false
+    return should_dim_convert(get_element_type(arg))
+end