Plots 2.0 #4904

BeastyBlacksmith · 2024-03-12T14:57:47Z

This is the development branch for Plots 2.0.

What changed

No default backend in PlotsBase, Plots becomes a wrapper for PlotsBase + GR (per [RFC] Plots 2.0 #4565 (comment)), making it non-breaking;
Backends are purely extensions with no @require;
The internals are more organized into modules and moved around;
Backend init and backend code is only found in its extensions, this should help with backend interference and annoyances about initialization and precompilation of plots. This should make backends modular and implemented much easier and without hacks into Plots.jl codebase;
Restored Preferences chosen backend;
...

Example:

import UnicodePlots
using Plots 
unicodeplots()
plot(1:10)

What works

PlotsBase and Plots tests;
Preferences based backend selection.

What does not work

Precompilation statements for Plots when using an alternate backend: cannot @eval Main import UnicodePlots => most of the precompile statements are cached in PlotsBase now.

Help needed:

write backend agnostic precompile statements for PlotsBase
backport changes from v1 since last rebase
report breakages
move documentation back into this repo
start updating the docs
update the news.md
register GraphRecipes and StatsPlots from this monorepo to the General registry.

Decisions to be made

deprecate inspectDR ?
move to column major order ?
change the defaults

Issues needing attention please add more or checkoff resolved:
#4565 (comment)
Copy of that:
Labeled issues:

Labeled PRs:

Closes #4326

* remove deprecated backends * move GR to extension * make plot3d work * make heatmap work * factor components into modules * finish Surface module * export is_horizontal from Fonts * start refactoring Plot, Subplot, etc. components * change version * continue module shuffling * continued refactoring * remove orientation * minimal usable state * format * remove makekw * anything up to Segments * move series stuff from utils * use Arrows * create Colorbars module * move axes stuff (precompiling state) * let's relax deps and add them back later if needed (macOS issue) * only 2 tests fail now * restructure extension logic * remove unicode for now * GR extension * update deps * init step of gr * proper init of GR * for now manually activate the backend * road to fixing gr * revert some changes * fix import exports * fix measures * clarify name space * might need to be reverted, shape->Shape (too confusing_ * return lost wrap * types and modules * fix more things * add gr function * remove stale and fix naming * added unicode backend * unicodeplots works * update runtests * only use explicitly imported names in backends * rename wrap to protect * remove pre post imports, move to alignment jl * alignment jl * remove comments * fix typo * finally fix InputWrapper, misc changes * import overloaded, namespace fix * fixing per backend exceptions * alost all tests are passing * fix exports * trying to fix project toml * try to resolve deps * move other backends to the extensions * start pgfplotsx * add backends to weakdeps * recover * already defined * fix GR plot area method overload * plotarea methods are overloaded in layouts.jl * more fixes * fix test * namespace fixes * fix namespaces * revert later * rm pyplot * remove requires * add PGFPlotsX imports * bump julia compat * create runtime functions for all backends * improve error handling * pgfplotsx loading * get tests running * fix names, skip StatsPlots examples * test on 1.10 * remove pyplot remainings * don't test julia < 1.9 * run julia formatter * add pythonplotbackend * fix init python plot * gaston working * reorganize plotly backends * update kaleido compat * format * remove PlotlyBase compat * make default backend functional * make plotly functional * naming and PlotlyJS * get gaston running * run more tests * move dev in ci * dev both at the same time * fix syntax * don't dev twice * only load Gtk if not on CI * add missing using statement for PlotlyJS * more imports * move merge_with_base_supported to Commons * make Plotly module * skip non-working tests * fix filtering * format * invert filter * fix pgfplotsx warning * don't test Gaston * fix cgrad missing * format --------- Co-authored-by: Simon Christ <simonchrist@gmx.de> Co-authored-by: Simon Christ <christ@cell.uni-hannover.de> Co-authored-by: = <=>

* rework extensions * need to fix `test_backends` * test `Plots` pass * fix test * restore tests * format * update

BeastyBlacksmith · 2024-04-02T07:53:12Z

@t-bltg do you intend to release this as a 1.x release?

t-bltg · 2024-04-02T07:55:26Z

If we can make it non breaking, it would make things easier and end endless discussions.

BeastyBlacksmith · 2024-04-02T08:04:21Z

Well, there would be a few changes that we'd need to roll back for this, like removal of pgfplots and I removed some functions also.

But I also don't quite see how to make switching of backends non-beraking since, what was

using Plots
pythonplot()

is now

using Plots
import PythonPlot
pythonplot()

isentropic · 2024-04-02T17:19:58Z

I still don't get why can't 2.0 be breaking and why having a helpful error message isn't enough as a solution as per my comment in the issue #4565. Having GR as the default backend feels bad to me. I understand that nobody changes the backend but this way backend malfunctioning could be diagnosed easier.

Then if the backend doesn't work people can learn about other backends. I really don't buy the keep gr argument by searching GitHub code. Plots 2.0 isn't just for the users but also for debugging and long term stability.

Keeping gr as the default is not good imo. Flux got rid of it's CUDA for this reason and it works fine, new users can adapt we just need to leave good info/warn messages

isentropic · 2024-04-02T17:27:30Z

I'm sorry that I'm late into the discussion and if some undoing needs to be done but we should prioritise long term good with 2.0. GR being the default is also what defers people from using plots, if users can learn about the flexibility that's great too. Pyplot imo works more stably with various fonts and scalings.

The purpose of package extensions is exactly this. Splitting into plotsbase and just plots to keep the standard interface means that we never need to go to 2.0 then.

2.0 by definition is breaking and that's ok. Plots is not a low level package and hardly any packages take dep on Plots. Interactive usage is easier to adapt to 2.0 anyways in case the help messages are well stated.

We also need to think about column major order to make Plots play nicely with others and streamline the column major way. These two changes are by far the most important, both being breaking and that's ok cause it's 2.0.

isentropic · 2024-04-02T17:36:57Z

I get why people might object with needing to import GR explicitly, but you know adding such import help long term in that people would realize that there are more backends than just GR, and that the errors are coming because of 'import GR' not 'using Plots'. Im not kidding half of our issues are because GR didn't build or something like that. In the current scenario users try to import Plots it errors and then there's frustration. With gr free plots GR will throw it's warnings right awa and we can throw warnings and stuff like your GR loading failed, try these other backends.

t-bltg · 2024-04-06T10:35:43Z

But I also don't quite see how to make switching of backends non-beraking since, what was

I've omitted this, so I've restored the 2.0 label.

We also need to think about column major order to make Plots play nicely with others and streamline the column major way.

I get this, but if no one implements it, this shouldn't block a release.

Having GR as the default backend feels bad to me.

I would be in favor of dropping GR as the default backend. But on the other end, adding PlotsBase without any backend dependency seems a good compromise.

In #4914, I've finalized the way modules and extensions should be written with a lot of cleanups, and restoration of the precompile statements, but unfortunately, it broke the reference tests in subtle ways. What a mess !

I like the new splitting of PlotsBase into smaller modules initiated in this branch, but it needs more work and cleanup, without breaking the tests.

t-bltg · 2024-04-07T07:33:52Z

I've made progress and fixed the tests in #4914, everything is passing locally, ~~and partially in CI~~.

Currently, there is no solution to generate precompile statements for a different backend than GR in Plots since we cannot @eval Main import UnicodePlots for example.

~~So we need at least to generate backend independent precompile statement for PlotsBase.~~
==> EDIT: done in #4914.

* rework extension `__init__` mechanism * checkpoint plotly * fix `pythonplot` * format * more `pythonplot` fixes * checkpoint replace * syntax * fix `plotlyjs` * restructure modules * stable `GR` * checkpoint broken `references` gr * broken viewport * fix `dev` * update * no project * remove `--project` * update * fix `plotarea` * fixes * stable `PlotsBase` * cleanup * format * restrcit gaston to `linux` * restrcit `PlolyJS` test * update format * update action * update `Aqua` * cleanup - fix `Plots` tests * move preferences back to `PlotsBase` * simplifications * format * change versions * rework ci * format * test downstream * update ci * move ci scripts * fix * fix * update * debug ci * fix * update versions * update * update * update * update * update * update * update * update * restore CI * update workflow * add `PlotsBase` precompile statements * update tests * update preference - simplifications * update test * update

t-bltg · 2024-04-07T13:58:27Z

@BeastyBlacksmith, how can we move forward now ?

Embedded modules need more cleanup, but at least the port to extensions is now complete, with 100% previous tests passing.

t-bltg · 2024-10-21T18:50:43Z

@BeastyBlacksmith, can you fix the DOCUMENTER_KEY issue so that we can have dev docs for v2 deployed ?

t-bltg · 2024-10-24T13:48:06Z

I've changed the deploydocs url to https://github.com/JuliaPlots/Plots.jl.git in a6562c1 specifically because I was hitting a deployment failure from Documenter.

So I think restoring it in 912824f will hit this error again.

I thought we were dropping the PlotDocs repo ?

BeastyBlacksmith · 2024-10-24T13:52:59Z

I thought we were dropping the PlotDocs repo ?

Not entirely, it should still host the artifacts to not blow the repo size of Plots, but the generating code should be here.

t-bltg · 2024-10-24T14:02:56Z

I see.

But still, we will hit this non-deployment criterion because of the url: https://github.com/JuliaPlots/Plots.jl/actions/runs/11317817828/job/31471690944#step:5:7953
│ - ✘ ENV["GITHUB_REPOSITORY"]="JuliaPlots/Plots.jl" occurs in repo="github.com/JuliaPlots/PlotDocs.jl.git"

After changing it in a6562c1 , we hit another error, related to DOCUMENTER_KEY: https://github.com/JuliaPlots/Plots.jl/actions/runs/11318156624/job/31484813837#step:5:7945
│ - ✔ ENV["GITHUB_REPOSITORY"]="JuliaPlots/Plots.jl" occurs in repo="github.com/JuliaPlots/Plots.jl.git"

, which was my original request in #4904 (comment).

t-bltg · 2024-10-24T14:07:19Z

I think we need to follow https://documenter.juliadocs.org/stable/man/hosting/#Out-of-repo-deployment.

Can you do "3. Add the DOCUMENTER_KEY secret to the "source" repository (that runs the documentation workflow)", since I do not have enough rights ?

BeastyBlacksmith · 2024-10-24T14:08:57Z

That key is already present. We used to trigger the build from here in v1 also. You just can't deploy to this repository with it.

t-bltg · 2024-10-24T14:11:36Z

Item 4. done in bb14fd5, let's see how it goes ...

t-bltg · 2024-10-24T15:08:13Z

As I suspected, there is still an issue with permissions, hence I think DOCUMENTER_KEY is the culprit here, but I have no further means to investigate:
https://github.com/JuliaPlots/Plots.jl/actions/runs/11501098562/job/32012907078?pr=4904#step:5:8035.

BeastyBlacksmith · 2024-10-26T19:14:28Z

Works now

t-bltg · 2024-10-26T19:48:44Z

Awesome thanks, https://docs.juliaplots.org/dev looks functional !

EDIT: except for the backend examples e.g. https://docs.juliaplots.org/dev/gallery/gr/@ref%20gr_ref001 is 404, when clicking on the thumbnails.

t-bltg · 2024-11-02T13:37:15Z

The docs are now fully functional on v2, e.g. https://docs.juliaplots.org/dev/gallery/gr/generated/gr-ref002/#gr_ref002.

t-bltg · 2024-11-02T16:09:45Z

@BeastyBlacksmith, would you be in favor of integration PlotThemes into this monorepo ?

BeastyBlacksmith · 2024-11-04T09:00:58Z

The docs are now fully functional on v2,

Thanks a lot !

would you be in favor of integration PlotThemes into this monorepo ?

Yeah, thats probably a good idea

BeastyBlacksmith added help wanted documentation - API WIP Work in progress, do not merge! 2.0 labels Mar 12, 2024

t-bltg mentioned this pull request Mar 31, 2024

do not run i/o when precompiling mbaz/Gaston.jl#185

Closed

t-bltg changed the title ~~Plots 2.0~~ Redesign backend loading Mar 31, 2024

t-bltg added DO NOT MERGE and removed 2.0 labels Mar 31, 2024

create PlotsBase (#4913)

b1811ba

* rework extensions * need to fix `test_backends` * test `Plots` pass * fix test * restore tests * format * update

t-bltg added the 2.0 label Apr 6, 2024

t-bltg changed the title ~~Redesign backend loading~~ Plots 2.0 Apr 6, 2024

t-bltg added 6 commits April 7, 2024 14:42

restore ci on v2

a5a028b

restore format check on v2

2979ab5

empty commit

5014e6c

Merge branch 'master' into v2

4ef0b18

make downstream test conditional

82f57ba

t-bltg assigned BeastyBlacksmith and unassigned BeastyBlacksmith Apr 7, 2024

cleanup

023b643

t-bltg added 3 commits October 14, 2024 00:20

fix url in docs build script

a6562c1

use devbranch in docs/make.jl

52c6340

revert devbranch changes

c4ca40d

BeastyBlacksmith and others added 2 commits October 24, 2024 15:39

correct deploy repo

912824f

animation: autoplay <video> in HTML (#5006)

b51c665

fix GITHUB_REPOSITORY

bb14fd5

add codespell action and correct spelling errors (#5011)

8119dac

t-bltg mentioned this pull request Nov 2, 2024

pagesonly documenter keyword JuliaDocs/DemoCards.jl#162

Open

t-bltg added 4 commits November 2, 2024 12:46

update docs for v2 (#5012)

43e7f65

temporary docs v2 fix (#5013)

4cb0a72

update build-docs.sh

675d4dc

update make.jl

5c95a62

update urls for v2 (#5014)

01701b6

t-bltg added 4 commits November 4, 2024 11:18

add PlotThemes into monorepo (#5015)

c7902ef

update PlotThemes README.md

f272860

CI: extend coverage to all packages

76863f3

fix typo

c664bfe

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plots 2.0 #4904

Plots 2.0 #4904

BeastyBlacksmith commented Mar 12, 2024 •

edited by t-bltg

Loading

BeastyBlacksmith commented Apr 2, 2024

t-bltg commented Apr 2, 2024

BeastyBlacksmith commented Apr 2, 2024

isentropic commented Apr 2, 2024

isentropic commented Apr 2, 2024

isentropic commented Apr 2, 2024

t-bltg commented Apr 6, 2024 •

edited

Loading

t-bltg commented Apr 7, 2024 •

edited

Loading

t-bltg commented Apr 7, 2024

t-bltg commented Oct 21, 2024

t-bltg commented Oct 24, 2024 •

edited

Loading

BeastyBlacksmith commented Oct 24, 2024

t-bltg commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024 •

edited

Loading

BeastyBlacksmith commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024

BeastyBlacksmith commented Oct 26, 2024

t-bltg commented Oct 26, 2024 •

edited

Loading

t-bltg commented Nov 2, 2024 •

edited

Loading

t-bltg commented Nov 2, 2024

BeastyBlacksmith commented Nov 4, 2024

Plots 2.0 #4904

Are you sure you want to change the base?

Plots 2.0 #4904

Conversation

BeastyBlacksmith commented Mar 12, 2024 • edited by t-bltg Loading

What changed

What works

What does not work

Help needed:

Decisions to be made

BeastyBlacksmith commented Apr 2, 2024

t-bltg commented Apr 2, 2024

BeastyBlacksmith commented Apr 2, 2024

isentropic commented Apr 2, 2024

isentropic commented Apr 2, 2024

isentropic commented Apr 2, 2024

t-bltg commented Apr 6, 2024 • edited Loading

t-bltg commented Apr 7, 2024 • edited Loading

t-bltg commented Apr 7, 2024

t-bltg commented Oct 21, 2024

t-bltg commented Oct 24, 2024 • edited Loading

BeastyBlacksmith commented Oct 24, 2024

t-bltg commented Oct 24, 2024 • edited Loading

t-bltg commented Oct 24, 2024 • edited Loading

BeastyBlacksmith commented Oct 24, 2024 • edited Loading

t-bltg commented Oct 24, 2024 • edited Loading

t-bltg commented Oct 24, 2024

BeastyBlacksmith commented Oct 26, 2024

t-bltg commented Oct 26, 2024 • edited Loading

t-bltg commented Nov 2, 2024 • edited Loading

t-bltg commented Nov 2, 2024

BeastyBlacksmith commented Nov 4, 2024

BeastyBlacksmith commented Mar 12, 2024 •

edited by t-bltg

Loading

t-bltg commented Apr 6, 2024 •

edited

Loading

t-bltg commented Apr 7, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024 •

edited

Loading

BeastyBlacksmith commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 24, 2024 •

edited

Loading

t-bltg commented Oct 26, 2024 •

edited

Loading

t-bltg commented Nov 2, 2024 •

edited

Loading