Why doesn't Grayskull use grayskull_pypi_mapping?

[maresb]

I noticed that there is a grayskull_pypi_mapping which, as far as I can tell, is not used by Grayskull. I was wondering why not? (Or did I miss something?)

I've been working a bit on conda-lock lately, and we now have some support for parsing pyproject.toml files into Conda dependencies. There is a lot of overlap with Grayskull, and it feels like we're developing very similar functionality. In general, it seems to me like it may be beneficial to the Conda ecosystem in general if various functionalities were split into separate Python packages for reusability. I wonder how much support/motivaion is there along these lines?

[dhirschfeld]

we now have some support for parsing pyproject.toml files into Conda dependencies

I was browsing around here exactly to see if grayskull could do this for me! I'll go and check out conda-lock...

[dhirschfeld]

AFAICT grayskull can't create a recipe from a local pyproject.toml - is that the case?

[dhirschfeld]

...and it looks like conda-lock can create a lock file from a pyproject.toml but not a conda/boa recipe?

[dhirschfeld]

I'd like to dynamically create conda/boa recipes from pyproject.toml files. It looks like conda-lock specifies the extra required metadata in the pyproject.toml but only creates lock files, not recipes... AFAICS?

[maresb]

Are you looking for code or ready-made tools?

Definitely take a look at the PRs of @srilman, he's working on abstracting the dependency-reading process in a sane manner. The main task is parsing the dependencies, right? (Not writing them in a particular output.)

I don't quite understand what you mean by "specifies the extra required metadata".

[dhirschfeld]

A ready-made tool would be less work than rolling something myself but if there is existing code already that's better than starting from scratch.

I don't quite understand what you mean by "specifies the extra required metadata".

...though I guess for an actual recipe you would need to be able to specify build/host/run deps for both Python and non-python packages.

[maresb]

If there's any hope of a public release of what you're working on, please keep me in the loop. I'd be very interested to see a unified collection of libraries that modularizes what conda-lock and grayskull currently do.

[dhirschfeld]

Unfortunately it's more something I wished existed already than something I'm working on. If I do end up hacking something together I'll make sure to share.

[marcelotrevisani]

Pyproject.toml from local files should work. If it is not working it might be a bug :/
Grayskull is not using that because we create that mapping first, for the logic of the events they should be using our mapping 😅

[maresb]

I'm confused what you mean by "we create that mapping first". The grayskull_pypi_mapping is created in CI by running this script which does some somewhat intensive graph theoretic computations in an attempt to guess the "best" (most authoritative) package names, and it also makes use of a static mapping so that wrong guesses can be corrected. Last time I checked it seemed like Grayskull was using a much simpler algorithm based purely on the package name.

If there is a better mapping somewhere, then I'd definitely like to use it!

[marcelotrevisani]

I am open to PR if you wish to improve it, it is just not on my radar right now.
What I meant is, we created the static mapping before that script was created, but if you wish to improve it and add support for it, I am pretty much open to it :)

[maresb]

Ok, great! I will see what I can do.

A big part of my confusion is that the mapping is called the "grayskull pypi mapping" but never actually seemed to have anything to do with Grayskull. Was it Grayskull-inspired, but was never integrated?