finding python - findpython and extensibility and uniformity

[bluss]

Pdm finds python through findpython. This mechanism is extensible in the sense that findpython can learn to find python from new sources.

I would like to have a framework for finding python that is more configurable, flexible and uniform. The current downside with findpython is that it can be a bit haphazard - you might find 3.10 from system python but 3.9 from a pyenv install. You'd want to ensure uniformity to make it easier for you - for example to make sure all your python versions in use are installed the same way.

Let's say you have different strategies:

• System python
• Pyenv
• Other python installers: py, asdf, rye

You would like to be able to configure: all your projects get their python from pyenv (asdf, rye, etc..) by default. In this configuration, pdm or findpython's mechanism only considers python versions from the configured source.

Some of the sources have the ability to install python on demand (pyenv, asdf, rye), even better then if we had a hook so that it would just fetch python using the configured source, if it was missing.

[frostming]

The library is easy to extend, just create a subclass of Provider and add it to the Finder object.

Here is an example in PDM:

pdm/src/pdm/project/core.py

Lines 660 to 668 in 4618fc8

	def _get_python_finder(self) -> Finder:
	from findpython import Finder
	from pdm.cli.commands.venv.utils import VenvProvider
	finder = Finder(resolve_symlinks=True)
	if self.config["python.use_venv"]:
	finder.add_provider(VenvProvider(self), 0)
	return finder

You can also modify the list of default providers by changing findpython.finder.ALL_PROVIDERS

[frostming]

The name "RyeProvider" would not be accurate, since the python provider has nothing to do with the package manager. It is powered by python-build-standalone. I would suggest something like build standalone provider though it is a bit long-winded.

[bluss]

I was thinking it will find python installed by rye. How they were built doesn't matter so much. They'll be installed into for example ~/.rye/py/cpython@3.10.11/ and that's how we find them. python-build-standalone builds could of course be installed by some other tool to some other place (let's imagine if asdf used the same builds).

[frostming]

Oh, I thought it not only finds installed Pythons, but also installs from the build-standalone if not available. In that way we don't depend on rye at all.

[bluss]

Aha that explains the difference. That's a different project than what I had in mind, I'll admit.

I'm thinking, pyenv, asdf (and rye) can all install python if you want to. To reimplement or implement an alternative to them would be a different project in itself.

When I said RyeProvider, to start with it's just something that's extending very straightforwardly, what AsdfProvider for example is doing, onto Rye.

[bluss]

The suggestion for the previously mentioned (1) part, selecting between available providers, was posted here: frostming/findpython#17