Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Additional python packaging documentation, version correction #9405

Merged
merged 3 commits into from
Mar 16, 2024
Merged

Additional python packaging documentation, version correction #9405

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7424f3f
setuptools version bump, numpy install details
vilhelmen Mar 7, 2024
e30b636
Better dependency adn installation info, outline pip issues
vilhelmen Mar 9, 2024
eff0fef
Typo fix, rework for clarity
vilhelmen Mar 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion swig/python/README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,11 @@ It will be necessary to have libgdal and its development headers installed
if pip is expected to do a source build because no wheel is available
for your specified platform and Python version.

To install with numpy support, you need to require the optional numpy component:
To install with numpy support, you need to require the optional numpy component as well as the Python packaging tools wheel and setuptools:

::

pip install wheel setuptools>=67
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For at least newer (very new?) versions of $ pip, I don't think wheel has to be explicitly installed. I think it is automatically installed as part of the build process?

Looking at the setuptools changelog, it isn't really clear to me what feature in v67 is required for installing GDAL. Maybe this?

#3790: Bump vendored version of packaging to 23.0 (pyparsing is no longer required and was removed). As a consequence, users will experience a more strict parsing of requirements. Specifications that don’t comply with PEP 440 and PEP 508 will result in build errors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uninstalled wheel and confirmed it's still necessary as of (seemingly) latest pip 24.0:

  Running command Getting requirements to build wheel
  WARNING: numpy not available!  Array support will not be enabled

I'm not up to date with the bleeding edge of packaging, but I was still under the impression that wheel was necessary for anything compiled/non-sdist.

I think the venvs based off system python are secretly finding their way to wheel from the system install, which is why it makes it look like setuptools and wheel aren't needed, but I haven't done any tracing to verify.

All I can say for setuptools is that I tired setuptools==64 through setuptools==69 and 67.0 made it work.

FWIW, here's a trimmed down Dockerfile of what I'm working with:

ARG BASE_CONTAINER=ghcr.io/osgeo/gdal:ubuntu-full-3.8.4
FROM $BASE_CONTAINER

# Base tools and gdal stuff
RUN export DEBIAN_FRONTEND=noninteractive \
  && apt-get update && apt-get install -y \
    ca-certificates \
    nano tmux htop less git jq wget \
    gcc g++ make cmake \
    software-properties-common \
  && apt-add-repository -yn ppa:deadsnakes/ppa \
  && apt-get purge --autoremove -y software-properties-common \
  && echo 'UGFja2FnZTogKgpQaW46IHJlbGVhc2Ugbz1MUC1QUEEtZGVhZHNuYWtlcwpQaW4tUHJpb3JpdHk6IDYwMAo=' | base64 -d | tee /etc/apt/preferences.d/snakes_prefer \
  && apt-get update \
  && apt-get install -y python3.12 python3.12-dev python3.12-distutils python3.12-venv \
  && update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 10 \
  && python3.12 -m ensurepip --upgrade --default-pip \
  && apt-get install -y geotiff-bin \
  && rm -rf /var/lib/apt/lists/*

# NO GDAL YOU'RE BAD. Reinstall it! May need to blast numpy tbh.
RUN pip3 install --no-cache --upgrade 'setuptools>=67' 'pip>=23' wheel 'numpy>=1.24' \
  && pip3 uninstall -y GDAL \
  && pip3 install --no-cache --no-cache-dir --upgrade 'GDAL[numpy]=='"$(gdal-config --version)"

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not up to date with the bleeding edge of packaging, but I was still under the impression that wheel was necessary for anything compiled/non-sdist.

Historically I don't think this is the case, but I'm more than willing to be wrong.All of the different configurations in the current packaging world are super confusing.

I looked through the pip source code, and this message:

Running command Getting requirements to build wheel

means that pip is installing wheel in the build environment. I couldn't figure out exactly how, though.

Doing some more experimenting, it does seem like wheel is required to install the gdal source distribution. I would have expected the version of wheel that pip seems to install in the build environment to satisfy this requirement, but I guess not.

So, yes, it seems like your install command here is correct.

pip install gdal[numpy]

To install the version of the Python bindings matching your native GDAL library:
Expand Down
2 changes: 1 addition & 1 deletion swig/python/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[build-system]
requires = ["setuptools>=61.0.0", "oldest-supported-numpy", "wheel"]
requires = ["setuptools>=67.0.0", "oldest-supported-numpy", "wheel"]
build-backend = "setuptools.build_meta"

[project]
Expand Down
Loading