Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use relative path for sources in setup.py #1130

Closed
mglisse opened this issue Sep 11, 2024 · 4 comments
Closed

Use relative path for sources in setup.py #1130

mglisse opened this issue Sep 11, 2024 · 4 comments
Labels
3.11.0 GUDHI version 3.11.0 build The build system (CMake, etc)

Comments

@mglisse
Copy link
Member

mglisse commented Sep 11, 2024

https://setuptools.pypa.io/en/latest/userguide/ext_modules.html#extension-api-reference says about the parameters of setuptools.Extension

sources (list[str]) – list of source filenames, relative to the distribution root (where the setup script lives), in Unix form (slash-separated) for portability.

However, we have in setup.py.in sources = [source_dir + module + '.pyx',], where source_dir='@CMAKE_CURRENT_SOURCE_DIR@/gudhi/', i.e. an absolute path.

While this has been working so far, it seems to be the cause (one of the causes) of our current difficulties building for python 3.13 on windows on conda-forge.

(relevant ref: pypa/setuptools#4645)

@mglisse mglisse added the build The build system (CMake, etc) label Sep 11, 2024
@VincentRouvreau
Copy link
Contributor

I gave a try to a relative one: source_dir='../../../src/python/gudhi', but:

gcc -pthread -B /miniconda/envs/gudhi-devel/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /miniconda/envs/gudhi-devel/include -fPIC -O2 -isystem /miniconda/envs/gudhi-devel/include -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/miniconda/envs/gudhi-devel/lib/python3.11/site-packages/numpy/core/include -I/workspace/gudhi/gudhi-devel/src/python/gudhi/ -I/miniconda/envs/gudhi-devel/include/eigen3 -I/workspace/gudhi/gudhi-devel/ext/hera/include -I/miniconda/envs/gudhi-devel/include -I/workspace/gudhi/gudhi-devel/src/common/include -I/workspace/gudhi/gudhi-devel/src/Alpha_complex/include -I/workspace/gudhi/gudhi-devel/src/Bitmap_cubical_complex/include -I/workspace/gudhi/gudhi-devel/src/Bottleneck_distance/include -I/workspace/gudhi/gudhi-devel/src/Collapse/include -I/workspace/gudhi/gudhi-devel/src/Contraction/include -I/workspace/gudhi/gudhi-devel/src/Coxeter_triangulation/include -I/workspace/gudhi/gudhi-devel/src/Cech_complex/include -I/workspace/gudhi/gudhi-devel/src/Hasse_complex/include -I/workspace/gudhi/gudhi-devel/src/Persistence_representations/include -I/workspace/gudhi/gudhi-devel/src/Persistent_cohomology/include -I/workspace/gudhi/gudhi-devel/src/Rips_complex/include -I/workspace/gudhi/gudhi-devel/src/Simplex_tree/include -I/workspace/gudhi/gudhi-devel/src/Skeleton_blocker/include -I/workspace/gudhi/gudhi-devel/src/Spatial_searching/include -I/workspace/gudhi/gudhi-devel/src/Subsampling/include -I/workspace/gudhi/gudhi-devel/src/Tangential_complex/include -I/workspace/gudhi/gudhi-devel/src/Toplex_map/include -I/workspace/gudhi/gudhi-devel/src/Witness_complex/include -I/workspace/gudhi/gudhi-devel/src/Nerve_GIC/include -I/workspace/gudhi/gudhi-devel/src/Persistence_matrix/include -I/workspace/gudhi/gudhi-devel/src/python/include -I/miniconda/envs/gudhi-devel/include/python3.11 -c ../../../src/python/gudhi/off_utils.cpp -o build/temp.linux-x86_64-cpython-311/../../../src/python/gudhi/off_utils.o -DBOOST_RESULT_OF_USE_DECLTYPE -DBOOST_ALL_NO_LIB -DBOOST_SYSTEM_NO_DEPRECATED -std=c++17 -frounding-math -DCGAL_EIGEN3_ENABLED -DCGAL_HEADER_ONLY -DCGAL_USE_GMP -DCGAL_USE_GMPXX -DCGAL_USE_MPFR
Assembler messages:
Fatal error: can't create build/temp.linux-x86_64-cpython-311/../../../src/python/gudhi/off_utils.o: No such file or directory

source_dir is waiting for a relative path but doesn't ../. I do not see how we could workaround this new behaviour.

@mglisse
Copy link
Member Author

mglisse commented Sep 13, 2024

Fatal error: can't create build/temp.linux-x86_64-cpython-311/../../../src/python/gudhi/off_utils.o: No such file or directory

😞 so much for the easy way.

I do not see how we could workaround

Apparently they want the sources and setup.py in the same place. Some possibilities would be

  • move the generated setup.py next to the sources
  • copy the sources next to setup.py
  • create a (sym)link to the source dir in the build dir
  • :trollface: completely rewrite the build system without setuptools
  • monkey patching distutils from setup.py (setuptools does something strange that makes it more complicated than usual? Trying to patch things in setuptools._distutils is undone during the (first) call to setup)
# After the existing imports
import os
from distutils.ccompiler import CCompiler
CCompiler._make_relative = lambda self, base : os.path.splitdrive(base)[1].lstrip(r'\/')

this new behaviour.

Supposedly what we were doing was never supported. We can keep our fingers crossed that they will "fix" the recent regression, but they don't promise that it won't break again.

@VincentRouvreau
Copy link
Contributor

I gave a try to symbolic links and it seems to work nicely on windows, osx and linux. CMake 3.14 is required, but cmake_minimum_required is already 3.15.

@VincentRouvreau
Copy link
Contributor

Fixed by #1131

@VincentRouvreau VincentRouvreau added the 3.11.0 GUDHI version 3.11.0 label Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11.0 GUDHI version 3.11.0 build The build system (CMake, etc)
Projects
None yet
Development

No branches or pull requests

2 participants