Fix CI for numpy 2.X support

[VincentRouvreau]

• Fix Pypi wheel build for gudhi 3.10.0rc1 (hopefully)
• new docker images from gudhi-deploy 2024.06.02
• python from 3.7 to 3.9 for Azure (OSx and Windows)
• PyKeops is now available on windows

[mglisse]

Do you have a source for this one? As far as I can see the issue on windows support is still open.

[VincentRouvreau]

I do not have the source, but in the logs it works

[mglisse]

In which logs?
(Reminder: it was possible to install pykeops, but trying to use it (even a plain import) did not work, and that's still what I am seeing)
It does not seem to hurt currently (CMake detects it as missing and disables the tests that would use it), but when we improve the detection of python modules, it may become an issue.

[VincentRouvreau]

Oh, yes you are right in Install build dependency:

• l.88:

Collecting pykeops (from -r ext/gudhi-deploy/test-requirements.txt (line 15))
  Downloading pykeops-2.2.3.tar.gz (92 kB)

• l.230:

Requirement already satisfied: pybind11 in c:\users\vssadministrator\appdata\roaming\python\python39\site-packages (from pykeops->-r ext/gudhi-deploy/test-requirements.txt (line 15)) (2.12.0)
Collecting keopscore==2.2.3 (from pykeops->-r ext/gudhi-deploy/test-requirements.txt (line 15))
  Downloading keopscore-2.2.3.tar.gz (100 kB)
     -------------------------------------- 100.3/100.3 kB 1.9 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'

• l. 435:

Building wheels for collected packages: sphinx-paramlinks, pykeops, keopscore, hnswlib
  Building wheel for sphinx-paramlinks (setup.py): started
  Building wheel for sphinx-paramlinks (setup.py): finished with status 'done'
  Created wheel for sphinx-paramlinks: filename=sphinx_paramlinks-0.6.0-py3-none-any.whl size=10142 sha256=cc7f7f304148aa8a2b7220595ba389983dad3829c0b520676d91874ce00636c7
  Stored in directory: c:\users\vssadministrator\appdata\local\pip\cache\wheels\c4\50\00\8029b47b05164cd8b62e70bd26d8adb1c997cd80381ae08034
  Building wheel for pykeops (setup.py): started
  Building wheel for pykeops (setup.py): finished with status 'done'
  Created wheel for pykeops: filename=pykeops-2.2.3-py3-none-any.whl size=119563 sha256=1eca9cc40ff279bc5a695bad22a0c849a41fb41126bfb1a6650f22811074c0c9
  Stored in directory: c:\users\vssadministrator\appdata\local\pip\cache\wheels\1d\d3\a0\405e7af8931a510e8c384c57b9e3ca7097f8322f6b2fe44971
  Building wheel for keopscore (setup.py): started
  Building wheel for keopscore (setup.py): finished with status 'done'
  Created wheel for keopscore: filename=keopscore-2.2.3-py3-none-any.whl size=173404 sha256=48758dc2047b35e032d4412fd4b590009a2740554750539ca9923ad8e58b75a2
  Stored in directory: c:\users\vssadministrator\appdata\local\pip\cache\wheels\ea\ff\c3\0abfd27081daa0ec3377a2425ce1e840c74c7477258e529f73
  Building wheel for hnswlib (pyproject.toml): started
  Building wheel for hnswlib (pyproject.toml): finished with status 'done'
  Created wheel for hnswlib: filename=hnswlib-0.8.0-cp39-cp39-win_amd64.whl size=145822 sha256=18fc07d915bd7d3b19878be0758d1e4321d3f27ae6a8301035d53430d6141abb
  Stored in directory: c:\users\vssadministrator\appdata\local\pip\cache\wheels\81\b3\1d\920a1ed1f9ede20be60d1108445e5789eef54c5144c9d746f6
Successfully built sphinx-paramlinks pykeops keopscore hnswlib
Installing collected packages: tbb, snowballstemmer, pytz, namex, mpmath, libclang, intel-openmp, flatbuffers, wrapt, urllib3, tzdata, typing-extensions, typeguard, threadpoolctl, termcolor, tensorflow-io-gcs-filesystem, tensorboard-data-server, sympy, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, sphinxcontrib-applehelp, soupsieve, six, PyYAML, pyparsing, Pygments, protobuf, pluggy, pillow, numpy, networkx, mkl, mdurl, MarkupSafe, latexcodec, kiwisolver, keopscore, joblib, iniconfig, importlib-resources, imagesize, idna, grpcio, gast, fsspec, fonttools, filelock, exceptiongroup, docutils, cycler, coverage, charset-normalizer, certifi, babel, alabaster, absl-py, werkzeug, tensorflow-addons, scipy, requests, python-dateutil, pytest, pykeops, pybtex, optree, opt-einsum, ml-dtypes, markdown-it-py, markdown, Jinja2, hnswlib, h5py, google-pasta, eagerpy, contourpy, beautifulsoup4, astunparse, accessible-pygments, torch, tensorboard, sphinx, scikit-learn, rich, pytest-cov, pybtex-docutils, POT, pandas, matplotlib, sphinxcontrib-bibtex, sphinx-paramlinks, pydata-sphinx-theme, keras, tensorflow-intel, tensorflow
  Attempting uninstall: numpy
    Found existing installation: numpy 2.0.0
    Uninstalling numpy-2.0.0:
      Successfully uninstalled numpy-2.0.0
Successfully installed Jinja2-3.1.4 MarkupSafe-2.1.5 POT-0.9.3 PyYAML-6.0.1 Pygments-2.18.0 absl-py-2.1.0 accessible-pygments-0.0.5 alabaster-0.7.16 astunparse-1.6.3 babel-2.15.0 beautifulsoup4-4.12.3 certifi-2024.6.2 charset-normalizer-3.3.2 contourpy-1.2.1 coverage-7.5.4 cycler-0.12.1 docutils-0.21.2 eagerpy-0.30.0 exceptiongroup-1.2.1 filelock-3.15.4 flatbuffers-24.3.25 fonttools-4.53.0 fsspec-2024.6.0 gast-0.5.4 google-pasta-0.2.0 grpcio-1.64.1 h5py-3.11.0 hnswlib-0.8.0 idna-3.7 imagesize-1.4.1 importlib-resources-6.4.0 iniconfig-2.0.0 intel-openmp-2021.4.0 joblib-1.4.2 keopscore-2.2.3 keras-3.4.0 kiwisolver-1.4.5 latexcodec-3.0.0 libclang-18.1.1 markdown-3.6 markdown-it-py-3.0.0 matplotlib-3.9.0 mdurl-0.1.2 mkl-2021.4.0 ml-dtypes-0.3.2 mpmath-1.3.0 namex-0.0.8 networkx-3.2.1 numpy-1.26.4 opt-einsum-3.3.0 optree-0.11.0 pandas-2.2.2 pillow-10.3.0 pluggy-1.5.0 protobuf-4.25.3 pybtex-0.24.0 pybtex-docutils-1.0.3 pydata-sphinx-theme-0.15.3 pykeops-2.2.3 pyparsing-3.1.2 pytest-8.2.2 pytest-cov-5.0.0 python-dateutil-2.9.0.post0 pytz-2024.1 requests-2.32.3 rich-13.7.1 scikit-learn-1.5.0 scipy-1.13.1 six-1.16.0 snowballstemmer-2.2.0 soupsieve-2.5 sphinx-7.3.7 sphinx-paramlinks-0.6.0 sphinxcontrib-applehelp-1.0.8 sphinxcontrib-bibtex-2.6.2 sphinxcontrib-devhelp-1.0.6 sphinxcontrib-htmlhelp-2.0.5 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.7 sphinxcontrib-serializinghtml-1.1.10 sympy-1.12.1 tbb-2021.13.0 tensorboard-2.16.2 tensorboard-data-server-0.7.2 tensorflow-2.16.1 tensorflow-addons-0.22.0 tensorflow-intel-2.16.1 tensorflow-io-gcs-filesystem-0.31.0 termcolor-2.4.0 threadpoolctl-3.5.0 torch-2.3.1 typeguard-2.13.3 typing-extensions-4.12.2 tzdata-2024.1 urllib3-2.2.2 werkzeug-3.0.3 wrapt-1.16.0

But finally in Build and tests:

• l. 16840

PYTHON_MODULE_NAME = pykeops
       - PYTHON_MODULE_RESULT = 1
       - PYTHON_MODULE_VERSION = [KeOps] Warning : Cuda libraries were not detected on the system or could not be loaded ; using cpu only mode

       - PYTHON_MODULE_ERROR = Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\VssAdministrator\AppData\Roaming\Python\Python39\site-packages\pykeops\__init__.py", line 76, in <module>
    from .numpy.test_install import test_numpy_bindings
  File "C:\Users\VssAdministrator\AppData\Roaming\Python\Python39\site-packages\pykeops\numpy\__init__.py", line 5, in <module>
    from .generic.generic_red import Genred
  File "C:\Users\VssAdministrator\AppData\Roaming\Python\Python39\site-packages\pykeops\numpy\generic\generic_red.py", line 4, in <module>
    from pykeops.common.operations import preprocess, postprocess
  File "C:\Users\VssAdministrator\AppData\Roaming\Python\Python39\site-packages\pykeops\common\operations.py", line 3, in <module>
    from pykeops.common.utils import get_tools
  File "C:\Users\VssAdministrator\AppData\Roaming\Python\Python39\site-packages\pykeops\common\utils.py", line 1, in <module>
    import fcntl
ModuleNotFoundError: No module named 'fcntl'

As Azure pipelines for windows is not for pip package build, I keep it like that.
I would need to change test-requirements.txt as my for loop in powershell was failing on comments.

[VincentRouvreau]

People tends to advise to make another requirements file named something like test-requirements-not-windows.txt where there is only pykeops in it. I think it is a good idea.

[VincentRouvreau]

cf. GUDHI/gudhi-deploy#35