Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add numba + llvmlite #1286

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

add numba + llvmlite #1286

wants to merge 2 commits into from

Conversation

mschubert
Copy link
Contributor

These packages have been in ::science before but were removed because llvmlite only supported llvm=14, which was no longer in the main gentoo tree.

Now all tests pass with llvm=15, and numba is used a lot, so from my point of view it makes sense re-adding this ebuild.

Signed-off-by: Michael Schubert <mschu.dev@gmail.com>
Signed-off-by: Michael Schubert <mschu.dev@gmail.com>
@AndrewAmmerlaan
Copy link
Member

AndrewAmmerlaan commented Jul 16, 2024

LLVM 15 is going out, we need at least LLVM 16 support before we can consider re-adding this: https://bugs.gentoo.org/920536

@mschubert
Copy link
Contributor Author

Good point, and there is ongoing work to enable at least experimental LLVM 16 support (git head, no release yet).

I intended to follow up this PR with adding scanpy that depends on numba and is the most widely used single-cell analysis tool currently (with basically all groups working with this kind of data in python using it).

Do you agree that it would make sense adding it here?

This probably is/depends on too experimental software for the main tree or ::guru, and it would be nice if it was a bit more visible than just in my own overlay.

In that case, would you consider:

  1. Using the experimental LLVM 16 bindings
  2. Keeping the LLVM 15 ebuild in ::science for a while after it gets dropped from main

@AndrewAmmerlaan
Copy link
Member

AndrewAmmerlaan commented Jul 18, 2024

In that case, would you consider:
1. Using the experimental LLVM 16 bindings

That depends on how experimental it is and if upstream will in the future actually keep up with llvm. Adding it now is pointless if we will have to remove it again when llvm:16 is removed and upstream did not port to llvm:17 in time.

2. Keeping the LLVM 15 ebuild in `::science` for a while after it gets dropped from main

I'm not going to maintain llvm/clang here, that is way too much work. And there is the additional complication of llvm:15 not being compatible with python:3.12, which will become a big problem when python:3.11 is dropped.

@mschubert
Copy link
Contributor Author

Ok.

That depends on how experimental it is

I think that's the version that goes into the conda-dev channel with passing CI, so should be good enough

and if upstream will in the future actually keep up with llvm

They said that they are moving as fast as they can, but that is not very fast

@littlewu2508
Copy link
Contributor

BTW I got build error for numba-0.60.0:

error: Command "x86_64-pc-linux-gnu-g++ -shared -Wl,-O1 -Wl,--as-needed -O2 -pipe -march=znver2 -DNDEBUG /tmp/portage/dev-python/numba-0.60.0/work/numba-0.60.0-python3_11/build/temp.linux-x86_64-cpython-311/numba/np/ufunc/gufunc_scheduler.o /tmp/portage/dev-python/numba-0.60.0/work/numba-0.60.0-python3_11/build/temp.linux-x86_64-cpython-311/numba/np/ufunc/tbbpool.o -L/usr/lib/intel64/gcc4.4 -L/usr/lib -L/usr/lib/intel64/vc_mt -L/usr/lib64 -ltbb -o /tmp/portage/dev-python/numba-0.60.0/work/numba-0.60.0-python3_11/build/lib.linux-x86_64-cpython-311/numba/np/ufunc/tbbpool.cpython-311-x86_64-linux-gnu.so" " failed with exit status 1

ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/../../../../lib64/crti.o is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/crtbeginS.o is incompatible with elf32-i386
ld: error: /tmp/portage/dev-python/numba-0.60.0/work/numba-0.60.0-python3_11/build/temp.linux-x86_64-cpython-311/numba/np/ufunc/gufunc_scheduler.o is incompatible with elf32-i386
ld: error: /tmp/portage/dev-python/numba-0.60.0/work/numba-0.60.0-python3_11/build/temp.linux-x86_64-cpython-311/numba/np/ufunc/tbbpool.o is incompatible with elf32-i386
ld: error: /usr/lib64/libtbb.so is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libstdc++.so is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc_s.so.1 is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_muldi3.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_negdi2.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_lshrdi3.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_ashldi3.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_ashrdi3.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_cmpdi2.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_ucmpdi2.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_clear_cache.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_trampoline.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(__main.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_absvsi2.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_absvdi2.o) is incompatible with elf32-i386
ld: error: /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc.a(_addvsi3.o) is incompatible with elf32-i386
ld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
collect2: error: ld returned 1 exit status

Turns out the -L/usr/lib is responsible for this

@littlewu2508
Copy link
Contributor

os.path.join(tbb_root, 'lib', 'intel64', 'gcc4.4') in setup.py should be patched

@littlewu2508
Copy link
Contributor

These packages have been in ::science before

It seems that you removed all the hacks of past ebuilds. However when I installed numba from this PR, it's not usable:

AttributeError: /usr/lib/python3.11/site-packages/llvmlite/binding/libllvmlite.so: undefined symbol: LLVMPY_GetVersionInfo

I added back the old llvmlite and numba ebuild, only modified LLVM_MAX_SLOT and numpy dependency string, and numba works. So those hacks in the removed ebuilds are necessary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants