Various fixes to get rules_cuda to work with Lambda-stack

[EthanSteinberg]

The rules for this repository do not currently work with the following commonly used CUDA configuration: https://lambdalabs.com/lambda-stack-deep-learning-software

The reason is because the Lambda deep stack installs CUDA in /usr/bin, which breaks a couple of assumptions with this software.

In particular:

• /usr/lib64 is the wrong path here, /usr/lib/x86_64-linux-gnu/ is correct.
• /usr/bin/** is invalid since it contains recursive symlinks

This fixes those two issues.

[cloudhan]

Just curious, in BUILD.local_cuda, all cc_import for .sos are non-versioned, will this work with Ubuntu's always versioned .so.<version>s?

[EthanSteinberg]

Ubuntu provides both versioned and unversioned copies so it works.

[cloudhan]

I need to add a test stage for this before a merge. In the future, we might need a process to re-compose a canonical tree, and it will also be useful for #72

[cloudhan]

@lalaland Could you please rebase on main and unconmment

rules_cuda/.github/workflows/build-tests.yaml

Line 20 in f2aed5e

# - { os: "ubuntu-22.04", cuda-version: "11.5.1-1ubuntu1", source: "ubuntu" }

[EthanSteinberg]

@cloudhan Rebased and enabled. Note that you will have to manually trigger the workflows.

[EthanSteinberg]

@cloudhan Can you try rerunning it? I think you forgot to include the nvidia toolkit in your build script. I just added it so it should work now.

[EthanSteinberg]

Hmm. It looks like ubuntu ships an incompatible gcc and nvidia toolkit in their default packages? That's really quite lame. I added some code to the build script to try to downgrade the compiler.

[EthanSteinberg]

Sorry for the bother, but could someone try triggering the workflow again? I think I finally have the solution.

[EthanSteinberg]

Looks like everything passes now! This should be good to merge.