make lld builds hermetic

[nickdesaulniers]

I'm finding out the hard way now, but I don't think the lld builds are necessarily hermetic (not using bfd at all); particularly for the arm64 vdso code. We should rename/mv aarch64-linux-gnu-ld for arm64 lld builds.

[nathanchance]

We'll want to do that in env-setup.sh since I don't think that's a good idea for force on everyone. I can handle that in an hour and a half or so if someone doesn't beat me to it.

[nickdesaulniers]

oh man, so --gcc-toolchain is set based on $LD. That's no bueno, and should be $AS for now...

[nathanchance]

Oh that's right... How has CI been working since ld.lld-9 isn't in /usr/bin...?

[nickdesaulniers]

just a guess, if --gcc-toolchain is given, and it's not a gcc toolchain, clang falls back to using aarch64-linux-gnu-ld from $PATH rather than warning that you dun messed up, A-A-RON!

[nickdesaulniers]

ok, I've got a set of patches for upstreaming, will post them tomorrow.

[nathanchance]

Ah, CI didn't catch this because...

root@57a824883302:/linux-next# which ld.lld-9
/usr/bin/ld.lld-9
root@57a824883302:/linux-next# ls -al $(which ld.lld-9)
lrwxrwxrwx 1 root root 24 Jan 29 08:26 /usr/bin/ld.lld-9 -> ../lib/llvm-9/bin/ld.lld
root@57a824883302:/linux-next# ls /usr/bin/aarch64*
/usr/bin/aarch64-linux-gnu-addr2line  /usr/bin/aarch64-linux-gnu-c++filt  /usr/bin/aarch64-linux-gnu-gprof   /usr/bin/aarch64-linux-gnu-ld.gold  /usr/bin/aarch64-linux-gnu-objdump  /usr/bin/aarch64-linux-gnu-size
/usr/bin/aarch64-linux-gnu-ar         /usr/bin/aarch64-linux-gnu-dwp      /usr/bin/aarch64-linux-gnu-ld      /usr/bin/aarch64-linux-gnu-nm       /usr/bin/aarch64-linux-gnu-ranlib   /usr/bin/aarch64-linux-gnu-strings
/usr/bin/aarch64-linux-gnu-as         /usr/bin/aarch64-linux-gnu-elfedit  /usr/bin/aarch64-linux-gnu-ld.bfd  /usr/bin/aarch64-linux-gnu-objcopy  /usr/bin/aarch64-linux-gnu-readelf  /usr/bin/aarch64-linux-gnu-strip

which doesn't resolve symlinks on its own so everything works when ld.lld and aarch64-linux-gnu-ld are in the same directory (as GCC_TOOLCHAIN_DIR will resolve to /usr/bin which will be passed along to --prefix= and GCC_TOOLCHAIN will be /usr).

This should make sure that the builds are hermetic, should I push it now or hold off until all of these other ld.lld issues are resolved?

diff --git a/env-setup.sh b/env-setup.sh
index 46a6e2b..90df470 100755
--- a/env-setup.sh
+++ b/env-setup.sh
@@ -20,3 +20,12 @@ ccache --set-config=cache_dir=/travis/.ccache
 
 # Clear out the stats so we actually know the cache stats
 ccache -z
+
+# Ensure that when compiling with LLD, all other linkers are not available
+case ${ARCH:=arm64} in
+    arm32*) CROSS_COMPILE=arm-linux-gnueabi- ;;
+    arm64) CROSS_COMPILE=aarch64-linux-gnu- ;;
+    ppc32) CROSS_COMPILE=powerpc-linux-gnu- ;;
+    ppc64le) CROSS_COMPILE=powerpc64le-linux-gnu- ;;
+esac
+[[ ${LD} =~ ld.lld ]] && command -v ${CROSS_COMPILE:-}{ld,ld.bfd,ld.gold} | while IFS=$'\n' read -r line; do rm "${line}"; done

[nickdesaulniers]

I think we should disable the lld builds for now as they're presenting a false positive signal. I'll work to get them turned back on.

[nathanchance]

Yeah that's a fair point. When we turn them back on, that change should ensure it never regresses.

[nathanchance]

I have a working POC for x86_64:

https://github.com/nathanchance/continuous-integration/commits/hermetic-builds

https://travis-ci.com/nathanchance/continuous-integration/builds/111671525