[WIP] ansible: Edit select-compiler.sh to select different versions of xcode

[AshCripps]

Edited select-compiler.sh to change the xcode version used based on which version of node is being built.

Works on my local machine but needs to be checked to ensure the paths are correct on the build machines.

Will also require sudoers being edited to allow for xcode-select to run with sudo permissions.

Discussion here - #1964

[mhdawson]

How does this select a different version? /Applications/Xcode.app/Contents/Developer looks identical to the section for 13?

[AshCripps]

Atm it is identical becauase 12 and 13 both build on 10 +. The actual min. xcode doesnt change to version 10 until node 13 (according to BUILDING.md) so the question is do we want to use latest for these versions or hardcode in a stable version and only change it if required (security patches etc.).

For example /Applications/Xcode.11.1.app/Contents/Developer for node 13 and then something like /Applications/Xcode10.x.app/Contents/Developer for node 12.

Depends if we want as few xcodes on the machine as possible or if we want a stable xcode for each node release?

[sam-github]

stable for each release, decided at time of first .0.0 for that release line, never changed afterwards.

[AshCripps]

Are these documented anywhere as to which version is used for each release?

[sam-github]

in the BUILDING.md on the relevant release branch of nodejs/node, if documented, if not.... #1522 :-(

[rvagg]

We're locked in to 8 for Node 12. We've historically locked in for the entirety of a release line. That could change and it would be lower risk on macOS than others, but not zero risk and the TSC would have to sign off on it and Build WG would shoulder the blame if things break. So .. for now consider it locked to Xcode 8 for Node 12.

[sam-github]

I think prev line should have been 8

[sam-github]

@AshCripps Would be good to add:

1. updates to https://github.com/nodejs/build/blob/master/ansible/MANUAL_STEPS.md#macos to describe how the sudoers needs to be setup
2. setup... how we will get the xcodes on the machine. There is a setup script: ./ansible/roles/package-upgrade/files/install-xcode.sh , can it do multiple versions?

[AshCripps]

2. setup... how we will get the xcodes on the machine. There is a setup script: ./ansible/roles/package-upgrade/files/install-xcode.sh , can it do multiple versions?

ATM it currently does 2 different versions based on what version of macos is on the box. So it should be easy enough to change it ive already tracked down where URLs for the different versions are so should be a simple change to install multiple versions

[AshCripps]

build/ansible/roles/package-upgrade/files/install-xcode.sh

Lines 7 to 15 in 9735105

	if [[ "$osx_vers" -ge 9 ]]; then
	touch "$cmd_line_tools_temp_file";
	PROD=$(softwareupdate -l |
	grep "\*.*Command Line" |
	head -n 1 | awk -F"*" '{print $2}' |
	sed -e 's/^ *//' |
	tr -d '\n')
	softwareupdate -i "$PROD";
	fi

The above lines don't work on the new nearform machines so the install-xcode script will have to be reworked to install xcode on the newer machines.

Also I found this blog post https://derflounder.wordpress.com/2018/06/10/updated-xcode-command-line-tools-installer-script-now-available/ which looks remarkably similar to what we have and the original file predates install-xcode.sh by about 3 years but the repo is under the MIT license but I don't see any link to the author or repo?

[AshCripps]

Ive edited the install-xcode script to allow for the installing of xcode - there are some problems here

• Firstly the xcodes will need to be hosted on a server somewhere - the apple site requires a login to download the files and this is tricky on command line
• Secondly it seems the xip files have certificates that expire as some old downloads I have from the other week I can no longer unzip
• These are full blown xcodes on the new oses - this is required for notorization but also makes xcode-select easier IMO but they are big files ~10GB

Ive also edited the MANUALSTEPS.md to include the sudo line needed to run xcode-select

Next step is to sort out the select compiler script to change xcodes based on the OS version and what version of node is being built

[rvagg]

We have a spot on ci.nodejs.org where we put files required by our infra. See aix61-standalone/manualBootstrap.md for some examples of using https://ci.nodejs.org/downloads/. I don't see anything in Ansible but there was a time when I was doing ARMv7 compiler stuff on there too and I see the files are still there.
There might be problems with making these publicly available as there are probably redistribution restrictions on the files. Can you see anything associated with the files that says anything about redistribution? I don't have any great ideas for limiting distribution without opening up whole new security holes or setting up new dedicated resources.

[rvagg]

the xip files have certificates that expire

Oh, what? so even if we stored these files they will stop being extractable? So we're going to have to put ourselves at the mercy of Apple continue to supply these files?

What's the structure inside them? Is it practical to unpack and pack them up again into a format that we can properly archive for ourselves?

[AshCripps]

What's the structure inside them? Is it practical to unpack and pack them up again into a format that we can properly archive for ourselves?

Not sure, when you extract them you got a .app file which is actually a directory I believe as the xcode path becomes /Applications/Xcode.10.3.app/Contents/Developer

Ill have to try packing it up myself and see if someone else can reuse it.

[AshCripps]

Ill have to try packing it up myself and see if someone else can reuse it.

Tried this and it didnt work - showed an not from apple warning on both 10.14 and 10.15, you could work around it with a cmd-click but not sure this works on command line

[AshCripps]

Closing this, dont think this will be needed as notorization has forced us to update the release machines to new OS