build: macOS package notarization

[rvagg]

Works locally, and I can cleanly install .pkg's created by this on Catalina, but there are some things we need to sort out before it can be deployed:

• NOTARIZATION_ID in the ci-release job to (1) signal that notarization is required after .pkg creation and (2) point to the correct Apple ID.
• An AC_PASSWORD entry in the Keychain on the release machine(s) for the Apple ID identified by NOTARIZATION_ID (xcrun altool --store-password-in-keychain-item "AC_PASSWORD" -u "<appleid>" -p '<password>' (I'm documenting this in nodejs/secrets and I think there's another place we have release machine docs in nodejs/build). (giving up on keychain, see jenkins & docs: macOS 10.15 build#2199).
• Xcode 11 on our release machines - we're going to have to bump everything to 11, even for Node 10. In the process we probably may as well ditch older release machines (we're still using 10.10 and 10.11 but are working on more modern machines). There's some risk involved in this but I think Build is mostly in agreement that it's minimal.
• A patch to gon, the tool I'm using here to handle the messy bits of the notarization process. Unfortunately we have an unsigned executable inside our packages, /usr/local/lib/node_modules/npm/node_modules/term-size/vendor/macos/term-size, used by a dependency of a dependency of npm's. Apple reports this as an "issue" but still notarizes the package. gon treats all issues as fatal because this can mean the package isn't installable (see Download notarization log, parse issues, error if any issues mitchellh/gon#6), but in our case, my testing suggests that this isn't a problem at all, I think because it all ends up being command-line applications anyway. I have a proposal @ IgnorePathIssues config option to treat some issues as non-fatal mitchellh/gon#14 (which is used here, "ignorable_path_issues") but we'll see how that goes.

Includes @gdams' hardened-runtime patch from #29216 (comment)

@nodejs/build @nodejs/platform-macos

Ref: #29216

[gdams]

Thanks for taking this on @rvagg! Let me know if you have any issues with my patch!

[gdams]

Unfortunately we have an unsigned executable inside our packages, /usr/local/lib/node_modules/npm/node_modules/term-size/vendor/macos/term-size, used by a dependency of a dependency of npm's. Apple reports this as an "issue" but still notarizes the package.

The only thing to be aware of is that going forward this won't work as Apple has only temporarily loosened the notarization requirements. (That warning will soon become an error).

One possible approach would be to codesign everything in node_modules with the Foundation codesign cert. You can add --deep I believe to loop through all files and folders.

[rvagg]

One possible approach would be to codesign everything in node_modules with the Foundation codesign cert. You can add --deep I believe to loop through all files and folders.

How sure are you of this? It doesn't raise any issues on the install side. Have they stated that all "issues" are going to be fatal in the future?

That will mean that we're taking responsibility, reputationally, for term-size. I suppose we're doing that anyway for everything we package, so maybe there's no difference? It just doesn't feel right to be signing other people's binaries!

[gdams]

Have they stated that all "issues" are going to be fatal in the future?

See https://developer.apple.com/news/?id=12232019a. Essentially after February 3 2020, the original requirements will come back into place (Those warnings about Unsigned code will turn to errors)

That will mean that we're taking responsibility, reputationally, for term-size. I suppose we're doing that anyway for everything we package, so maybe there's no difference? It just doesn't feel right to be signing other people's binaries!

I'm kind of torn on this, part of me thinks that we should be taking responsibility, reputationally for everything that we ship but I'd much rather work with the developers to put it right at their end. The main issue is that it requires a codesign cert which cost money and therefore can be challenging for small open-source projects.

If we took on the role of codesigning all the binaries we would also have to assume a set of entitlements. One possible solution might be to check each binary and only sign what is not already signed (term-size in this case)

[rvagg]

OK, so I've pushed a commit that does a codesign for term-size ("$PKGDIR"/lib/node_modules/npm/node_modules/term-size/vendor/macos/term-size) as well as node ("$PKGDIR"/bin/node) and this gets it past the notarization without any warnings. It's using the same entitlements plist, which is probably overkill, but I don't think that matters too much does it @gdams? Or maybe we should craft a new one just for term-size? I don't really know how but I can see some things in there that probably shouldn't (allow-jit, allow-unsigned-executable-memory, disable-executable-page-protection, probably the other two as well!).

I asked sindresorhus about bundling a signed version of the executable but I also think it's reasonable to say no to that request. We just have to be comfortable with codesigning code we aren't really responsible for.

[gdams]

@rvagg I would leave it as the same entitlements for now. We can always narrow it down later, if you've got a clean report without any warnings then you should be good to go.

Just be prepared for more packages to possibly appear later on that also aren't codesigned.

[sam-github]

@rvagg I know sindrehous by reputation, I don't suspect him even a bit of being untrustworthy.... but I didn't find the code to the term-size executable, so we are notarizing an executable that has unknown capabilities. Of course, even if the code was reviewable, that doesn't guarantee the executable came from that code. Maybe that's a good thing about notarization, if its notarized as only being able to run and query term widths (whatever capability that is), but then the exe tries to do anything else, alaram bells will go off? For that purpose, its probably worth the extra effort to reduce its capabilities.

I was looking for the code because if its a feature that is important enough npm is going to depend on it, then perhaps it should be a uv/node.js API. I can see why he provides the two executables rather than a C++ addon, its a clever workaround for the nastiness of addons, but I wish there was another way.

[sindresorhus]

I asked sindresorhus about bundling a signed version of the executable but I also think it's reasonable to say no to that request. We just have to be comfortable with codesigning code we aren't really responsible for.

I will look into getting it notarized.

I'm also happy to have you build the binary, notarize it, and include the binary you built into term-size if that would be even better? It only has to be done once as the code is unlikely to change. The source is here and it's tiny: https://github.com/sindresorhus/macos-term-size/blob/master/term-size.c, so you could probably easily decompile the existing binary to confirm too.

I'm also willing to explore other options. The module will work without those binaries, so there is an option for you to just run a script to remove them.

I was looking for the code because if its a feature that is important enough npm is going to depend on it, then perhaps it should be a uv/node.js API. I can see why he provides the two executables rather than a C++ addon, its a clever workaround for the nastiness of addons, but I wish there was another way.

I think it's definitely something that should be supported by Node.js built-in.

I can see why he provides the two executables rather than a C++ addon, its a clever workaround for the nastiness of addons, but I wish there was another way.

If only Node.js had built-in FFI support, we could solve most of these problems in user-land without binaries or messy native addons...

[rvagg]

@nodejs/build would there be any objections to submitting a PR with a codesigned version of this binary to @sindresorhus' term-size repo using the "Node.js Foundation" codesigning certificate?
It would set a precedent of course, not just for this repo but anything that npm wants to bundle that has an executable. But a single signing means that we'll compile it and be verifying the specific code that the binary comes from and locks it in place, gating updates via access to the certificate.

btw @sindresorhus if you wanted to do this yourself you only need to codesign, not notarize, but you'd need to compile with a hardened runtime profile too.

[sam-github]

No objection from me.

I thought it might be possible to do what term-size does in node, but it didn't work. It should not be so hard :-(, maybe worth opening an issue about. But that's a bit tangential to getting what we have notarized now.

% nvm run 13 -p 'require("readline").createInterface({input: require("fs").createReadStream("/dev/tty"), output: require("fs").createWriteStream("/dev/tty")}).getCursorPos()'                                                                                                                    
Running node v13.6.0 (npm v6.13.4)                                                                                                                                            
readline.js:755                                                                                                                                                               
  const strBeforeCursor = this._prompt + this.line.substring(0, this.cursor);
                                                   ^
                                                                                                                                                                              
TypeError: Cannot read property 'substring' of undefined                                                                                                                      
    at Interface.getCursorPos (readline.js:755:52)                        

[rvagg]

Removed the "ignore warnings" feature, added a note in codesign.sh and pointed back here.

This is ready for review I think. It's still pending new build machines that we can switch to and start producing these, plus some agreement from @nodejs/build that we're committing to switching to Xcode 11 and the latest macOS for all of our release builds.

I've removed this env var from the pkg build script in iojs+release in ci-release: NOTARIZATION_ID=build@iojs.org, the lack of that env var will mean the notarization isn't invoked. But when it's present it'll try to run notarization, so we should put that back when we're ready.

I believe that macOS release machines are pending in nodejs/build#1695, we have some new boxes being set up at nearForm. We're currently having a bit of trouble getting responses from MacStadium about whether we can get access to newer hardware (and software). Tracking that in nodejs/build#1732

[richardlau]

@nodejs/build would there be any objections to submitting a PR with a codesigned version of this binary to @sindresorhus' term-size repo using the "Node.js Foundation" codesigning certificate?
It would set a precedent of course, not just for this repo but anything that npm wants to bundle that has an executable. But a single signing means that we'll compile it and be verifying the specific code that the binary comes from and locks it in place, gating updates via access to the certificate.

If we're codesigning the term-size binary with the Node.js Foundation certificate I'd much prefer that we compiled it from source, codesigned that and then PR'ed that into https://github.com/sindresorhus/term-size. It's a much simpler audit trail than trying to track backwards that the binary hasn't been tampered with (and plus would mean signing the binary once rather than for every release we do).

[sindresorhus]

If we're codesigning the term-size binary with the Node.js Foundation certificate I'd much prefer that we compiled it from source , codesigned that and then PR'ed that into sindresorhus/term-size.

Yes, that's what I'm suggesting.

[rvagg]

Yes, that's a TODO, but it needs some bandwidth to write a signing script, figuring out the right hardening configuration, submit that back to macos-term-size and then a binary to term-size.

I'm deferring that right now because I don't have that bandwidth, maybe later in the week.

If this is a requirement to merging this PR, know that it'll delay getting notarized builds of Node out because we have to go through a chain of releases of term-size -> boxen -> npm, then an npm PR back to nodejs/node, then a backport of that npm to all active release lines.

Which is fine, but expect Catalina users to squawk louder as the Apple turn on strict-mode on February 3rd and they can no longer install .pkgs.

Getting release machines sorted out may be more of a bother than we anticipate too.

[rvagg]

btw there is a shortcut to this dependency problem of waiting for npm to land with a signed term-size, we just merge one in to our tree inside npm, just got to get it compiled and signed though.

[rvagg]

https://github.com/nodejs-private/secrets/pull/75 has hopefully enough stuff to get a new release machine set up, @sam-github @AshCripps let me know how you go with the nearForm machines. From memory, doing things from cmdline is a pain in the backside because it requires proper keyboard interactive passwords for some key things. I don't know what the current state of the ansible scripts is, hopefully that has all that's needed. 🤞

[rvagg]

Hey folks in this issue, @sindresorhus has a new release of term-size out so there's a new binary that's not signed. I don't know if npm is still using term-size and what a timeline might look like for it making its way here but the flag is that we may have notarization issues if/once it bubbles its way up here.

We previously contributed a signed binary to make everything happy, we may need to do so again. Heads-up: sindresorhus/macos-terminal-size#5

I'll let someone else make the call wrt necessity and urgency but I can help get the signing done if that's needed, and the help is needed.