nix: nixBuildPackage’s depRoots are costly on the store

[nomeata]

I recently ran out of inodes on my /nix store partition, and I think nixBuildPackage’s depRoot derivations are the cause.

Status quo

Consider a module dependency graph A ← B ← C. Right now, nixBuildPackage will create 6 store paths, roughly like this:

/nix/store/…-A-depRoot:
  (empty)
/nix/store/…-A:
  A.olean
/nix/store/…-B-depRoot:
  A.olean → /nix/store/…-A/A.olean
/nix/store/…-B:
  B.olean
/nix/store/…-B-depRoot:
  A.olean → /nix/store/…-A/A.olean
  B.olean → /nix/store/…-B/B.olean
/nix/store/…-C:
  C.olean

So for each module, there is a -depRoot derivation that contains a symlinks to all the olean’s of its (transitive) dependencies. So if we have a dependency chain of length n, we’ll add O(n²) symlinks to the store.

The problem

These symlinks are small, but still take up considerable filesystem resources.

Moreover, these depRoot store paths don’t seem to be that useful: Mod-depRoot is only re-used when Mod changes. For all unchanged modules, the -depRoot path is not used again. And for all modules whose dependencies change, the -depRoot is not used either.

Moreover, having two store paths for each module doubles the time spent on querying nix caches when building.

A possible fix

Given that analysis (if it turns out to be accurate), it seems reasonable to just not persist the depRoot store path used when building a module, and instead just create it temporarily when building the module.

It’s still useful to have such a store path for the overall library (the foo.modRoot output).

Extension: lean-deps file

Right now, the /nix/store/…-C store path does not depend on anything. Is it ever useful to have C.olean without A.olean and B.olean? It might be prudent to add a text file lean-deps to /nix/store/…-C that lists the (direct) dependencies, so that nix knows about that relation.

Also, the derivation build to build, say, Mathlib-depRoot (or, with this proposal, Mathlib) is rather large (1.3MB), because it lists all it’s dependencies, twice, in deps and depRoots, and then again twice in inputDrvs. I don't have evidence, but I wonder if this might be rather costly in terms of nix evaluation etc. Probably the derivation doesn’t have to mention the indirect dependencies, and could use the lean-deps text file to list all the dependencies.

As before, this is not highly critical, but I wanted to write it down. If @Kha roughly agrees with the analysis and conclusion, I might give it a try at some point.

[Kha]

Yes, this is definitely something I've considered changing before. I don't remember all that many details but one weak reason was that changing it would make nix develop on a module derivation less convenient.

Is it ever useful to have C.olean without A.olean and B.olean?

Not right now, but with a module system that would change.

I wonder if this might be rather costly in terms of nix evaluation

Heh, I would not be surprised. We're taking Nix out of its comfort zone in various ways here.

[Kha]

The Nix build has been officially deprecated #4895