Can't build for a custom target with build-std

[lzcunt]

Haiii I'm trying to use crane to build a no_std crate with a custom JSON target and build-std, and it fails like this:

cargoArtifacts not set, will not reuse any cargo artifacts
Running phase: unpackPhase
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking source archive /nix/store/78ll3w4fr45zfhifz2zi4xggbadwzg0a-source
source root is source
Running phase: patchPhase
@nix { "action": "setPhase", "phase": "patchPhase" }
Executing configureCargoCommonVars
Running phase: updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
Running phase: configurePhase
@nix { "action": "setPhase", "phase": "configurePhase" }
will append /build/source/.cargo-home/config.toml with contents of /nix/store/x1jndww1xvrl9hvg2wiir4km9mj8ljzx-vendor-cargo-deps/config.toml
default configurePhase, nothing to do
Running phase: buildPhase
@nix { "action": "setPhase", "phase": "buildPhase" }
++ command cargo --version
cargo 1.83.0-nightly (8f40fc59f 2024-08-21)
++ command cargo check --release -Z build-std -p kernel --target /nix/store/p4gmb50dc0qs7af5bxp1kh909ah61wc9-x86_64-kernel.json
   Compiling compiler_builtins v0.1.123
   Compiling core v0.0.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/core)
   Compiling libc v0.2.158
   Compiling memchr v2.5.0
   Compiling std v0.0.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std)
   Compiling kernel v0.1.0 (/build/source/kernel)
   Compiling rustc-std-workspace-core v1.99.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/alloc)
   Compiling cfg-if v1.0.0
   Compiling adler v1.0.2
   Compiling rustc-demangle v0.1.24
   Compiling unwind v0.0.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/unwind)
   Compiling rustc-std-workspace-alloc v1.99.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/rustc-std-workspace-alloc)
   Compiling panic_unwind v0.0.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/panic_unwind)
   Compiling panic_abort v0.0.0 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/panic_abort)
   Compiling gimli v0.29.0
   Compiling hashbrown v0.14.5
   Compiling miniz_oxide v0.7.4
   Compiling std_detect v0.1.5 (/nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/stdarch/crates/std_detect)
   Compiling object v0.36.2
   Compiling addr2line v0.22.0
error[E0432]: unresolved imports `crate::sys::thread_local::key::get`, `crate::sys::thread_local::key::set`, `crate::sys::thread_local::key::Key`, `crate::sys::thread_local::key::LazyKey`
 --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/sys/thread_local/os.rs:5:37
  |
5 | use crate::sys::thread_local::key::{get, set, Key, LazyKey};
  |                                     ^^^  ^^^  ^^^  ^^^^^^^ no `LazyKey` in `sys::thread_local::key`
  |                                     |    |    |
  |                                     |    |    no `Key` in `sys::thread_local::key`
  |                                     |    no `set` in `sys::thread_local::key`
  |                                     no `get` in `sys::thread_local::key`
  |
note: module `crate::collections::hash::set` exists but is inaccessible
 --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/collections/hash/mod.rs:4:1
  |
4 | pub mod set;
  | ^^^^^^^^^^^^ not accessible

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
  --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/sys/alloc/mod.rs:59:42
   |
59 |         let new_ptr = GlobalAlloc::alloc(alloc, new_layout);
   |                       ------------------ ^^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
   |                       |
   |                       required by a bound introduced by this call

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
  --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/sys/alloc/mod.rs:63:34
   |
63 |             GlobalAlloc::dealloc(alloc, ptr, old_layout);
   |             -------------------- ^^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
   |             |
   |             required by a bound introduced by this call

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:142:47
    |
142 |                     GlobalAlloc::alloc_zeroed(self, layout)
    |                     ------------------------- ^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
    |                     |
    |                     required by a bound introduced by this call

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:144:40
    |
144 |                     GlobalAlloc::alloc(self, layout)
    |                     ------------------ ^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
    |                     |
    |                     required by a bound introduced by this call

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:178:52
    |
178 |                 let raw_ptr = GlobalAlloc::realloc(self, ptr.as_ptr(), old_layout, new_size);
    |                               -------------------- ^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
    |                               |
    |                               required by a bound introduced by this call

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:220:43
    |
220 |             unsafe { GlobalAlloc::dealloc(self, ptr.as_ptr(), layout) }
    |                      -------------------- ^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
    |                      |
    |                      required by a bound introduced by this call

error[E0277]: the trait bound `System: core::alloc::GlobalAlloc` is not satisfied
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:270:52
    |
270 |                 let raw_ptr = GlobalAlloc::realloc(self, ptr.as_ptr(), old_layout, new_size);
    |                               -------------------- ^^^^ the trait `core::alloc::GlobalAlloc` is not implemented for `System`
    |                               |
    |                               required by a bound introduced by this call

error[E0599]: no method named `alloc` found for struct `System` in the current scope
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:401:20
    |
132 | pub struct System;
    | ----------------- method `alloc` not found for this struct
...
401 |             System.alloc(layout)
    |                    ^^^^^
    |
    = help: items from traits can only be used if the trait is implemented and in scope
    = note: the following trait defines an item `alloc`, perhaps you need to implement it:
            candidate #1: `core::alloc::GlobalAlloc`
help: there is a method `allocate` with a similar name
    |
401 |             System.allocate(layout)
    |                    ~~~~~~~~

error[E0599]: no method named `dealloc` found for struct `System` in the current scope
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:409:25
    |
132 | pub struct System;
    | ----------------- method `dealloc` not found for this struct
...
409 |         unsafe { System.dealloc(ptr, Layout::from_size_align_unchecked(size, align)) }
    |                         ^^^^^^^
    |
    = help: items from traits can only be used if the trait is implemented and in scope
    = note: the following trait defines an item `dealloc`, perhaps you need to implement it:
            candidate #1: `core::alloc::GlobalAlloc`
help: there is a method `deallocate` with a similar name
    |
409 |         unsafe { System.deallocate(ptr, Layout::from_size_align_unchecked(size, align)) }
    |                         ~~~~~~~~~~

error[E0599]: no method named `realloc` found for struct `System` in the current scope
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:423:20
    |
132 | pub struct System;
    | ----------------- method `realloc` not found for this struct
...
423 |             System.realloc(ptr, old_layout, new_size)
    |                    ^^^^^^^ method not found in `System`
    |
    = help: items from traits can only be used if the trait is implemented and in scope
    = note: the following trait defines an item `realloc`, perhaps you need to implement it:
            candidate #1: `core::alloc::GlobalAlloc`

error[E0599]: no method named `alloc_zeroed` found for struct `System` in the current scope
   --> /nix/store/rs2w80amnvwlgs95jv28mk8m0h7zq75q-rust-default-1.83.0-nightly-2024-09-02/lib/rustlib/src/rust/library/std/src/alloc.rs:433:20
    |
132 | pub struct System;
    | ----------------- method `alloc_zeroed` not found for this struct
...
433 |             System.alloc_zeroed(layout)
    |                    ^^^^^^^^^^^^
    |
    = help: items from traits can only be used if the trait is implemented and in scope
    = note: the following trait defines an item `alloc_zeroed`, perhaps you need to implement it:
            candidate #1: `core::alloc::GlobalAlloc`
help: there is a method `allocate_zeroed` with a similar name
    |
433 |             System.allocate_zeroed(layout)
    |                    ~~~~~~~~~~~~~~~

Some errors have detailed explanations: E0277, E0432, E0599.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `std` (lib) due to 12 previous errors

How can I fix this and is this (custom JSON targets) supported by crane? Here is the package I'm trying to build for more context:

{
  buildPackages,
  craneLib,
  release ? false
}: let
  src = craneLib.cleanCargoSource ./.;
in {
  kernel = craneLib.buildPackage {
    inherit (craneLib.crateNameFromCargoToml {
      cargoToml = ./kernel/Cargo.toml;
    }) pname version;
    inherit src;

    cargoVendorDir = craneLib.vendorMultipleCargoDeps {
      inherit (craneLib.findCargoFiles ./kernel) cargoConfigs;
      cargoLockList = [
        ./Cargo.lock

        # Unfortunately this approach requires IFD (import-from-derivation)
        # otherwise Nix will refuse to read the Cargo.lock from our toolchain
        # (unless we build with `--impure`).
        #
        # Another way around this is to manually copy the rustlib `Cargo.lock`
        # to the repo and import it with `./path/to/rustlib/Cargo.lock` which
        # will avoid IFD entirely but will require manually keeping the file
        # up to date!
        "${buildPackages.rustToolchain.passthru.availableComponents.rust-src}/lib/rustlib/src/rust/library/Cargo.lock"
      ];
    };

    cargoExtraArgs = "-Z build-std -p kernel --target ${./kernel/targets/x86_64-kernel.json}";

    # buildType = if release then "release" else "debug";
    doCheck = false;
    # dontPatchELF = true;
  };
}

[ipetkov]

Hi @AsyaTheAbove do you have a(n ideally minimal) reproduction flake that i can try out?

I don't have any experience building with custom targets, and it's hard for me to tell if there is something wrong with the build configuration or if the custom target itself is missing something

[lzcunt]

I got it to work, I think it was something to do with the depsOnly build not including config.toml but I fixed it. I think there could be a simple example for this tho

[ipetkov]

Sure, if you have a simple/pared down example of building with a custom target I'd be happy to review a PR!

[lzcunt]

I have a not-so-simple example but it can be adapted to be even simpler. It would be great if you could also check it out and tell me if there's anything wrong I did there. Sadly I haven't published one of its dependencies yet so it's not buildable

[ipetkov]

I guess the interesting parts to include are any build configs needed to build the stdlib (e.g. how to specify a custom target)? Anything else can be stubbed/todo!() code, whatever makes the example as short and interesting as possible!

[Patryk27]

@AsyaTheAbove try passing -Z build-std=core (or alloc,core etc., depending on what you need) instead of just -Z build-std.

I think that's a regression (or just a change?) in recent toolchains - I'm facing the same problem in my risc-v project, but passing explicit -Z build-std=alloc,core in my case solves it.