Switch to fully upstream Rust

[lrettig]

It looks like this is now possible, and like it's time to retire https://github.com/athenavm/rustc-rv32e-toolchain

See paritytech/polkavm#178

[jellonek]

When trying rustup run nightly-2024-07-10 cargo build -Zbuild-std --target ../../../riscv32em-unknown-none-athenavm.json in examples/wallet/program with target json configured as:

{
  "arch": "riscv32",
  "cpu": "generic-rv32",
  "crt-objects-fallback": "false",
  "data-layout": "e-m:e-p:32:32-i64:64-n32-S32",
  "eh-frame-header": false,
  "emit-debug-gdb-scripts": false,
  "features": "+e,+m,+lui-addi-fusion,+fast-unaligned-access,+xtheadcondmov",
  "linker": "rust-lld",
  "linker-flavor": "ld.lld",
  "llvm-abiname": "ilp32e",
  "llvm-target": "riscv32",
  "max-atomic-width": 32,
  "panic-strategy": "abort",
  "relocation-model": "static",
  "target-pointer-width": "32",
  "singlethread": true,
  "pre-link-args": {
    "ld": [
      "--emit-relocs",
      "--relocatable",
      "--unique"
    ]
  },
  "env": "athenavm"
}

i'm getting:

error[E0432]: unresolved imports `crate::sys::thread_local::key::get`, `crate::sys::thread_local::key::set`, `crate::sys::thread_local::key::Key`, `crate::sys::thread_local::key::LazyKey`
 --> /home/jell/.local/rust/rustup/toolchains/nightly-2024-07-10-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/thread_local/os.rs:5:37
  |
5 | use crate::sys::thread_local::key::{get, set, Key, LazyKey};
  |                                     ^^^  ^^^  ^^^  ^^^^^^^ no `LazyKey` in `sys::thread_local::key`
  |                                     |    |    |
  |                                     |    |    no `Key` in `sys::thread_local::key`
  |                                     |    no `set` in `sys::thread_local::key`
  |                                     no `get` in `sys::thread_local::key`
  |
note: module `crate::collections::hash::set` exists but is inaccessible
 --> /home/jell/.local/rust/rustup/toolchains/nightly-2024-07-10-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/collections/hash/mod.rs:4:1
  |
4 | pub mod set;
  | ^^^^^^^^^^^^ not accessible

I'm still digging into it.

[jellonek]

Adding "os": "zkvm" allows to pass it further, but at the end i see:

error[E0463]: can't find crate for `panic_abort`

That can be fixed by using -Z build-std=std,panic_abort but then linker fails.

I'm still digging into it.

[jellonek]

Above works fine after setting RUSTFLAGS="-C passes=loweratomic -C link-arg=-Ttext=0x00200800".

[jellonek]

So far:

• setting -C link-arg=-Ttext=0x00200800 in rustflags leads to linker errors like multiple:

          rust-lld: error: section .text file range overlaps with .text
          >>> .text range is [0x800, 0x843]
          >>> .text range is [0x800, 0x89B]

• setting -C passes=loweratomic in rustflags does not work with never compiler suite
• setting -Z build-std-features=compiler-builtins-mem -Z build-std=std,panic_abort solves most issues except missing __atomic_* which we are expecting due setting in target max-atomic-width to non 0 value. 0 value leads to:

error[E0432]: unresolved import `core::sync::atomic::AtomicUsize`
 --> /home/user/.local/rust/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/stdarch/crates/std_detect/src/detect/cache.rs:8:5
  |
8 | use core::sync::atomic::AtomicUsize;
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `AtomicUsize` in `sync::atomic`

Current plan is to add somewhere (e.g. to vm/entrypoint/src/lib.rs) stubs for:

__atomic_fetch_or_1
__atomic_store_1
__atomic_fetch_sub_4
__atomic_fetch_and_1
__atomic_fetch_add_4

like:

#[cfg(target_os = "zkvm")]
#[no_mangle]
pub fn __atomic_load_4(arg: *const usize, _ordering: usize) -> usize {
    unsafe { *arg }
}

#[cfg(target_os = "zkvm")]
#[no_mangle]
pub fn __atomic_load_1(arg: *const usize, _ordering: usize) -> usize {
    unsafe { *arg }
}

All that was tested with riscv32em-unknown-zkvm-athenavm.json

{
  "arch": "riscv32",
  "cpu": "generic-rv32",
  "crt-objects-fallback": "false",
  "data-layout": "e-m:e-p:32:32-i64:64-n32-S32",
  "eh-frame-header": false,
  "emit-debug-gdb-scripts": false,
  "features": "+e,+m",
  "linker": "rust-lld",
  "linker-flavor": "ld.lld",
  "llvm-abiname": "ilp32e",
  "llvm-target": "riscv32",
  "max-atomic-width": 32,
  "atomic-cas": true,
  "executables": true,
  "panic-strategy": "abort",
  "relocation-model": "static",
  "target-pointer-width": "32",
  "singlethread": true,
  "os": "zkvm",
  "pre-link-args": {
    "ld": [
      "--emit-relocs",
      "--unique"
    ]
  },
  "env": "athenavm"
}

used with command: cargo +nightly build -Z build-std=std,panic_abort -Z build-std-features=compiler-builtins-mem --target ../../../riscv32em-unknown-zkvm-athenavm.json

[lrettig]

IIRC -C passes=loweratomic is what previously fixed the atomic issues you're seeing. What do you mean that it

does not work with never compiler suite