Discuss the Rust version policy of zebra

[oxarbitrage]

Opening a discussion to discuss Zebra rust version policy, specifically when playing together with ECC dependencies which are rust version specific.

Please add your experience, ideas, etc.

Current Policy

Zebra is tested with the latest stable Rust version. Earlier versions are not supported or tested. Any Zebra release can start depending on new features in the latest stable Rust.

https://github.com/ZcashFoundation/zebra/blob/main/README.md#building-zebra

[teor2345]

Can you give an example of an ECC dependency and its Rust version restrictions?

[oxarbitrage]

Here is an example: zcash/librustzcash@main...oxarbitrage:librustzcash:nu6-test

The QEdit team also bringed up this related to the orchard crate: https://github.com/zcash/orchard/blob/main/Cargo.toml#L12

It is a bit annoying switching rust versions around when working with zebra and ECC crates at the same time, not sure if there is much we can do.

[teor2345]

The QEdit team also bringed up this related to the orchard crate: zcash/orchard@main/Cargo.toml#L12

The rust-version is the minimum supported Rust version, so any later version should work, including the current stable:
https://doc.rust-lang.org/cargo/reference/manifest.html#the-rust-version-field

[teor2345]

Here is an example: zcash/librustzcash@main...oxarbitrage:librustzcash:nu6-test

This selects the Rust version when building in that directory, or in a directory inside that directory:
https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file

So I think it can just be ignored as long as we are building in the Zebra directory.

[str4d]

@oxarbitrage If the problem you are running into is that when developing across multiple repos with different Rust versions in their rust-toolchain.toml files, that you have to keep multiple versions of the Rust toolchain around: I sympathize, I have the same issue (my ~/.rustup/toolchains/ on my dev server is currently 18GiB). If you really want to avoid it, you can set a directory override to just use stable Rust with all crates under that directory (in which you can clone your multiple repositories). A directory override takes precedence over a rust-toolchain.toml file (and a cargo +VERSION foo takes precedence over both).

[str4d]

(Obviously when developing with a newer Rust version, you run the risk of accidentally using features that are incompatible with a given crate's MSRV, but that repo's CI should catch the problem, so it's just a developer time sink tradeoff.)

[oxarbitrage]

https://github.com/zcash/orchard/blob/main/rust-toolchain.toml#L2

[teor2345]

Same as this one, I think it can just be ignored when building in the Zebra directory:
#7979 (reply in thread)

[str4d]

The rust-toolchain.toml file is present so that developers working in a repository are automatically working with the MSRV, so they don't accidentally use features that require later Rust versions. It is not pinning a specific version for downstream users.

[str4d]

As pointed out in comments above, none of the crates maintained by ECC are Rust-version-specific. (An example of a Rust-version-specific crate would be one that requires a specific version of nightly Rust; all of the relevant crates here work on stable Rust.)

The rust-version field in each crate sets the minimum Rust version (MSRV) that the crate will compile with; cargo will immediately fail if an older Rust version is used (which is more usable than failing later in the build process with a more obscure and user-unfriendly error message). The edition field is a coarser version of the same thing (as it sets the minimum Rust version to "one that knows what this edition is").

As for how we choose an MSRV, we generally pick a single MSRV for an entire repository (for development ease). We don't bother bumping MSRV unless we need a feature that a newer Rust version provides, or if a necessary non-dev dependency bumps its MSRV. The latter we are now working around by checking in Cargo.lock to pin known-working dependency versions. When we do raise the MSRV for a repository, we don't generally raise it to more recent than n - 3 unless there's a very good reason to. See zcash/librustzcash#759 for an analysis I did back when we were decided to bump MSRV from 1.56 to 1.60.

Technically to fully support every Rust version between the MSRV and latest stable, we'd need CI to test every possible Rust version in between. That obviously doesn't scale, so what we do instead in our zcash/librustzcash CI (which I'm in the process of porting to the other repos) is:

• Build and test with MSRV and pinned dependencies (via the checked-in Cargo.lock).
• Build with latest stable and latest dependencies.

[dmidem]

Hi everyone,

Thank you for initiating this interesting discussion and sharing your insights. I would like to contribute by presenting another example of the Rust version conflict problem, which we have periodically encountered at QEDIT. The following example illustrates a recent incident. I apologize for the lengthy explanation.

Let's consider the latest version of the original Orchard crate (0.6.0). It has a dependency crate called blake2b_simd, referenced in Orchard's Cargo.toml as follows:

...
[dependencies]
...
blake2b_simd = "1"
...

Attempting to build the latest Orchard version from the GitHub repository (https://github.com/zcash/orchard) using cargo check commands results in an error:

error: package `constant_time_eq v0.3.0` cannot be built because it requires rustc 1.66.0 or newer, while the currently active rustc version is 1.65.0
Either upgrade to rustc 1.66.0 or newer, or use
cargo update -p constant_time_eq@0.3.0 --precise ver
where `ver` is the latest version of `constant_time_eq` supporting rustc 1.65.0

The issue lies in the fact that blake2b_simd version 1.0.2, selected by Cargo, requires constant_time_eq v0.3.0, which, in turn, necessitates Rust version 1.66.0.

This discrepancy is understandable; Orchard is explicitly limited to Rust 1.65.0, but constant_time_eq v0.3.0 cannot be compiled with this version. Notably, the last successful CI build of Orchard occurred at the time when the latest version of blake2b_simd was v1.0.1, referencing constant_time_eq = "0.2.4", which required Rust version 1.59.

Now, attempting to build the latest version of Zebra's zebra-chain (which depends on orchard v0.6.0) encounters no issues. Why? Because zebra-chain does not impose a Rust version limit, and Cargo uses the latest Rust version for both zebra-chain and its dependencies, including orchard. Thus, the Rust version restriction specified in rust-toolchain.toml of orchard is ignored when building orchard as a dependency of zebra-chain.

This leads to two confusions:

Firstly

The problem with orchard, when some of its dependency changed their MSVR, becomes apparent only when building it directly, not through Zebra.

Secondly

In our fork of orchard (named orchard_zsa within Zebra), when encountering an issue with blake2b_simd incompatibility, we resolved it by pinning blake2b_simd to v1.0.1 in Orchard ZSA's Cargo.toml.

However, integrating this solution with Zebra introduced a new challenge. While the zebra-chain itself refers to blake2b_simd v1.0.2 in its Cargo.toml, orchard_zsa explicitly pins it to blake2b_simd v1.0.1. This dual usage creates a conflict, and cargo check fails with an error.

For now, we've addressed this conflict in our Zebra fork by pinning blake2b_simd to 1.0.1 in zebra-chain's Cargo.toml too. While this resolves the issue, similar problems may arise in the future, when deep sub-dependencies change their MSVR.

For me finding a universal solution remains a challenge. One alternative could be committing the Cargo.lock file in the orchard repository. Rust Cargo team didn't recommend to do that for a long time, but recently they changed their guidance, here's an article about that:

https://blog.rust-lang.org/2023/08/29/committing-lockfiles.html

Committing Cargo.lock may make the build of orchard deterministic, but it may cause another kind of issue, like they say in their new version of guidance:

However, this determinism can give a false sense of security because Cargo.lock does not affect the consumers of your package, only Cargo.toml does that.

[str4d]

I suggest you read my comments made prior to yours; I address most of what you've described. I'll comment on the development aspects after the holidays.

[dmidem]

@str4d thanks for noting that. Absolutely, you already described that. I just wanted to provide a concrete example to illustrate that our issue with Rust version conflicts is essentially the same.

[oxarbitrage]

Thank you all for helping each other and sharing experiences. I feel that there is some good content here, we might want to create a document where the differences between the rust version policy of Zebra and ECC crates are explained.

Do we think this could have value for new and current participants ?

I suggest you read my comments made prior to yours; I address most of what you've described. I'll comment on the development aspects after the holidays.

We should wait for this before taking any action, thank you @str4d .