RELEASE_PROCESS.md

Release Process

• Release Process
  • Major Release Procedure
    • Changelog
      • Creating a new release branch
      • Cutting a new release
    • Release Notes
    • Tagging Procedure
      • Test building artifacts
      • Installing goreleaser
  • Non-major Release Procedure
  • Major Release Maintenance
  • Stable Release Policy

This document outlines the release process for Cosmos Hub (Gaia).

Gaia follows semantic versioning, but with the following deviations to account for state-machine and API breaking changes:

• State-machine breaking changes will result in an increase of the major version X (X.y.z).
• Emergency releases & API breaking changes (changes in node interactions e.g. queries) will result in an increase of the minor version Y (x.Y.z | x > 0).
• All other changes will result in an increase of the patch version Z (x.y.Z | x > 0).

State compatibility: It is critical for the patch releases to be state-machine compatible with prior releases in the same minor version. For example, v9.2.1 must be compatible with v9.2.0.

Minor releases will be handled on a case-by-case basis, but generally should only arise in case of safety or security issues that require a co-ordinated network upgrade without a governance process.

This is to ensure determinism, i.e. that given the same input, the nodes will always produce the same output. State-incompatibility is allowed for major upgrades because all nodes in the network perform it at the same time. Therefore, after the upgrade, the nodes continue functioning in a deterministic way.

Note: State-machine breaking changes include changes that impact the amount of gas needed to execute a transaction as it results in a different apphash after the code is executed.

Major Release Procedure

A major release is an increment of the first number (eg: v9.1.0 → v10.0.0). Each major release opens a stable release series and receives updates outlined in the Major Release Maintenance section.

Note: Generally, PRs should target either main or a long-lived feature branch (see CONTRIBUTING.md). An exception are PRs open via the Github mergify integration (i.e., backported PRs).

• Once the team feels that main is feature complete, we create a release/vY branch (going forward known a release branch), where Y is the version number, with the minor and patch part substituted to x (eg: 11.x).
  • Update the GitHub mergify integration by adding instructions for automatically backporting commits from main to the release/vY using the A:backport/vY label.
  • PRs targeting directly a release branch can be merged only when exceptional circumstances arise.
• In the release branch
  • Create a new version section in the CHANGELOG.md (follow the procedure described below)
  • Create release notes, in RELEASE_NOTES.md, highlighting the new features and changes in the version. This is needed so the bot knows which entries to add to the release page on GitHub.
  • (To be added in the future) Additionally verify that the UPGRADING.md file is up to date and contains all the necessary information for upgrading to the new version.
• We freeze the release branch from receiving any new features and focus on releasing a release candidate.
  • Finish audits and reviews.
  • Add more tests.
  • Fix bugs as they are discovered.
• After the team feels that the release branch works fine (i.e., has ~90% chance of reaching mainnet), we cut a release candidate.
  • Create a new annotated git tag for a release candidate in the release branch (follow the Tagging Procedure).
  • The release verification on public testnets must pass.
  • When bugs are found, create a PR for main, and backport fixes to the release branch.
  • Create new release candidate tags after bugs are fixed.
• After the team feels the release candidate is mainnet ready, create a full release:
  • Note: The final release MUST have the same commit hash as the latest corresponding release candidate.
  • Create a new annotated git tag in the release branch (follow the Tagging Procedure). This will trigger the automated release process (which will also create the release artifacts).
  • Once the release process completes, modify release notes if needed.

Changelog

For PRs that are changing production code, please add a changelog entry in .changelog (for details, see contributing guidelines).

To manage and generate the changelog on Gaia, we currently use unclog.

Creating a new release branch

Unreleased changes are collected on main in .changelog/unreleased/. However, .changelog/ on main contains also existing releases (e.g., v10.0.0). Thus, when creating a new release branch (e.g., release/v11.x), the following steps are necessary:

• move to the release branch, e.g., release/v11.x
• delete all the sub-folders in .changelog/ except unreleased/
• replace the content of .changelog/epilogue.md with the following text

## Previous Versions

[CHANGELOG of previous versions](https://github.com/cosmos/gaia/blob/main/CHANGELOG.md)

Cutting a new release

Before cutting a release candidate (e.g., v11.0.0-rc0), the following steps are necessary:

• move to the release branch, e.g., release/v11.x
• move all entries in ".changelog/unreleased" to the release version, e.g., v11.0.0, i.e.,

unclog release v11.0.0

• update the CHANGELOG.md, i.e.,

unclog build > CHANGELOG.md

❗Once the final release is cut, the new changelog section must be added to main:

• checkout a new branch from the main branch, i.e.,
• bring the new changelog section from the release branch into this branch, e.g.,

git checkout release/v11.x .changelog/v11.0.0

• remove duplicate entries that are both in .changelog/unreleased/ and the new changelog section, e.g., .changelog/v11.0.0
• update the CHANGELOG.md

unclog build > CHANGELOG.md

• open a PR (from this new created branch) against main

Release Notes

Release notes will be created using the RELEASE_NOTES.md from the release branch. Once the automated releases process is completed, please add any missing information the the release notes using Github UI.

With every release, the goreleaser tool will create a file with all the build artifact checksums and upload it alongside the artifacts. The file is called SHA256SUMS-{{.version}}.txt and contains the following:

098b00ed78ca01456c388d7f1f22d09a93927d7a234429681071b45d94730a05  gaiad_0.0.4_windows_arm64.exe
15b2b9146d99426a64c19d219234cd0fa725589c7dc84e9d4dc4d531ccc58bec  gaiad_0.0.4_darwin_amd64
604912ee7800055b0a1ac36ed31021d2161d7404cea8db8776287eb512cd67a9  gaiad_0.0.4_darwin_arm64
76e5ff7751d66807ee85bc5301484d0f0bcc5c90582d4ba1692acefc189392be  gaiad_0.0.4_linux_arm64
bcbca82da2cb2387ad6d24c1f6401b229a9b4752156573327250d37e5cc9bb1c  gaiad_0.0.4_windows_amd64.exe
f39552cbfcfb2b06f1bd66fd324af54ac9ee06625cfa652b71eba1869efe8670  gaiad_0.0.4_linux_amd64

Tagging Procedure

Important: Always create tags from your local machine since all release tags should be signed and annotated. Using Github UI will create a lightweight tag, so it's possible that gaiad version returns a commit hash, instead of a tag. This is important because most operators build from source, and having incorrect information when you run make install && gaiad version raises confusion.

The following steps are the default for tagging a specific branch commit using git on your local machine. Usually, release branches are labeled release/v*:

Ensure you have checked out the commit you wish to tag and then do:

git pull --tags

# test tag creation and releasing using goreleaser
make create-release-dry-run TAG=v11.0.0

# after successful test push the tag
make create-release TAG=v11.0.0

To re-create a tag:

# delete a tag locally
git tag -d v11.0.0  

# push the deletion to the remote
git push --delete origin v11.0.0 

# redo create-release
make create-release-dry-run TAG=v11.0.0
make create-release TAG=v11.0.0

Test building artifacts

Before tagging a new version, please test the building releasing artifacts by running:

TM_VERSION=$(go list -m github.com/tendermint/tendermint | sed 's:.* ::') goreleaser release --snapshot --clean --debug

Installing goreleaser

Check the instructions for installing goreleaser locally for your platform

• https://goreleaser.com/install/

Non-major Release Procedure

A minor release_ is an increment of the point number (eg: v9.0.0 → v9.1.0, also called point release). A patch release is an increment of the patch number (eg: v10.0.0 → v10.0.1).

Important: Non-major releases must not break consensus.

Updates to the release branch should come from main by backporting PRs (usually done by automatic cherry pick followed by a PRs to the release branch). The backports must be marked using backport/Y label in PR for main. It is the PR author's responsibility to fix merge conflicts, update changelog entries, and ensure CI passes. If a PR originates from an external contributor, a member of the stewarding team assumes responsibility to perform this process instead of the original author. Lastly, it is the stewarding team's responsibility to ensure that the PR meets all the Stable Release Update (SRU) criteria.

Non-major Release must follow the Stable Release Policy.

After the release branch has all commits required for the next patch release:

• Update the changelog and the release notes.
• Create a new annotated git tag in the release branch (follow the Tagging Procedure). This will trigger the automated release process (which will also create the release artifacts).
• Once the release process completes, modify release notes if needed.

Major Release Maintenance

Major Release series continue to receive bug fixes (released as either a Minor or a Patch Release) until they reach End Of Life. Major Release series is maintained in compliance with the Stable Release Policy as described in this document.

Note: Not every Major Release is denoted as stable releases.

After two major releases, a supported major release will be transitioned to unsupported and will be deemed EOL with no further updates. For example, release/v10.x is deemed EOL once the network upgrades to release/v12.x.

Stable Release Policy

Once a Gaia release has been completed and published, updates for it are released under certain circumstances and must follow the Non-major Release Procedure.

The intention of the Stable Release Policy is to ensure that all major release series that are not EOL, are maintained with the following categories of fixes:

• Tooling improvements (including code formatting, linting, static analysis and updates to testing frameworks)
• Performance enhancements for running archival and synching nodes
• Test and benchmarking suites, ensuring that fixes are sound and there are no performance regressions
• Library updates including point releases for core libraries such as IBC-Go, Cosmos SDK, Tendermint and other dependencies
• General maintenance improvements, that are deemed necessary by the stewarding team, that help align different releases and reduce the workload on the stewarding team
• Security fixes

Issues that are likely excluded, are any issues that impact operating a block producing network.