Skip to content

Latest commit

 

History

History
75 lines (48 loc) · 4.94 KB

TOOLS.md

File metadata and controls

75 lines (48 loc) · 4.94 KB

Tools and methods allowing to verify that Java builds are reproducible

As part of Reproducible Builds efforts for the JVM, this "Reproducible Central" project attempts to:

  1. Write .buildspecrebuild instructions for the artifacts available in the Central Repository, equivalent to the packaging instructions that are maintained by every Linux distribution (for example Debian's debian/rules or ArchLinux's PKGBUILD), whatever the build tool used (Central Repository is not used by Maven only)
  2. Evaluate the level of reproducibility obtained using previous instructions. How many output files from the rebuild are strictly equal to the reference in Central Repository? How many output files are not yet reproducible and should be improved before the next release?

Reproducible journey

If you want to start playing with the reproducible builds, we recommend the following steps.

1) Rebuild Yourself To Check Results

Prerequisites:

  • Docker or Podman
  • dos2unix - DOS to MAC/UNIX text file format converter.
    Can be installed via homebrew on MAC via: brew install dos2unix.

You can rebuild a project release by running:

./rebuild.sh content/<path/to/...>/<project>-<version>.buildspec

rebuild.sh script will use the build specification file (= .buildspec file) to choose a Docker image to rebuild the project and check output against Central Repository reference binaries.

For example:

./rebuild.sh content/org/apache/maven/shared/archiver/maven-archiver-3.5.1.buildspec

You can also use podman as a container engine by defining these environment variables before you run rebuild.sh:

# The engine to use. Defaults to 'docker', but also tested with 'podman'.
RB_OCI_ENGINE

# Extra build-options to provide to the container engine. Defaults for docker: "", podman: "--format docker"
RB_OCI_ENGINE_BUILD_OPTS

# Extra run-options to provide to the container engine. Defaults for docker: "", podman: "--userns=keep-id"
RB_OCI_ENGINE_RUN_OPTS

# Extra flags to use when mounting volumes in the container. Defaults to "", but for podman running on an SELinux host, you need ":Z,rw"
RB_OCI_VOLUME_FLAGS

2) Contribute to a new buildspec

If you know of a project released to Central Repository that has enabled Reproducible Builds (see Configuring Build Tools for Reproducible Builds), please tell us by opening an issue with details.

You can also choose one from our list of projects waiting for a .buildspec: follow our instructions to write a new .buildspec) that you can contribute back with a PR.

3) Improve Reproducibility Score Of A Project Release

If a rebuild published here is not fully reproducible (it still has some ⚠️ after having applied Configuring Build Tools for Reproducible Builds), there is an issue: please help to improve the situation.

You'll need to rebuild the release yourself (see previous instructions), then use diffoscope 🔍 to easily explore the precise difference between the reference file from Central Repository and the effective rebuild file, then debug up to the root cause of this unwanted difference:

  • rebuilder bug: if the improvement has to happen at buildspec or rebuild script level, don't hesitate to open an issue or a PR here,
  • upstream project reproducibility issue 📝: please contact the upstream project and help them improve the reproducibility for their next release, creating an issue in their issue tracker and adding it to Reproducible Central buildspec as issue parameter that will link to it with a 📝.

4) Add Reproducible Builds Badge to a Project

Once a project is listed here, a badge endpoint is provided to display proven reproducibility success level: it provides a badge like Reproducible Builds pointing to its README.md:

Example for Markdown:

[![Reproducible Builds](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fjvm-repo-rebuild%2Freproducible-central%2Fmaster%2F...path to project directory...%2Fbadge.json)](https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/...path to project directory.../README.md)

This uses the Endpoint Badge from shields.io for rendering the image.