readme: rewrite for single Docker image

Fix the 2nd step: now instead of preparing a statement we need to build a binary circuit file and an assignment table.
NilFoundation · Nov 14, 2023 · 7ee640f · 7ee640f
1 parent a9143c6
commit 7ee640f
Show file tree

Hide file tree

Showing 3 changed files with 114 additions and 86 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -25,11 +25,11 @@ jobs:
         with:
           submodules: recursive
 
-      - name: Compile a circuit
+      - name: Compile a circuit in IR format
         run: scripts/run.sh --verbose --docker compile
 
-      - name: Make constraint and assignment table for EVM verifier endpoint
-        run: scripts/run.sh --verbose --docker build_constraint
+      - name: Build a binary circuit and assignment table
+        run: scripts/run.sh --verbose --docker run_assigner
 
       - name: Calculate a proof
         run: scripts/run.sh --verbose --docker prove

diff --git a/README.md b/README.md
@@ -166,84 +166,30 @@ zkLLVM can also produce circuits in another LLVM's IR format, `.bc`, but we won'
 </details>
 
 As a result of this step, we get a byte-code file `./build/src/template.ll`.
-This is what we call a circuit itself.
 It's a binary file in the LLVM's intermediate representation format.
 
-## Step 2: Build a circuit statement
+## Step 2: Build a circuit file and an assignment table
 
-The Proof Market works with circuits in the form of circuit statements.
-A statement is basically a JSON containing the circuit and various metadata
-that identifies it.
-
-The `build_statement` command will build a circuit statement from the circuit
-that we compiled earlier:
+Next step is to make a compiled circuit and assignment table.
 
 ```bash
-scripts/run.sh --docker build_statement
+scripts/run.sh --docker run_assigner
 ```
 
-The `build_statement` command does the following:
-
-1. Runs a new container based on `nilfoundation/proof-market-toolchain`.
-2. In the container, runs `prepare_statement.py` to produce a circuit statement.
-
-<details>
-  <summary><code>🧰 [manual mode]</code></summary>
-
-To build a statement, we will use the `prepare_statement.py` script,
-which is a part of the [Proof Market toolchain](https://github.com/nilfoundation/proof-market-toolchain).
-
-First, start a new container with the Proof Market toolchain.
-Remember to exit the `zkllvm` container with `exit` command or start a new console session:
+On this step, we run the `assigner`, giving it the circuit in LLVM IR format (`template.lls`)
+and the input data (`./src/main-input.json`).
+The `assigner` produces the following files:
 
-```bash
-docker run -it --rm \
-    --platform=linux/amd64 \
-    --volume $(pwd):/opt/zkllvm-template \
-    --volume $(pwd)/.config:/.config/ \
-    --volume $(pwd)/.config:/root/.config/ \
-    --volume $(pwd)/.config/.user:/proof-market-toolchain/scripts/.user \
-    --volume $(pwd)/.config/.secret:/proof-market-toolchain/scripts/.secret \
-    --volume $(pwd)/../proof-market-toolchain:/proof-market-toolchain/ \
-    --user $(id -u ${USER}):$(id -g ${USER}) \
-    ghcr.io/nilfoundation/proof-market-toolchain:0.0.33
-```
-
-> The `.config` directory is where you will put the credentials to the Proof Market later on.
-Two extra volume mounts make this directory available in places where
-parts of the Proof Market toolchain might look for it.
-
-Now pack the circuit into a statement:
-
-```bash
-cd /opt/zkllvm-template/
-python3 \
-    /proof-market-toolchain/scripts/prepare_statement.py \
-    --circuit /opt/zkllvm-template/build/src/template.ll \
-    --name template \
-    --type placeholder-zkllvm \
-    --private \
-    --output /opt/zkllvm-template/build/template.json
-
-# -c, --circuit: path to the circuit file
-# -n, --name: statement name
-# -o, --output: path to write the statement file
-# --private: make the statement private, as it's not intended for production usage
-# -t, --type: type of proofs that will be generated with this statement
-# (Placeholder is the name of our proof system, see
-# https://crypto3.nil.foundation/papers/placeholder.pdf)
-```
-</details>
-
-As a result, we have the circuit statement file `./build/template.json`.
-Later we will use it to generate a proof locally.
-We will also push this circuit statement to the Proof Market.
+* Circuit file `./build/template.crct` is the circuit in a binary format that is
+  usable by the `proof-generator`.
+* Assignment table `./build/template.tbl` is a representation of input data,
+  prepared for proof computation with this particular circuit.
 
 ## Step 3: Produce and verify a proof locally
 
 Now we have everything ready to produce our first proof.
 As a circuit developer, we want to first build it locally, to check that our circuit is working.
-We'll use the `proof-generator` CLI, which is a part of the Proof Market toolchain.
+We'll use the `proof-generator` CLI, which is a part of the =nil; toolchain.
 
 ```bash
 scripts/run.sh --docker prove
@@ -252,7 +198,6 @@ scripts/run.sh --docker prove
 <details>
   <summary><code>🧰 [manual mode]</code></summary>
 
-Continue in the `proof-market-toolchain` container that you made in step 2.
 Run the `proof-generator` binary to generate a proof:
 
 ```bash
@@ -270,11 +215,15 @@ proof-generator \
 Note the following lines in the build log:
 
 ```
-generatring zkllvm proof...
+Preprocessing public data...
+Preprocessing private data...
+Generating proof...
+Proof generated
 Proof is verified
+...
 ```
 
-In the first line, `proof-generator` creates a proof, and in the second — verifies it.
+In the first lines, `proof-generator` creates a proof, and in the last one it verifies the proof.
 The resulting proof is in the file `./build/template.proof`.
 
 Congratulations!
@@ -320,6 +269,77 @@ This command will save your username and password in two files in the container:
 These files in the container are mounted to `.config/.user` and `.config/.secret` on your machine.
 This way, when you stop the container, the files will persist until you run it again.
 
+## Step 2: Build a circuit statement
+
+The Proof Market works with circuits in the form of circuit statements.
+A statement is basically a JSON containing the circuit and various metadata
+that identifies it.
+
+The `build_statement` command will build a circuit statement from the circuit
+that we compiled earlier:
+
+```bash
+scripts/run.sh --docker build_statement
+```
+
+The `build_statement` command does the following:
+
+1. Runs a new container based on `nilfoundation/proof-market-toolchain`.
+2. In the container, runs `prepare_statement.py` to produce a circuit statement.
+
+<details>
+  <summary><code>🧰 [manual mode]</code></summary>
+
+To build a statement, we will use the `prepare_statement.py` script,
+which is a part of the [Proof Market toolchain](https://github.com/nilfoundation/proof-market-toolchain).
+
+First, start a new container with the Proof Market toolchain.
+Remember to exit the `zkllvm` container with `exit` command or start a new console session:
+
+```bash
+docker run -it --rm \
+    --platform=linux/amd64 \
+    --volume $(pwd):/opt/zkllvm-template \
+    --volume $(pwd)/.config:/.config/ \
+    --volume $(pwd)/.config:/root/.config/ \
+    --volume $(pwd)/.config/.user:/proof-market-toolchain/scripts/.user \
+    --volume $(pwd)/.config/.secret:/proof-market-toolchain/scripts/.secret \
+    --volume $(pwd)/../proof-market-toolchain:/proof-market-toolchain/ \
+    --user $(id -u ${USER}):$(id -g ${USER}) \
+    ghcr.io/nilfoundation/proof-market-toolchain:0.0.33
+```
+
+> The `.config` directory is where you will put the credentials to the Proof Market later on.
+Two extra volume mounts make this directory available in places where
+parts of the Proof Market toolchain might look for it.
+
+Now pack the circuit into a statement:
+
+```bash
+cd /opt/zkllvm-template/
+python3 \
+    /proof-market-toolchain/scripts/prepare_statement.py \
+    --circuit /opt/zkllvm-template/build/src/template.ll \
+    --name template \
+    --type placeholder-zkllvm \
+    --private \
+    --output /opt/zkllvm-template/build/template.json
+
+# -c, --circuit: path to the circuit file
+# -n, --name: statement name
+# -o, --output: path to write the statement file
+# --private: make the statement private, as it's not intended for production usage
+# -t, --type: type of proofs that will be generated with this statement
+# (Placeholder is the name of our proof system, see
+# https://crypto3.nil.foundation/papers/placeholder.pdf)
+```
+</details>
+
+As a result, we have the circuit statement file `./build/template.json`.
+Later we will use it to generate a proof locally.
+We will also push this circuit statement to the Proof Market.
+
+
 ## Step 5: Publish the circuit statement
 
 Remember the statement that we've packed in step 2?

diff --git a/scripts/run.sh b/scripts/run.sh
@@ -13,10 +13,11 @@ set -euo pipefail
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 REPO_ROOT="$SCRIPT_DIR/.."
 
-# Set image versions in the environment before running this script:
-# export TOOLCHAIN_VERSION=0.1.7
-
-# If unset, default values will be used:
+# You can set image version as an environment variable before running this script:
+# export TOOLCHAIN_VERSION=<custom version here>
+# See all available versions at
+# https://github.com/orgs/NilFoundation/packages/container/package/toolchain
+# If unset, default value will be used:
 echo "using nilfoundation/toolchain:${TOOLCHAIN_VERSION:=0.1.7}"
 
 # podman is a safer option for using on CI machines
@@ -42,17 +43,20 @@ check_file_exists() {
     fi
 }
 
-run_zkllvm() {
+run_toolchain() {
     cd $REPO_ROOT
     # silently stop the existing container if it's running already
-    $DOCKER rm zkllvm 2>/dev/null || true
+    $DOCKER rm nil-tolchain 2>/dev/null || true
     $DOCKER run -it --rm \
-        --name zkllvm \
+        --name nil-tolchain \
         --volume $(pwd):/opt/zkllvm-template \
         --user $(id -u ${USER}):$(id -g ${USER}) \
         ghcr.io/nilfoundation/toolchain:${TOOLCHAIN_VERSION}
 }
 
+
+# Deprecated.
+# Python scripts will be moved to the main image, nilfoundation/toolchain
 run_proof_market_toolchain() {
     cd $REPO_ROOT
     # create files for storing credentials, so that they would persist
@@ -95,10 +99,9 @@ compile() {
     fi
 }
 
-# Use assigner to produce a constraint file and an assignment table.
-# This is not a part of the basic development workflow,
-# but can be used for debugging circuits.
-build_constraint() {
+# Run assigner to produce a circuit file and an assignment table.
+# The proof-generator CLI uses these files to compute a proof.
+run_assigner() {
     if [ "$USE_DOCKER" = true ] ; then
         cd "$REPO_ROOT"
         $DOCKER run $DOCKER_OPTS \
@@ -107,7 +110,7 @@ build_constraint() {
           --user $(id -u ${USER}):$(id -g ${USER}) \
           --volume $(pwd):/opt/zkllvm-template \
           ghcr.io/nilfoundation/toolchain:${TOOLCHAIN_VERSION} \
-          sh -c "bash ./scripts/run.sh build_constraint"
+          sh -c "bash ./scripts/run.sh run_assigner"
         cd -
     else
         cd "$REPO_ROOT/build"
@@ -123,6 +126,10 @@ build_constraint() {
     fi
   }
 
+
+# Build circuit parameter / gate argument files.
+# They should be deployed on-chain and provided as inputs to the
+# EVM Placeholder Verifier.
 build_circuit_params() {
     if [ "$USE_DOCKER" = true ] ; then
         cd "$REPO_ROOT"
@@ -146,7 +153,7 @@ build_circuit_params() {
         check_file_exists "$REPO_ROOT/build/template/gate_argument.sol"
         check_file_exists "$REPO_ROOT/build/template/linked_libs_list.json"
         check_file_exists "$REPO_ROOT/build/template/public_input.json"
-        # todo: replace with gen-circuit-paramsg
+
         transpiler \
           -m gen-circuit-params \
           -i ../src/main-input.json \
@@ -255,12 +262,13 @@ while [[ "$#" -gt 0 ]]; do
         -v|--verbose) set -x ;;
         all) SUBCOMMAND=run_all ;;
         compile) SUBCOMMAND=compile ;;
-        build_constraint) SUBCOMMAND=build_constraint ;;
+        run_assigner) SUBCOMMAND=run_assigner ;;
+        build_constraint) SUBCOMMAND=run_assigner ;; # keeping old command name for compatibilty
         build_circuit_params) SUBCOMMAND=build_circuit_params ;;
         build_statement) SUBCOMMAND=build_statement ;;
         prove) SUBCOMMAND=prove ;;
         verify) SUBCOMMAND=verify ;;
-        run_zkllvm) SUBCOMMAND=run_zkllvm ;;
+        run_toolchain) SUBCOMMAND=run_toolchain ;;
         run_proof_market_toolchain) SUBCOMMAND=run_proof_market_toolchain ;;
         *) echo "Unknown parameter passed: $1"; exit 1 ;;
     esac