Failed to build local-node image due to the loss of utils in Dockerfile

[LBruyne]

🐛 Bug Report

📝 Description

I try to build a local-node image from the Dockerfile provided in docker/local-node folder. However, the build fails due to a loss of dependency utils which is located in etc/utils package. Part of log is below and full log is deferred to the last part of this issue:

 => ERROR [13/23] RUN cd /infrastructure/zk && yarn && yarn build && cd /                                                                                               30.6s
------                                                                                                                                                                        
 > [13/23] RUN cd /infrastructure/zk && yarn && yarn build && cd /:                                
21.54 [4/4] Building fresh packages...
27.21 success Saved lockfile.
27.21 Done in 26.79s.
27.39 yarn run v1.22.22
27.42 $ tsc
28.92 src/clean.ts(4,31): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/compiler.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/config.ts(8,36): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/contract.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/contract_verifier.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/database.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/docker.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/down.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/env.ts(4,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/fmt.ts(3,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/format_sql.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/format_sql.ts(165,56): error TS7006: Parameter 'file' implicitly has an 'any' type.
28.92 src/hyperchain_wizard.ts(16,27): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/init.ts(3,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/init.ts(4,27): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/lint.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/prover_setup.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/reinit.ts(4,27): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/run.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/server.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/setup_en.ts(9,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/test/integration.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/test/test.ts(3,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/up.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.

🔄 Reproduction Steps

Run cmd sudo ZKSYNC_HOME=pwd zk docker build local-node --custom-tag "my_custom_local_tag".

Then the image will be built and this error happens.

📋 Additional Context

I try to fix this by adding utils to Dockerfile as well. But I find it hard to manage it smoothly.

📎 Log Output

➜  zksync-era git:(main) ✗ sudo ZKSYNC_HOME=`pwd` zk docker build local-node --custom-tag "my_custom_local_tag"
[+] Building 36.3s (17/26)                                                                                                                                     docker:default
 => [internal] load build definition from Dockerfile                                                                                                                     0.0s
 => => transferring dockerfile: 3.02kB                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/matterlabs/server-v2:latest2.0                                                                                                0.2s
 => [internal] load .dockerignore                                                                                                                                        0.0s
 => => transferring context: 1.18kB                                                                                                                                      0.0s
 => [internal] load build context                                                                                                                                        0.8s
 => => transferring context: 2.48MB                                                                                                                                      0.8s
 => [ 1/23] FROM docker.io/matterlabs/server-v2:latest2.0@sha256:e2d79fa5f331d5855f514e561d77f41a1da748ba835e87697882d013895a806a                                        0.0s
 => CACHED [ 2/23] RUN apt-get update; apt-get install -y make bash git openssl libssl-dev gcc g++ curl pkg-config software-properties-common jq wget vim-tiny           0.0s
 => CACHED [ 3/23] RUN apt-get install -y curl gnupg libpq5 ca-certificates postgresql-client && rm -rf /var/lib/apt/lists/*                                             0.0s
 => CACHED [ 4/23] RUN mkdir -p /etc/apt/keyrings &&     wget -c -O - https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/no  0.0s
 => CACHED [ 5/23] RUN curl https://sh.rustup.rs -sSf | bash -s -- -y &&     rustup install stable                                                                       0.0s
 => CACHED [ 6/23] RUN cargo install sqlx-cli --version 0.7.3                                                                                                            0.0s
 => CACHED [ 7/23] COPY infrastructure/zk /infrastructure/zk                                                                                                             0.0s
 => CACHED [ 8/23] COPY infrastructure/local-setup-preparation /infrastructure/local-setup-preparation                                                                   0.0s
 => CACHED [ 9/23] COPY etc/env /etc/env                                                                                                                                 0.0s
 => [10/23] COPY etc/test_config /etc/test_config                                                                                                                        0.1s
 => [11/23] COPY contracts /contracts/                                                                                                                                   2.9s
 => [12/23] RUN yarn global add typescript                                                                                                                               1.6s
 => ERROR [13/23] RUN cd /infrastructure/zk && yarn && yarn build && cd /                                                                                               30.6s
------                                                                                                                                                                        
 > [13/23] RUN cd /infrastructure/zk && yarn && yarn build && cd /:                                                                                                           
0.422 yarn install v1.22.22                                                                                                                                                   
0.441 info No lockfile found.                                                                                                                                                 
0.446 [1/4] Resolving packages...                                                                                                                                             
4.326 warning hardhat > glob@7.2.0: Glob versions prior to v9 are no longer supported                                                                                         
4.347 warning hardhat > mocha > glob@8.1.0: Glob versions prior to v9 are no longer supported
4.473 warning hardhat > glob > inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
4.473 warning hardhat > mocha > glob > inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
4.632 warning hardhat > solc > fs-extra > rimraf@2.7.1: Rimraf versions prior to v4 are no longer supported
4.632 warning hardhat > solc > fs-extra > rimraf > glob@7.2.3: Glob versions prior to v9 are no longer supported
4.634 warning hardhat > solc > fs-extra > rimraf > glob > inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
5.529 [2/4] Fetching packages...
11.79 [3/4] Linking dependencies...
21.54 [4/4] Building fresh packages...
27.21 success Saved lockfile.
27.21 Done in 26.79s.
27.39 yarn run v1.22.22
27.42 $ tsc
28.92 src/clean.ts(4,31): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/compiler.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/config.ts(8,36): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/contract.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/contract_verifier.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/database.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/docker.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/down.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/env.ts(4,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/fmt.ts(3,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/format_sql.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/format_sql.ts(165,56): error TS7006: Parameter 'file' implicitly has an 'any' type.
28.92 src/hyperchain_wizard.ts(16,27): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/init.ts(3,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/init.ts(4,27): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/lint.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/prover_setup.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/reinit.ts(4,27): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/run.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/server.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/setup_en.ts(9,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/test/integration.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/test/test.ts(3,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.92 src/up.ts(2,24): error TS2307: Cannot find module 'utils' or its corresponding type declarations.
28.95 error Command failed with exit code 1.
28.95 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
------
Dockerfile:55
--------------------
  53 |     # RUN cd /etc/utils && yarn add --dev @types/node && yarn && yarn build && cd /
  54 |     # Build `zk` tool
  55 | >>> RUN cd /infrastructure/zk && yarn && yarn build && cd /
  56 |     # Build `local-setup-preparation` tool
  57 |     RUN cd /infrastructure/local-setup-preparation && yarn && cd /
--------------------
ERROR: failed to solve: process "/bin/sh -c cd /infrastructure/zk && yarn && yarn build && cd /" did not complete successfully: exit code: 1
Error: Child process exited with code 1

[LBruyne]

By the way, I find it hard to build the zk tool out of Docker as well. The doc here https://github.com/matter-labs/zksync-era/blob/main/docs/guides/development.md does not mention the util package in total, but it is a essential dependency of building zk.

When running zk init, I suggest you replace this command with

sudo ZKSYNC_HOME=`pwd` PATH=$ZKSYNC_HOME/bin:$PATH zk init 

Because sudo cannot identify ZKSYNC_HOME even though you have set this in your .bash_rc.

[LBruyne]

Another question is, how to monitor the local-node in a blockchain browser (like etherscan)?

[LBruyne]

I will be grateful if anyones see this and make a reply.

[LBruyne]

I resolve this by linking utils module into the docker env.

# Build `utils` tool and link
RUN cd /etc/utils && yarn && yarn link && cd /
# Build `zk` tool
RUN cd /infrastructure/zk && yarn link utils && yarn && yarn build && cd /
# RUN cd /infrastructure/zk && yarn && yarn build && cd /

Feel strange why this is not written in the original code.
BTW, it remains unclear for me how to monitor the local-node in a blockchain explorer?