diff --git a/README.md b/README.md index a1692c444..29a9705d7 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ supported. Zombienet releases are available in `github`. Each one provides an executable for both `linux` and `macos` created with [pkg](https://github.com/vercel/pkg) and allows to run `zombienet` cli -*without* having `Node.js` installed **but** each `provider` defines it's own requirements (e.g. +*without* having `Node.js` installed **but** each `provider` defines its own requirements (e.g. `k8s`, `podman`). **Note:** Currently, it is only possible to use `podman` for Zombienet users on Linux machines. @@ -60,7 +60,7 @@ Zombienet project has it's own `k8s` cluster in GCP, to use it please ping ### With Podman -Zombienet support [Podman](https://podman.io/) *rootless* as provider, you only need to have +Zombienet supports [Podman](https://podman.io/) *rootless* as provider, you only need to have `podman` installed in your environment to use and either set in the *network* file or with the `--provider` flag in the cli. `Podman` for `zombienet` is currently only supported for Linux machines. This is mostly related to paths and directories used by @@ -72,7 +72,7 @@ Zombienet `native` provider allows you to run the nodes as a local process in yo only need to have the `binaries` used in your `network` (e.g `polkadot` or `polkadot-parachain`). To use it either set in the *network* file or with the `--provider` flag in the cli. -**NOTE:** The `native` provider **only** use the `command` config for nodes/collators, both relative +**NOTE:** The `native` provider **only** uses the `command` config for nodes/collators, both relative and absolute paths are supported. You can use `default_command` config to set the binary to spawn all the `nodes` in the relay chain. @@ -425,6 +425,6 @@ Below can be found some of the projects that are currently using Zombienet as in ## Acknowledgement -This project take inspiration and some patterns from +This project takes inspiration and some patterns from [polkadot-launch](https://github.com/paritytech/polkadot-launch) and [simnet](https://gitlab.parity.io/parity/simnet/-/tree/master). diff --git a/docs/src/acknowledgement.md b/docs/src/acknowledgement.md index 69d6db5ef..5f4923e0d 100644 --- a/docs/src/acknowledgement.md +++ b/docs/src/acknowledgement.md @@ -1,4 +1,4 @@ # Acknowledgement -This project take inspiration and some patterns from [polkadot-launch](https://github.com/paritytech/polkadot-launch) and [simnet](https://gitlab.parity.io/parity/simnet/-/tree/master). +This project takes inspiration and some patterns from [polkadot-launch](https://github.com/paritytech/polkadot-launch) and [simnet](https://gitlab.parity.io/parity/simnet/-/tree/master). diff --git a/docs/src/guide.md b/docs/src/guide.md index 66e9b83fc..0056ec83d 100644 --- a/docs/src/guide.md +++ b/docs/src/guide.md @@ -12,7 +12,7 @@ In this guide we will go through simple use cases as well as more complex ones, --- -In this example, is shown the alternateive options of a configuration file. The different extensions supported are: `yaml`, `json` and of course `.toml`. +In this example, is shown the alternative options of a configuration file. The different extensions supported are: `yaml`, `json` and of course `.toml`. This simple config exists only in order to show how the same configuration exists in 3 different formats, and thus uses only 1 relay chain (`rococo-local`) with 2 nodes (`alice` and `bob`); The configuration files are: @@ -21,7 +21,7 @@ The configuration files are: - [0000-test-config-small-network.yaml](https://github.com/paritytech/zombienet/blob/main/examples/0000-test-config-small-network.yaml); - [0000-test-config-small-network.toml](https://github.com/paritytech/zombienet/blob/main/examples/0000-test-config-small-network.toml); -and the test files for running each confgiration are: +and the test files for running each configuration are: - [0000-test-json-config-small-network.zndsl](https://github.com/paritytech/zombienet/blob/main/examples/0000-test-json-config-small-network.zndsl); - [0000-test-yaml-config-small-network.zndsl](https://github.com/paritytech/zombienet/blob/main/examples/0000-test-yaml-config-small-network.zndsl); @@ -126,7 +126,7 @@ alice: reports histogram polkadot_pvf_execution_time has at least 2 samples in b # logs bob: log line matches glob "*rted #1*" within 10 seconds bob: log line matches "Imported #[0-9]+" within 10 seconds -bob: count of log lines maching "Error" is 0 within 10 seconds +bob: count of log lines matching "Error" is 0 within 10 seconds # system events bob: system event contains "A candidate was included" within 20 seconds @@ -152,7 +152,7 @@ Now, run the tests with the following command to get both the _launching_ output --- In Example 1, we used some _hardcoded_ default values but sometimes it's more useful to be able to change them dynamically. -For example, if you are building images in your `CI` and those images have an unique tag. To address these cases, Zombienet uses a templating language called [Nunjucks](https://mozilla.github.io/nunjucks/) that allows defining environment variables which then can get updated at _runtime_. +For example, if you are building images in your `CI` and those images have a unique tag. To address these cases, Zombienet uses a templating language called [Nunjucks](https://mozilla.github.io/nunjucks/) that allows defining environment variables which then can get updated at _runtime_. Following the previous example, we will replace the _images_ with variables that will read the value from the environment. An interesting addon test that needs to be noticed here, is that the bob validator, is using a different version of polkadot (v0.9.37) instead of the latest (which at the time of writing this documentation was v0.9.39): @@ -206,7 +206,7 @@ And again we get the network info with direct links: This example shows that we can use custom environmental variables in our config file in order to dynamically alter the configuration of the network. -In addition that the validators support the capability of running different versions of the image. +In addition the validators support the capability of running different versions of the image. > Note: In general all the config fields that start with `default_*` can be overridden in the `nodes` or `collators` config. @@ -316,7 +316,7 @@ When you use `podman` you can follow the logs of the pods with the command sugge #### Logs in native -With the `native` provider you can follow the logs of the pods with the command suggestion that `zombienet` gives you when it spawn a new process. +With the `native` provider you can follow the logs of the pods with the command suggestion that `zombienet` gives you when it spawns a new process. For example: ``` diff --git a/docs/src/install.md b/docs/src/install.md index 4eae72529..4d781f465 100644 --- a/docs/src/install.md +++ b/docs/src/install.md @@ -20,7 +20,7 @@ Or a particular release can be specified. e.g. to run `v1.3.40`, use: nix run github:paritytech/zombienet/v1.3.40 -- spawn config.toml ``` -Another option is to add he `zombienet` binary to the `PATH` for the current shell. This can be done with: +Another option is to add the `zombienet` binary to the `PATH` for the current shell. This can be done with: ``` nix shell github:paritytech/zombienet/v1.3.40 diff --git a/docs/src/intro.md b/docs/src/intro.md index 3e51a0be9..f1cef6ff0 100644 --- a/docs/src/intro.md +++ b/docs/src/intro.md @@ -1,5 +1,5 @@ # What is ZombieNet? -ZombieNet aims to be a testing framework for Substrate based blockchains, providing a simple CLI tool that allowS users to spawn and test ephemeral networks. The assertions used in the tests can include on-chain storage, metrics, logs and custom JS scripts that interact with the chain. To make these easy to define, Zombienet uses a built-in natural language tool to write tests as smoothly as possible. +ZombieNet aims to be a testing framework for Substrate based blockchains, providing a simple CLI tool that allows users to spawn and test ephemeral networks. The assertions used in the tests can include on-chain storage, metrics, logs and custom JS scripts that interact with the chain. To make these easy to define, Zombienet uses a built-in natural language tool to write tests as smoothly as possible. -Internally, it's a JS library designed to run on NodeJS and support different backend providers to run the various blockchain nodes. Currently, Kubernetes, Podman and native are the supported providers. \ No newline at end of file +Internally, it's a JS library designed to run on NodeJS and support different backend providers to run the various blockchain nodes. Currently, Kubernetes, Podman and native are the supported providers. diff --git a/docs/src/network-definition-spec.md b/docs/src/network-definition-spec.md index 98a1592e7..d3ceee033 100644 --- a/docs/src/network-definition-spec.md +++ b/docs/src/network-definition-spec.md @@ -28,7 +28,7 @@ The network config can be provided both in `json` or `toml` format and each sect - `chain_spec_path`: (String) Path to the chain spec file, **NOTE** should be the `plain` version to allow customizations. - `chain_spec_command`: (String) Command to generate the chain spec, **NOTE** can't be used in combination with `chain_spec_path`. - `default_args`: (Array of strings) An array of arguments to use as default to pass to the `command`. -- `default_substrate_cli_args_version`: (0|1) Allow to set the substrate cli args version (see: https://github.com/paritytech/substrate/pull/13384). By default zombienet will evaluate your binary and set the correct version, but that produce an small overhead that could be skipped if you set directly with this key. +- `default_substrate_cli_args_version`: (0|1) Allow to set the substrate cli args version (see: https://github.com/paritytech/substrate/pull/13384). By default zombienet will evaluate your binary and set the correct version, but that produces a small overhead that could be skipped if you set directly with this key. - `default_overrides`: (Array of objects) An array of overrides to upload to the nodes, objects with: - `local_path`: string; - `remote_name`: string; @@ -42,7 +42,7 @@ The network config can be provided both in `json` or `toml` format and each sect - `command`: (String) Override default command. - `command_with_args`: (String) Override default command and args. - `args`: (Array of strings) Arguments to be passed to the `command`. - - `substrate_cli_args_version`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produce an small overhead that could be skipped if you set directly with this key. + - `substrate_cli_args_version`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produces a small overhead that could be skipped if you set directly with this key. - `validator`: (Boolean, default true) Pass the `--validator` flag to the `command`. - `invulnerable`: (Boolean, default false) If true, the node will be added to `invulnerables` in the chain spec. - `balance`: (number, default 2000000000000) Balance to set in `balances` for node's account. @@ -70,7 +70,7 @@ The network config can be provided both in `json` or `toml` format and each sect - `overrides`: Array of `overrides` definitions. - `prometheus_prefix`: A parameter for customizing the metric's prefix for the specific node. Will apply to all the nodes of the group; Defaults to 'substrate'. - `resources`: (Object) **Only** available in `kubernetes`, represent the resources `limits`/`reservations` needed by the node. - - `substrate_cli_args_version`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produce an small overhead that could be skipped if you set directly with this key. + - `substrate_cli_args_version`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produces a small overhead that could be skipped if you set directly with this key. ## `parachains` @@ -90,7 +90,7 @@ The network config can be provided both in `json` or `toml` format and each sect - `image`: (String) Image to use. - `command`: (String, default `polkadot-parachain`) Command to run. - `args`: (Array of strings) An array of arguments to use as default to pass to the `command`. - - `packages/orchestrator/src/providers/k8s/index.ts`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produce an small overhead that could be skipped if you set directly with this key. + - `packages/orchestrator/src/providers/k8s/index.ts`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produces a small overhead that could be skipped if you set directly with this key. - `command_with_args`: (String) Overrides `command` and `args`. - `env`: Array of env vars Object to set in the container. - name: (String) name of the `env` var. @@ -108,7 +108,7 @@ The network config can be provided both in `json` or `toml` format and each sect - `env`: Array of env vars Object to set in the container. - name: (String) name of the `env` var. - value: (String| number) Value of the env var. - - `substrate_cli_args_version`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produce an small overhead that could be skipped if you set directly with this key. + - `substrate_cli_args_version`: (0|1) By default zombienet will evaluate your binary and set the correct version, but that produces a small overhead that could be skipped if you set directly with this key. - `onboard_as_parachain`: (Boolean, default true) flag to specify whether the para should be onboarded as a parachain or stay a parathread - `register_para`: (Boolean, default true) flag to specify whether the para should be registered. The `add_to_genesis` flag **must** be set to false for this flag to have any effect. diff --git a/javascript/package-lock.json b/javascript/package-lock.json index c2344cc5d..ce7c45a5e 100644 --- a/javascript/package-lock.json +++ b/javascript/package-lock.json @@ -25,7 +25,7 @@ "zombienet": "packages/cli/dist/cli.js" }, "devDependencies": { - "@types/node": "^20.9.0", + "@types/node": "^20.9.1", "@types/tmp": "^0.2.6", "@typescript-eslint/eslint-plugin": "^6.11.0", "@typescript-eslint/parser": "^6.11.0", @@ -1144,9 +1144,9 @@ "dev": true }, "node_modules/@types/node": { - "version": "20.9.0", - "resolved": "https://registry.npmjs.org/@types/node/-/node-20.9.0.tgz", - "integrity": "sha512-nekiGu2NDb1BcVofVcEKMIwzlx4NjHlcjhoxxKBNLtz15Y1z7MYf549DFvkHSId02Ax6kGwWntIBPC3l/JZcmw==", + "version": "20.9.1", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.9.1.tgz", + "integrity": "sha512-HhmzZh5LSJNS5O8jQKpJ/3ZcrrlG6L70hpGqMIAoM9YVD0YBRNWYsfwcXq8VnSjlNpCpgLzMXdiPo+dxcvSmiA==", "dependencies": { "undici-types": "~5.26.4" } @@ -6508,11 +6508,11 @@ }, "packages/cli": { "name": "@zombienet/cli", - "version": "1.3.79", + "version": "1.3.80", "license": "GPL-3.0-or-later", "dependencies": { "@zombienet/dsl-parser-wrapper": "^0.1.10", - "@zombienet/orchestrator": "^0.0.62", + "@zombienet/orchestrator": "^0.0.63", "@zombienet/utils": "^0.0.24", "cli-progress": "^3.12.0", "commander": "^11.1.0", @@ -6534,7 +6534,7 @@ }, "packages/orchestrator": { "name": "@zombienet/orchestrator", - "version": "0.0.62", + "version": "0.0.63", "license": "GPL-3.0-or-later", "dependencies": { "@polkadot/api": "^10.10.1", diff --git a/javascript/package.json b/javascript/package.json index d96f67b30..30cb305d5 100644 --- a/javascript/package.json +++ b/javascript/package.json @@ -39,7 +39,7 @@ "chai": "^4.3.10" }, "devDependencies": { - "@types/node": "^20.9.0", + "@types/node": "^20.9.1", "@types/tmp": "^0.2.6", "@typescript-eslint/eslint-plugin": "^6.11.0", "@typescript-eslint/parser": "^6.11.0", diff --git a/javascript/packages/cli/package.json b/javascript/packages/cli/package.json index b28babfa5..be22b7cd5 100644 --- a/javascript/packages/cli/package.json +++ b/javascript/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "@zombienet/cli", - "version": "1.3.79", + "version": "1.3.80", "description": "ZombieNet aim to be a testing framework for substrate based blockchains, providing a simple cli tool that allow users to spawn and test ephemeral Substrate based networks", "main": "dist/index.js", "scripts": { @@ -54,7 +54,7 @@ }, "dependencies": { "@zombienet/dsl-parser-wrapper": "^0.1.10", - "@zombienet/orchestrator": "^0.0.62", + "@zombienet/orchestrator": "^0.0.63", "@zombienet/utils": "^0.0.24", "cli-progress": "^3.12.0", "commander": "^11.1.0", diff --git a/javascript/packages/orchestrator/package.json b/javascript/packages/orchestrator/package.json index d11f66fdd..f4a9f8d5a 100644 --- a/javascript/packages/orchestrator/package.json +++ b/javascript/packages/orchestrator/package.json @@ -1,6 +1,6 @@ { "name": "@zombienet/orchestrator", - "version": "0.0.62", + "version": "0.0.63", "description": "ZombieNet aim to be a testing framework for substrate based blockchains, providing a simple cli tool that allow users to spawn and test ephemeral Substrate based networks", "main": "dist/index.js", "types": "dist/index.d.ts", diff --git a/javascript/packages/orchestrator/src/providers/k8s/kubeClient.ts b/javascript/packages/orchestrator/src/providers/k8s/kubeClient.ts index c34691ef0..dc840b9f1 100644 --- a/javascript/packages/orchestrator/src/providers/k8s/kubeClient.ts +++ b/javascript/packages/orchestrator/src/providers/k8s/kubeClient.ts @@ -543,10 +543,7 @@ export class KubeClient extends Client { }, { type: "deployment", - files: [ - settings.backchannel ? "backchannel-pod.yaml" : null, - "fileserver-pod.yaml", - ], + files: [settings.backchannel ? "backchannel-pod.yaml" : null], }, ]; @@ -557,6 +554,10 @@ export class KubeClient extends Client { } // wait until fileserver is ready, fix race condition #700. + const xinfra = process.env.X_INFRA_INSTANCE || "ondemand"; + await this.createStaticResource("fileserver-pod.yaml", this.namespace, { + xinfra, + }); await this.waitPodReady("fileserver"); sleep(3 * 1000); let fileServerOk = false; diff --git a/javascript/packages/orchestrator/static-configs/fileserver-pod.yaml b/javascript/packages/orchestrator/static-configs/fileserver-pod.yaml index e39b23384..b90695c6f 100644 --- a/javascript/packages/orchestrator/static-configs/fileserver-pod.yaml +++ b/javascript/packages/orchestrator/static-configs/fileserver-pod.yaml @@ -4,7 +4,8 @@ "metadata": { "name": "fileserver", "labels": { - "app": "fileserver" + "app": "fileserver", + "x-infra-instance": "{{xinfra}}" } }, "spec": { diff --git a/javascript/words.txt b/javascript/words.txt index 2b95ab864..754a1cc05 100644 --- a/javascript/words.txt +++ b/javascript/words.txt @@ -175,6 +175,7 @@ westend westmint willbe xcmp +xinfra xzvf zipkin zndsl diff --git a/scripts/jobs_status_exporter/requirements.txt b/scripts/jobs_status_exporter/requirements.txt index 007b272e3..86b1d388f 100644 --- a/scripts/jobs_status_exporter/requirements.txt +++ b/scripts/jobs_status_exporter/requirements.txt @@ -1,3 +1,3 @@ Flask==2.0.1 -requests==2.26.0 +requests==2.31.0 prometheus_client==0.11.0