diff --git a/docs/dev/index.md b/docs/dev/index.md index b97f162b0..24a4d9cc5 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -3,7 +3,6 @@ !!! info inline end "Quick links" * [Network Configuration](./reference/network-config.md) - * [How to run a node](../infra/observation/index.md) All Flare networks are a fork of the Avalanche project, which runs the [Ethereum Virtual Machine](glossary.md#evm). Therefore, all Ethereum contracts and tools work on Flare, Songbird and Coston. diff --git a/docs/index.md b/docs/index.md index acd94aa47..1ae956d0d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -66,22 +66,6 @@ Quick links: -#### For infrastructure providers - -
- -## Infrastructure Guides - -Step-by-step guides on how to deploy the different components that make up the Flare ecosystem - -
- -Quick links: - -* [Deploying an Observer Node](./infra/observation/deploying.md) -* [Deploying a Validator Node](./infra/validation/deploying.md) -* [Observer Nodes](./infra/observation/index.md) -
diff --git a/docs/infra/data/operating.md b/docs/infra/data/operating.md index 12df60765..53a25765e 100644 --- a/docs/infra/data/operating.md +++ b/docs/infra/data/operating.md @@ -231,7 +231,7 @@ To run on the real network, you need to: Use the following tips: -* Run your own [observer node](../observation/deploying.md) and submit all your data through it. +* Run your own [observer node](https://dev.flare.network/run-node/rpc-node) and submit all your data through it. This will allow you to more efficiently and securely operate your data provider. * Gather your data directly from each source instead of using APIs provided by data aggregators. * Write your own code instead of relying entirely on third-party code. diff --git a/docs/infra/index.md b/docs/infra/index.md index 08a1c7074..38daf06a2 100644 --- a/docs/infra/index.md +++ b/docs/infra/index.md @@ -6,5 +6,3 @@ Select one of the topics below: * [Deploying FAssets Agent](./fassets/index.md) * [FTSO Data Providers](./data/index.md) -* [Observer Nodes](./observation/index.md) -* [Validator Nodes](./validation/index.md) diff --git a/docs/infra/observation/deploying-docker.md b/docs/infra/observation/deploying-docker.md deleted file mode 100644 index d251fce99..000000000 --- a/docs/infra/observation/deploying-docker.md +++ /dev/null @@ -1,540 +0,0 @@ ---- -title: Deploying with Docker ---- - -# Deploying an Observer Node with Docker - -The Flare Foundation distributes official Docker images for both Songbird (`flarefoundation/go-songbird`) and Flare (`flarefoundation/go-flare`) on [Docker Hub](https://hub.docker.com/u/flarefoundation). -These images can be used to deploy containerized observer nodes for all of Flare's networks. - -This guide explains how to deploy your own observer node via Docker and Docker Compose. - -## Prerequisites - -This guide assumes you have first read [Deploying an Observer Node](./deploying.md) to understand hardware requirements, starting the node and setting up a configuration file. -This guide focuses entirely on using Docker and Docker Compose to start an observer node, using sections from 'Deploying an Observer Node' to assist the initial set up. - -This guide also contains different instructions depending on which Flare network you want to deploy to, so [make sure you are aware of the available networks](../../tech/flare.md#the-flare-networks). - -Ensure that you have [Docker](https://docs.docker.com/get-started/overview/) installed. -Please note, the installation can vary depending on the operating system and whether you decide to use Docker Engine or Docker Desktop. -For simplicity this guide is using a Docker Engine installation on a Linux Debian machine. - -If you do not have Docker already installed follow any of the guides below. - -Recommended: - -* [Docker Engine Installation Guide (Linux distributions)](https://docs.docker.com/engine/install/) - -Alternatives: - -* [Docker Desktop Installation Guide (Windows)](https://docs.docker.com/desktop/install/windows-install/) -* [Docker Desktop Installation Guide (macOS)](https://docs.docker.com/desktop/install/mac-install/) - -To avoid using `sudo` each time you run the `docker` command, add your user to the docker group, post-installation: - -```bash -sudo usermod -a -G docker $USER -# Log out and log back in or restart your system for the changes to take effect -``` - -It is useful to install `jq` to improve readability of JSON outputs from your nodes [RPC](glossary.md#rpc). - -## Guide - -### 1. Disk Setup - -This setup varies depending on your use case, but essentially you need to have a local directory with sufficient space for the blockchain data to be stored and persisted in. -Refer to disk space requirements defined in [Deploying an Observer Node](./deploying.md#prerequisites). - -In this guide, there is an additional disk mounted at `/mnt/db`, which is used to persist the blockchain data. - -After you have a machine set up and ready to go, find the additional disk, format it if necessary, and mount to a directory: - -```bash -lsblk -# ---- -# NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT -# sda 8:0 0 10G 0 disk -# ├─sda1 8:1 0 9.9G 0 part / -# ├─sda14 8:14 0 3M 0 part -# └─sda15 8:15 0 124M 0 part /boot/efi -# sdb 8:16 0 300G 0 disk <- Device identified as db disk via size -# ---- -sudo mkdir /mnt/db -sudo chown -R : /mnt/db -sudo mkfs.ext4 -m 0 \ - -E lazy_itable_init=0,lazy_journal_init=0,discard \ - /dev/sdb -sudo mount /dev/sdb /mnt/db -``` - -!!! info - - * Replace `` with the user you wish to start your containerized observer node with. - It is recommended that this isn't the root user for security reasons. - * Ensure you are replacing `/dev/sdb` with your actual device, since it could be different to the example. - -Confirm new disk is mounted: - -```bash hl_lines="11" -df -h -# ----- -# Filesystem Size Used Avail Use% Mounted on -# udev 3.9G 0 3.9G 0% /dev -# tmpfs 796M 376K 796M 1% /run -# /dev/sda1 9.7G 1.9G 7.3G 21% / -# tmpfs 3.9G 0 3.9G 0% /dev/shm -# tmpfs 5.0M 0 5.0M 0% /run/lock -# /dev/sda15 124M 11M 114M 9% /boot/efi -# tmpfs 796M 0 796M 0% /run/user/1009 -# /dev/sdb 295G 28K 295G 1% /mnt/db -``` - -Look for your device name and mount point specified in the output to confirm the mount worked. - -Backup original `fstab` file (to revert changes if needed) and update `/etc/fstab` to make sure device is mounted when the system reboots: - -```bash -sudo -i -cp /etc/fstab /etc/fstab.backup -fstab_entry="UUID=$(blkid -o value -s UUID /dev/sdb) \ - /mnt/db \ - ext4 \ - discard,defaults \ - 0 2" -echo $fstab_entry >> /etc/fstab -exit -``` - -### 2. Configuration File and Logs Directory Setup - -Once your database directory is ready, the next step is to correctly define your configuration file and logs directory for the observer node. -Later, these are mounted from your local machine to the specified directories on your containerized observer node. - -Mounting the logs directory provides you access to the logs generated by the workload on your local machine, at the specified local directory. -This can save you some effort from using `docker logs` and instead inspecting the files in your local directory. - -This example uses the configuration provided in the "Additional Configuration" section of [Deploying an Observer Node](./deploying.md#4-additional-configuration) - -Create the local directories, change ownership to a non-root user of your choice and create configuration file: - -```bash -sudo mkdir -p /opt/flare/conf -sudo mkdir /opt/flare/logs -sudo chown -R : /opt/flare -cat > /opt/flare/conf/config.json << EOL -{ - "snowman-api-enabled": false, - "coreth-admin-api-enabled": false, - "eth-apis": [ - "eth", - "eth-filter", - "net", - "web3", - "internal-eth", - "internal-blockchain", - "internal-transaction-pool" - ], - "rpc-gas-cap": 50000000, - "rpc-tx-fee-cap": 100, - "pruning-enabled": true, - "local-txs-enabled": false, - "api-max-duration": 0, - "api-max-blocks-per-request": 0, - "allow-unfinalized-queries": false, - "allow-unprotected-txs": false, - "remote-tx-gossip-only-enabled": false, - "log-level": "info" -} -EOL -``` - -!!! info - Replace `` with the user you wish to start your containerized observer node with. - It is recommended that this isn't the root user for security reasons. - -### 3. Running with Docker - -This step in the guide demonstrates using Docker CLI or Docker Compose to run a Flare node. -Configuration of the node is the same in both steps, the Docker CLI is an easy copy and paste command to get a node running and inspect its behavior. -For something more permanent and usable for the future, follow the Docker Compose section of this step. - -#### Using Docker CLI - -=== "Flare" - - For Flare observer nodes you need to pull an image from Flare Foundation's [go-flare repository](https://hub.docker.com/r/flarefoundation/go-flare). - - Read the 'Overview' tab in the repository linked above to understand the various parameters of running a node via the Docker image. - Some important things to note are the default directory locations and the environment variables available for you to adjust the behaviour of your observer node. - - Start the container with the the parameters defined: - - ```bash - docker run -d \ - --name flare-observer \ - -e AUTOCONFIGURE_BOOTSTRAP="1" \ - -e NETWORK_ID="flare" \ - -e AUTOCONFIGURE_PUBLIC_IP="1" \ - -e AUTOCONFIGURE_BOOTSTRAP_ENDPOINT="https://flare.flare.network/ext/info" \ - -v /mnt/db:/app/db \ - -v /opt/flare/conf:/app/conf/C \ - -v /opt/flare/logs:/app/logs \ - -p 0.0.0.0:9650:9650 \ - -p 0.0.0.0:9651:9651 \ - flarefoundation/go-flare:v1.7.1807 - ``` - - Confirm your container is running and inspect that logs are printing: - - ```bash - docker ps - docker logs flare-observer -f - ``` - -=== "Songbird" - - For Songbird observer nodes you need to pull an image from Flare Foundation's [go-songbird repository](https://hub.docker.com/r/flarefoundation/go-songbird). - - Read the 'Overview' tab in the repository linked above to understand the various parameters of running a node via the Docker image. - Some important things to note are the default directory locations and the environment variables available for you to adjust the behaviour of your observer node. - - Start the container with the the parameters defined: - - ```bash - docker run -d \ - --name songbird-observer \ - -e AUTOCONFIGURE_BOOTSTRAP="1" \ - -e NETWORK_ID="songbird" \ - -e AUTOCONFIGURE_PUBLIC_IP="1" \ - -e AUTOCONFIGURE_BOOTSTRAP_ENDPOINT="https://songbird.flare.network/ext/info" \ - -v /mnt/db:/app/db \ - -v /opt/flare/conf:/app/conf/C \ - -v /opt/flare/logs:/app/logs \ - -p 0.0.0.0:9650:9650 \ - -p 0.0.0.0:9651:9651 \ - flarefoundation/go-songbird:v0.6.4 - ``` - - Confirm your container is running and inspect that logs are printing: - - ```bash - docker ps - docker logs songbird-observer -f - ``` - -=== "Coston" - - For Coston observer nodes you need to pull an image from Flare Foundation's [go-songbird repository](https://hub.docker.com/r/flarefoundation/go-songbird). - This is because Coston is Songbird's testnet and uses the same code. - - Read the 'Overview' tab in the repository linked above to understand the various parameters of running a node via the Docker image. - Some important things to note are the default directory locations and the environment variables available for you to adjust the behaviour of your observer node. - - Start the container with the the parameters defined: - - ```bash - docker run -d \ - --name coston-observer \ - -e AUTOCONFIGURE_BOOTSTRAP="1" \ - -e NETWORK_ID="coston" \ - -e AUTOCONFIGURE_PUBLIC_IP="1" \ - -e AUTOCONFIGURE_BOOTSTRAP_ENDPOINT="https://coston.flare.network/ext/info" \ - -v /mnt/db:/app/db \ - -v /opt/flare/conf:/app/conf/C \ - -v /opt/flare/logs:/app/logs \ - -p 0.0.0.0:9650:9650 \ - -p 0.0.0.0:9651:9651 \ - flarefoundation/go-songbird:v0.6.4 - ``` - - Confirm your container is running and inspect that logs are printing: - - ```bash - docker ps - docker logs coston-observer -f - ``` - -=== "Coston2" - - For Coston2 observer nodes you need to pull an image from Flare Foundation's [go-flare repository](https://hub.docker.com/r/flarefoundation/go-flare). - This is because Coston2 is Flare's testnet and uses the same code. - - Read the 'Overview' tab in the repository linked above to understand the various parameters of running a node via the Docker image. - Some important things to note are the default directory locations and the environment variables available for you to adjust the behaviour of your observer node. - - Start the container with the the parameters defined: - - ```bash - docker run -d \ - --name coston2-observer \ - -e AUTOCONFIGURE_BOOTSTRAP="1" \ - -e NETWORK_ID="costwo" \ - -e AUTOCONFIGURE_PUBLIC_IP="1" \ - -e AUTOCONFIGURE_BOOTSTRAP_ENDPOINT="https://coston2.flare.network/ext/info" \ - -v /mnt/db:/app/db \ - -v /opt/flare/conf:/app/conf/C \ - -v /opt/flare/logs:/app/logs \ - -p 0.0.0.0:9650:9650 \ - -p 0.0.0.0:9651:9651 \ - flarefoundation/go-flare:v1.7.1807 - ``` - - Confirm your container is running and inspect that logs are printing: - - ```bash - docker ps - docker logs coston2-observer -f - ``` - -Once you have confirmed that the container is running, use Ctrl+C to exit the following of logs and check your container's `/ext/health` endpoint. -Only when the observer node is fully synced will you see `"healthy": true`, but this otherwise confirms your container's HTTP port (9650) is accessible from your local machine. - -```bash -curl http://localhost:9650/ext/health | jq -``` - -Command arguments explained: - -Volumes: - -* `-v /mnt/db:/app/db` - - Mount the local database directory to the default database directory of the container. - -* `-v /opt/flare/conf:/app/conf/C` - - Mount the local configuration directory to the default location of `config.json`. - -* `-v /opt/flare/logs:/app/logs` - - Mount the local logs directory to the workloads default logs directory. - -Ports: - -* `-p 0.0.0.0:9650:9650` - - Mapping the container's HTTP port to your local machine, enabling the querying of the containerized observer node's HTTP port via your local machine's IP and port. - - !!! warning - Only use binding `0.0.0.0` for port 9650 if you wish to publicly expose your containerized observer node's RPC endpoint from your machine's public IP address. - If you require it to be publicly accessible for another application to use, ensure you set up a firewall rule to only allow port 9650 to be accessible via specific source IP addresses. - -* `-p 0.0.0.0:9651:9651` - - Mapping the container's peering port to your local machine so other peers can query the node. - -Environment Variables: - -* `-e AUTOCONFIGURE_BOOTSTRAP="1"` - - Retrieves the bootstrap endpoints Node-IP and Node-ID automatically. - -* `-e NETWORK_ID=""` - - Sets the correct network ID from the provided options below: - - * `coston` - * `costwo` - * `songbird` - * `flare` - -* `-e AUTOCONFIGURE_PUBLIC_IP="1"` - - Retrieves your local machine's IP automatically. - -* `-e AUTOCONFIGURE_BOOTSTRAP_ENDPOINT="/ext/info"` - - Defines the bootstrap endpoint used to initialize chain sync. - Flare nodes can be used to bootstrap your node for each chain: - - * `https://coston.flare.network/ext/info` - * `https://costwo.flare.network/ext/info` - * `https://songbird.flare.network/ext/info` - * `https://flare.flare.network/ext/info` - -#### Using Docker Compose - -Docker Compose for this use case is a good way to simplify your setup of running the observer node. -Adding all necessary configurations into a single file that can be run with a simple command. - -In this guide the `docker-compose.yaml` file is created in `/opt/observer` but the location is entirely up to you. - -Create the working directory, set the ownership and create the `docker-compose.yaml` file: - -=== "Flare" - - ```bash - sudo mkdir /opt/observer - sudo chown -R : /opt/observer - cat > /opt/observer/docker-compose.yaml << EOL - version: '3.6' - - services: - observer: - container_name: flare-observer - image: flarefoundation/go-flare:v1.7.1807 - restart: on-failure - environment: - - AUTOCONFIGURE_BOOTSTRAP=1 - - NETWORK_ID=flare - - AUTOCONFIGURE_PUBLIC_IP=1 - - AUTOCONFIGURE_BOOTSTRAP_ENDPOINT=https://flare.flare.network/ext/info - volumes: - - /mnt/db:/app/db - - /opt/flare/conf:/app/conf/C - - /opt/flare/logs:/app/logs - ports: - - 0.0.0.0:9650:9650 - - 0.0.0.0:9651:9651 - EOL - ``` - -=== "Songbird" - - ```bash - sudo mkdir /opt/observer - sudo chown -R : /opt/observer - cat > /opt/observer/docker-compose.yaml << EOL - version: '3.6' - - services: - observer: - container_name: songbird-observer - image: flarefoundation/go-songbird:v0.6.4 - restart: on-failure - environment: - - AUTOCONFIGURE_BOOTSTRAP=1 - - NETWORK_ID=songbird - - AUTOCONFIGURE_PUBLIC_IP=1 - - AUTOCONFIGURE_BOOTSTRAP_ENDPOINT=https://songbird.flare.network/ext/info - volumes: - - /mnt/db:/app/db - - /opt/flare/conf:/app/conf/C - - /opt/flare/logs:/app/logs - ports: - - 0.0.0.0:9650:9650 - - 0.0.0.0:9651:9651 - EOL - ``` - -=== "Coston" - - ```bash - sudo mkdir /opt/observer - sudo chown -R : /opt/observer - cat > /opt/observer/docker-compose.yaml << EOL - version: '3.6' - - services: - observer: - container_name: coston-observer - image: flarefoundation/go-songbird:v0.6.4 - restart: on-failure - environment: - - AUTOCONFIGURE_BOOTSTRAP=1 - - NETWORK_ID=coston - - AUTOCONFIGURE_PUBLIC_IP=1 - - AUTOCONFIGURE_BOOTSTRAP_ENDPOINT=https://coston.flare.network/ext/info - volumes: - - /mnt/db:/app/db - - /opt/flare/conf:/app/conf/C - - /opt/flare/logs:/app/logs - ports: - - 0.0.0.0:9650:9650 - - 0.0.0.0:9651:9651 - EOL - ``` - -=== "Coston2" - - ```bash - sudo mkdir /opt/observer - sudo chown -R : /opt/observer - cat > /opt/observer/docker-compose.yaml << EOL - version: '3.6' - - services: - observer: - container_name: coston2-observer - image: flarefoundation/go-flare:v1.7.1807 - restart: on-failure - environment: - - AUTOCONFIGURE_BOOTSTRAP=1 - - NETWORK_ID=costwo - - AUTOCONFIGURE_PUBLIC_IP=1 - - AUTOCONFIGURE_BOOTSTRAP_ENDPOINT=https://coston2.flare.network/ext/info - volumes: - - /mnt/db:/app/db - - /opt/flare/conf:/app/conf/C - - /opt/flare/logs:/app/logs - ports: - - 0.0.0.0:9650:9650 - - 0.0.0.0:9651:9651 - EOL - ``` - -Start Docker Compose: - -```bash -docker compose -f /opt/observer/docker-compose.yaml up -d -``` - -When the Docker Compose command completes, check the container is running and inspect that logs are printing: - -```bash -docker ps -docker compose logs -f -``` - -Once you have confirmed the container is running, use Ctrl+C to exit the following of logs and check your container's `/ext/health` endpoint. -Only when the observer node is fully synced will you see `"healthy": true`, but this otherwise confirms your container's HTTP port (9650) is accessible from your local machine. - -```bash -curl http://localhost:9650/ext/health | jq -``` - -### 4. Additional Configuration - -There are plenty of environment variables at your disposal to adjust your workload at runtime. -The example Docker and Docker Compose guides above assumed some defaults and utilized built-in automation scripts for most of the configuration. -Outlined below are all options available: - -| Name | Default | Description | -| ------------------------: | :---------- | :------------------------------------------------------------------- | -| `HTTP_HOST` | `0.0.0.0` | HTTP host binding address | -| `HTTP_PORT` | `9650` | The listening port for the HTTP host | -| `STAKING_PORT` | `9651` | The staking port for bootstrapping nodes | -| `PUBLIC_IP` | (empty) | Public facing IP. Must be set if `AUTOCONFIGURE_PUBLIC_IP=0` | -| `DB_DIR` | `/app/db` | The database directory location | -| `DB_TYPE` | `leveldb` | The database type to be used | -| `BOOTSTRAP_IPS` | (empty) | A list of bootstrap server IPs | -| `BOOTSTRAP_IDS` | (empty) | A list of bootstrap server IDs | -| `CHAIN_CONFIG_DIR` | `/app/conf` | Configuration folder where you should mount your configuration file | -| `LOG_DIR` | `/app/logs` | Logging directory | -| `LOG_LEVEL` | `info` | Logging verbosity level that is logged into the file | -| `AUTOCONFIGURE_PUBLIC_IP` | `0` | Set to 1 to autoconfigure `PUBLIC_IP`, skipped if `PUBLIC_IP` is set | -| `AUTOCONFIGURE_BOOTSTRAP` | `0` | Set to 1 to autoconfigure `BOOTSTRAP_IPS` and `BOOTSTRAP_IDS` | -| `EXTRA_ARGUMENTS` | (empty) | Extra arguments passed to flare binary | - -Additionally: - -* `NETWORK_ID` - - **Default:** The default depends on the image you use, so either go-songbird (`default: coston`) or go-flare (`default: costwo`) - - **Description:** Name of the network you want to connect to. - -* `AUTOCONFIGURE_BOOTSTRAP_ENDPOINT` - - **Default:** `https://coston2.flare.network/ext/info` or `https://flare.flare.network/ext/info` - - **Description:** Endpoint used to automatically retrieve the Node-ID and Node-IP from. - -* `AUTOCONFIGURE_FALLBACK_ENDPOINTS` - - **Default:** (empty) - - **Description:** Comma-divided fallback bootstrap endpoints, used if `AUTOCONFIGURE_BOOTSTRAP_ENDPOINT` is not valid, such as the bootstrap endpoint being unreachable. - Tested from first-to-last, until one is valid. diff --git a/docs/infra/observation/deploying.md b/docs/infra/observation/deploying.md deleted file mode 100644 index 27b82df78..000000000 --- a/docs/infra/observation/deploying.md +++ /dev/null @@ -1,398 +0,0 @@ -# Deploying an Observer Node - -Observer nodes enable anyone to observe the network and submit transactions. -Unlike [validator nodes](../../tech/validators.md), which provide state consensus and add blocks, observer nodes remain outside the network and have no effect on consensus or blocks. - -Running an observer node is optional. -However, submitting transactions through your own node offers a number of benefits: - -* Transactions are sent directly to the network instead of through a third party, removing a potential security risk. -* Public nodes are usually rate-limited (the amount of requests they accept per second is restricted). - Your own node does not have such restriction. -* The time savings described above allow [FTSO data providers](glossary.md#data_provider) to submit their data a few seconds later, thus having more time to gather data before submitting. - -This guide explains how to deploy your own observer node so you can reap the benefits. - -## Prerequisites - -This guide contains different instructions depending on which Flare network you want to deploy to, so [make sure you are aware of the available networks](../../tech/flare.md#flare-networks). - -=== "Flare" - - | | Hardware | | Software | - | --------------: | :---------- | -------------------: | ---------------------------------------------------- | - | **CPU cores** | 8 | **Operating System** | Ubuntu (18.04 or 20.04) or macOS (>= 10.15 Catalina) | - | **RAM** | 32 GB | **Dependencies** | [Go](https://golang.org/doc/install) (>= 1.18.5) | - | **Disk space** | 1 TB SSD | | [gcc](https://gcc.gnu.org/) | - | **Disk growth** | 2.5 TB/year | | [g++](https://gcc.gnu.org/) | - | | | | [jq](https://stedolan.github.io/jq/) | - -=== "Songbird" - - | | Hardware | | Software | - | --------------: | :---------- | -------------------: | ---------------------------------------------------- | - | **CPU cores** | 8 | **Operating System** | Ubuntu (18.04 or 20.04) or macOS (>= 10.15 Catalina) | - | **RAM** | 32 GB | **Dependencies** | [Go](https://golang.org/doc/install) (>= 1.16.8) | - | **Disk space** | 3.5 TB SSD | | [gcc](https://gcc.gnu.org/) | - | **Disk growth** | 2.5 TB/year | | [g++](https://gcc.gnu.org/) | - | | | | [jq](https://stedolan.github.io/jq/) | - -=== "Coston" - - | | Hardware | | Software | - | --------------: | :---------- | -------------------: | ---------------------------------------------- | - | **CPU cores** | 4 | **Operating System** | Ubuntu (20.04 or 22.04) | - | **RAM** | 16 GB | **Dependencies** | [Go](https://golang.org/doc/install) (>= 1.21) | - | **Disk space** | 500 GB SSD | | [gcc](https://gcc.gnu.org/) | - | **Disk growth** | 250 GB/year | | [g++](https://gcc.gnu.org/) | - | | | | [jq](https://stedolan.github.io/jq/) | - -=== "Coston2" - - | | Hardware | | Software | - | --------------: | :---------- | -------------------: | ---------------------------------------------------- | - | **CPU cores** | 4 | **Operating System** | Ubuntu (18.04 or 20.04) or macOS (>= 10.15 Catalina) | - | **RAM** | 16 GB | **Dependencies** | [Go](https://golang.org/doc/install) (>= 1.18.5) | - | **Disk space** | 500 GB SSD | | [gcc](https://gcc.gnu.org/) | - | **Disk growth** | 250 GB/year | | [g++](https://gcc.gnu.org/) | - | | | | [jq](https://stedolan.github.io/jq/) | - -Plus a reliable IPv4 or IPv6 network connection, with an open public port. - -Keep in mind that enabling [pruning](glossary.md#pruning) as [described below](#4-additional-configuration) can reduce the required disk space by as much as 60%. - -## Guide - -### 1. Installation - -=== "Flare, Coston & Coston2" - - Clone the [go-flare](https://github.com/flare-foundation/go-flare) repository and run the `build.sh` script. - - !!! note - For Coston, use the `v0.6.6-coston` tag. - - ```bash - git clone https://github.com/flare-foundation/go-flare.git - cd go-flare/avalanchego - ./scripts/build.sh - ``` - - The resulting executable will be `build/avalanchego`. - - !!! note - - You can verify the installation by running: - - ```bash - go test $(go list ./... | grep -v /tests/) # avalanchego unit tests - cd ../coreth - go test ./... # coreth unit tests - cd ../avalanchego - ``` - -=== "Songbird" - - Clone the [go-songbird](https://github.com/flare-foundation/go-songbird) repository and run the `build.sh` script: - - ```bash - git clone https://github.com/flare-foundation/go-songbird.git - cd go-songbird/avalanchego - ./scripts/build.sh - ``` - - The resulting executable will be `build/flare`. - - !!! note - - You can verify the installation by running: - - ```bash - go test $(go list ./... | grep -v /tests/) # avalanchego unit tests - cd coreth - go test ./... # coreth unit tests - cd .. - ``` - -### 2. Songbird Node Whitelisting - -While the **Songbird** network is being tested, all nodes wanting to peer with it, including observer nodes, need to have their IP address **whitelisted**. - -To do this, please **contact Tom T.** over Discord (`Tom T#7603`), Telegram (`@TampaBay7`), or email ([tom@flare.network](mailto:tom@flare.network)), and request to be whitelisted. -To have greater redundancy, you can whitelist multiple nodes per single provider. - -??? tip "Checking the status of your Songbird whitelisting request" - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' \ - https://songbird.flare.network/ext/info - ``` - - If your IP address is whitelisted, this command returns a JSON response. - Otherwise you will get a 403 error ("Forbidden"). - -Please note that whitelisting is **not needed** on the **Flare network** or any of the **Coston networks**. - -### 3. Run the Node - -This is the minimum command to quickly get your node up and running. -To understand each parameter read the following step before launching the node. - -=== "Flare" - - ``` bash - ./build/avalanchego --network-id=flare --http-host= \ - --bootstrap-ips="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' https://flare.flare.network/ext/info \ - | jq -r ".result.ip")" \ - --bootstrap-ids="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' https://flare.flare.network/ext/info \ - | jq -r ".result.nodeID")" - ``` - -=== "Songbird" - - ``` bash - ./build/flare --network-id=songbird --http-host= \ - --bootstrap-ips="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' https://songbird.flare.network/ext/info \ - | jq -r ".result.ip")" \ - --bootstrap-ids="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' https://songbird.flare.network/ext/info \ - | jq -r ".result.nodeID")" - ``` - -=== "Coston" - - ``` bash - ./build/avalanchego --network-id=coston --http-host= \ - --bootstrap-ips="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' https://coston.flare.network/ext/info \ - | jq -r ".result.ip")" \ - --bootstrap-ids="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' https://coston.flare.network/ext/info \ - | jq -r ".result.nodeID")" - ``` - -=== "Coston2" - - ``` bash - ./build/avalanchego --network-id=costwo --http-host= \ - --bootstrap-ips="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' https://coston2.flare.network/ext/info \ - | jq -r ".result.ip")" \ - --bootstrap-ids="$(curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' https://coston2.flare.network/ext/info \ - | jq -r ".result.nodeID")" - ``` - -After a lot of log messages the node should start **synchronizing** with the network, which might take a long time (currently about 4 hours for Flare, over a week for Songbird, depending on network speed and machine specs). - -You can **stop** the node at any time by pressing `Ctrl-C`. -Use the same command line as before to **restart** the node. -Synchronization will resume where it left if it is interrupted. - -You will know your node is fully booted and accepting transactions when the output of this command: - -```bash -curl http://127.0.0.1:9650/ext/health -``` - -Contains the field `"healthy":true` in the returned JSON object. - -!!! note - - If the node gets stuck during bootstrap (it takes far longer than the estimates given above), try to add the parameter [`--bootstrap-retry-enabled=false`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--bootstrap-retry-enabled-boolean). - -### 4. Additional Configuration - -These are some of the most relevant command line parameters you can use. -You can read about all of them in the [Avalanche documentation](https://docs.avax.network/nodes/maintain/avalanchego-config-flags). - -* [`--bootstrap-ips`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--bootstrap-ips-string), - [`--bootstrap-ids`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--bootstrap-ids-string): - IP address and node ID of the peer used to connect to the rest of the network for bootstrapping. - - You can use Flare's public nodes for this, as shown in the quick start command given above: - - === "Flare" - - Peer's IP address: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' \ - https://flare.flare.network/ext/info | jq -r ".result.ip" - ``` - - Peer's node ID: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' \ - https://flare.flare.network/ext/info | jq -r ".result.nodeID" - ``` - - === "Songbird" - - Peer's IP address: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' \ - https://songbird.flare.network/ext/info | jq -r ".result.ip" - ``` - - Peer's node ID: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' \ - https://songbird.flare.network/ext/info | jq -r ".result.nodeID" - ``` - - === "Coston" - - Peer's IP address: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' \ - https://coston.flare.network/ext/info | jq -r ".result.ip" - ``` - - Peer's node ID: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' \ - https://coston.flare.network/ext/info | jq -r ".result.nodeID" - ``` - - === "Coston2" - - Peer's IP address: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeIP" }' \ - -H 'content-type:application/json;' \ - https://coston2.flare.network/ext/info | jq -r ".result.ip" - ``` - - Peer's node ID: - - ``` bash - curl -m 10 -sX POST \ - --data '{ "jsonrpc":"2.0", "id":1, "method":"info.getNodeID" }' \ - -H 'content-type:application/json;' \ - https://coston2.flare.network/ext/info | jq -r ".result.nodeID" - ``` - - Remember that you need to whitelist your node's IP address or your queries will always be answered with 403 error codes. - -* [`--http-host`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--http-host-string): - Use `--http-host=` (empty) to allow connections from other machines. - Otherwise, only connections from `localhost` are accepted. - -* [`--http-port`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--http-port-int): - The port through which the node will listen to API requests. - The default value is `9650`. - -* [`--staking-port`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--staking-port-int): - The port through which the network peers will connect to this node externally. - Having this port accessible from the internet is required for correct node operation. - The default value is `9651`. - -* [`--db-dir`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--db-dir-string-file-path): - Directory where the database is stored. - Make sure to use a disk with enough space as recommended in the [Hardware prerequisites](#prerequisites) section. - It defaults to `~/.avalanchego/db` on Flare and Coston2, and to `~/.flare/db` on Songbird and Coston. -{ #database-location } - - For example, you can use this option to store the database on an external drive. - -* [`--chain-config-dir`](https://docs.avax.network/nodes/maintain/avalanchego-config-flags#--chain-config-dir-string): - Optional JSON configuration file, in case you want to use lots of non-default values. - - ??? tip "Sample configuration file for observer nodes" - - These are the most common configuration options. - Put them in a file in the `{chain-config-dir}/C/config.json` folder. - - ```json - { - "snowman-api-enabled": false, - "coreth-admin-api-enabled": false, - "eth-apis": [ - "eth", - "eth-filter", - "net", - "web3", - "internal-eth", - "internal-blockchain", - "internal-transaction-pool" - ], - "rpc-gas-cap": 50000000, - "rpc-tx-fee-cap": 100, - "pruning-enabled": true, - "local-txs-enabled": false, - "api-max-duration": 0, - "api-max-blocks-per-request": 0, - "allow-unfinalized-queries": false, - "allow-unprotected-txs": false, - "remote-tx-gossip-only-enabled": false, - "log-level": "info" - } - ``` - -**Archival nodes**: An archival node keeps the whole history of the blockchain, instead of [pruning](glossary.md#pruning) old transactions which is the default setting. -Use the [`pruning-enabled`](https://docs.avax.network/nodes/maintain/chain-config-flags#pruning-enabled-boolean) configuration setting to control whether your node performs pruning or not. -Archival nodes have significantly increased disk requirements. - -### 5. Maintaining a Healthy Node - -In some cases, your node might not work correctly or you might receive unusual messages that appear difficult to troubleshoot. -Use the following solutions to ensure your node stays healthy: - -* Remember that when your node has less than 16 peers, your node will not work correctly. - To retrieve the number of connected peers, run the following command and find the line that contains `connectedPeers`: - - ```bash - curl http://127.0.0.1:9650/ext/health | jq - ``` - - To automate the process, use: - - ```bash - curl -s http://127.0.0.1:9650/ext/health | \ - jq -r ".checks.network.message.connectedPeers" - ``` - -* If your node does not sync after a long time and abruptly stops working, ensure the [database location](#database-location) has sufficient disk space, and remember the database size might change a lot during bootstrapping. -* If you receive unusual messages after you make submissions or when transactions are reverted, your node might not be connected correctly. - First, ensure the database location has sufficient disk space, and then restart the node. -* If you receive this error related to `GetAcceptedFrontier` during bootstrapping, your node was disconnected during bootstrapping. - Restart the node. - - ```text - failed to send GetAcceptedFrontier(MtF8bVH241hetCQJgsKEdKyJBs8vhp1BC, 11111111111111111111111111111111LpoYY, NUMBER) - ``` - -* If you sync your node, but it stays unhealthy for no discernible reason, restart the node. diff --git a/docs/infra/observation/index.md b/docs/infra/observation/index.md deleted file mode 100644 index f2e24fa68..000000000 --- a/docs/infra/observation/index.md +++ /dev/null @@ -1,6 +0,0 @@ -# Observer Nodes - -Select one of the guides below: - -* [Deploying an Observer Node](./deploying.md) -* [Deploying with Docker](./deploying-docker.md) \ No newline at end of file diff --git a/docs/infra/validation/deploying.md b/docs/infra/validation/deploying.md deleted file mode 100644 index e03582296..000000000 --- a/docs/infra/validation/deploying.md +++ /dev/null @@ -1,147 +0,0 @@ -# Deploying a Validator Node - -Deploy validator nodes to participate in consensus and collect rewards that the network provides to those who help secure it. - -As explained in the [Validator Nodes](../../tech/validators.md) page, they are servers that fulfill a critical role in securing the network. -They: - -* Check that all received [transactions](glossary.md#transaction) are valid. -* Run a [consensus](glossary.md#consensus) algorithm so that all validators in the network agree on the transactions to add to the blockchain. -* Finally, add the agreed-upon transactions to their copy of the [ledger](glossary.md#ledger). - -Additionally, all blockchains employ certain measures against [Sybil attacks](glossary.md#sybil_resistance). -In this regard, Flare validators must: - -* Be associated with an [FTSO Data Provider](glossary.md#data_provider). - The performance of the data provider has an impact on the validation rewards. -* Stake native tokens, as in [Proof of Stake](glossary.md#proof_of_stake). - See any of the [staking guides](../../user/staking/index.md) to learn how to do it and the minimum amount required. - -To maintain decentralization, these limits apply: - -* A single FTSO data provider can be associated with up to 4 validators, each with its own stake and each claiming their own rewards. - Each validator can be associated with only one FTSO provider. -* The [staking cap](../../user/staking/index.md#limits) on each validators' rewards is 5% of the total staked funds on the network. - -!!! info "The following instructions apply to the Flare network only. Validators on Songbird are managed by the Flare Foundation." - -## Prerequisites - -Validators run the same software as regular **observer nodes**, therefore, this guide assumes you have already read the [Deploying an Observer Node](../observation/deploying.md) guide. - -The requirements to deploy a validator node are the same as for observer nodes, except on the CPU and RAM front which are heavier due to the extra work required: - -| | Hardware | | Software | -| --------------: | :---------- | -------------------: | ---------------------------------------------------- | -| **CPU cores** | 16 | **Operating System** | Ubuntu (18.04 or 20.04) or macOS (>= 10.15 Catalina) | -| **RAM** | 64 GB | **Dependencies** | [Go](https://golang.org/doc/install) (>= 1.18.5) | -| **Disk space** | 1 TB SSD | | [gcc](https://gcc.gnu.org/) | -| **Disk growth** | 2.5 TB/year | | [g++](https://gcc.gnu.org/) | -| | | | [jq](https://stedolan.github.io/jq/) | -| | | | [npm](https://docs.npmjs.com) (>= 8.11) | - -## Guide - -A validator node is deployed like an observer node, but there are some additional considerations. - -Firstly, validators do more work than plain observer nodes so please consider the recommended **hardware specifications** above. - -And secondly, validator **security** impacts the whole network, so configure the node for security, as follows, before running it. - -### 1. Configure the Node - -Please consider the following security items carefully: - -#### Mandatory security measures - -* Ensure port 9650 is not externally reachable. - This is the port used to answer API requests and validators should not be doing that. - -* Disallow password authentication over SSH. - -* Don't run any non-validator services on the same IP (website, mail server, etc). - -!!! warning - A monitoring tool run by Flare periodically checks that the above measures are followed by all validators. - - Failure to comply impacts the validator's rewards. - -#### Suggested security measures - -* Disallow any ICMP traffic. - -* Have the machine **firewalled**. - Only the ports required for validator operation should be open (i.e. only the staking port, which defaults to 9651). - - If you use a virtual server, use only its web interface for management and close the SSH port. - - If the SSH port must be open, it should ideally be restricted to a private IP (i.e. only accessible through VPN) or only temporarily open to the operator's office/home static IP or a bastion SSH VM that can be turned off between use. - -* The node should **only act as a validator**, and not accept RPC API calls. - - You should deploy a separate observer node for tasks requiring RPC API access. - Additionally, this observer node can point to your validator for peering and bootstrapping. - -* The validator should only enable the minimum set of [EVM](glossary.md#evm) APIs by adding this line to a [configuration file](../observation/deploying.md#additional-configuration): - - ```json - "eth-apis": [ - "web3" - ] - ``` - - ??? example "Sample configuration file for validator nodes" - - ```json - { - "snowman-api-enabled": false, - "coreth-admin-api-enabled": false, - "coreth-admin-api-dir": "", - "eth-apis": [ - "web3" - ], - "continuous-profiler-dir": "", - "continuous-profiler-frequency": 900000000000, - "continuous-profiler-max-files": 5, - "rpc-gas-cap": 50000000, - "rpc-tx-fee-cap": 100, - "preimages-enabled": false, - "pruning-enabled": true, - "snapshot-async": true, - "snapshot-verification-enabled": false, - "metrics-enabled": true, - "metrics-expensive-enabled": false, - "local-txs-enabled": false, - "api-max-duration": 30000000000, - "ws-cpu-refill-rate": 0, - "ws-cpu-max-stored": 0, - "api-max-blocks-per-request": 30, - "allow-unfinalized-queries": false, - "allow-unprotected-txs": false, - "keystore-directory": "", - "keystore-external-signer": "", - "keystore-insecure-unlock-allowed": false, - "remote-tx-gossip-only-enabled": false, - "tx-regossip-frequency": 60000000000, - "tx-regossip-max-size": 15, - "log-level": "info", - "offline-pruning-enabled": false, - "offline-pruning-bloom-filter-size": 512, - "offline-pruning-data-directory": "" - } - ``` - -### 2. Run the Node - -After taking the above considerations into account, you can now start up your node by following the [Deploying an Observation Node](../observation/deploying.md) guide. - -!!! info "Preregistered validator keys" - - Some users have received **preregistered validator keys**, this is, the keys required to deploy a node which has already been registered as a validator. - - If that is your case, you just need to add these parameters to the launch command line: - - ```bash - --staking-tls-cert-file= \ - --staking-tls-key-file= - ``` diff --git a/docs/infra/validation/index.md b/docs/infra/validation/index.md deleted file mode 100644 index 617f44937..000000000 --- a/docs/infra/validation/index.md +++ /dev/null @@ -1,5 +0,0 @@ -# Validator Nodes - -The following guide explains how to set up and manage [validators](../../tech/validators.md). - -* [Deploying a Validator Node](./deploying.md) diff --git a/docs/tech/validators.md b/docs/tech/validators.md index 2e292d8e3..785376689 100644 --- a/docs/tech/validators.md +++ b/docs/tech/validators.md @@ -112,7 +112,7 @@ The goal is that funds staked on the P-chain will have the same rights as wrappe ## Related Infrastructure Guides -* [Deploying a Validator Node](../infra/validation/deploying.md) +* [Deploying a Validator Node](https://dev.flare.network/run-node/validator-node)