From 426135a9defbee1e16b4c625e3b3c7b159f41b4e Mon Sep 17 00:00:00 2001 From: hatskier Date: Mon, 7 Oct 2024 19:16:10 +0200 Subject: [PATCH 1/4] chore: proper images restored --- docs/Introduction.md | 4 ++-- docs/get-started/data-formatting-processing.md | 14 +++++++++----- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/Introduction.md b/docs/Introduction.md index cbc8ce63..1d5830e9 100644 --- a/docs/Introduction.md +++ b/docs/Introduction.md @@ -41,6 +41,6 @@ RedStone was designed with a modular architecture making it easy to incorporate - Our code was audited by multiple security experts including [ABDK](https://abdk.consulting/) [Peckshield](https://peckshield.com/) and a co-founder of [L2Beat](https://pl.linkedin.com/company/l2beat#:~:text=Join%20Piotr%20Szlachciak%20Cofounder%20%26%20CEO,insights%20shaping%20the%20%23DeFi%20landscape!). - RedStone supports leading projects like [Morpho](https://morpho.org/), [Venus](https://venus.io/), and [Pendle Finance](https://www.pendle.finance/). - - RedStone Architecure + + RedStone Architecure diff --git a/docs/get-started/data-formatting-processing.md b/docs/get-started/data-formatting-processing.md index 3c21d964..b702edcd 100644 --- a/docs/get-started/data-formatting-processing.md +++ b/docs/get-started/data-formatting-processing.md @@ -1,12 +1,12 @@ --- sidebar_position: 3 -sidebar_label: "Data Formatting & Processing" +sidebar_label: "💾 Data Formatting & Processing" --- # How Data Flows to the Blockchain - - RedStone Architecture + + RedStone Architecture ## Overview @@ -32,13 +32,17 @@ All of the steps are executed automatically by the ContractWrapper and is transp 2. Data is packed into a message based on the structure of the ‘Transaction Payload’ diagram below… - - Payload Wrapping + + RedStone Payload 3. The package is appended to the original transaction message, signed, and submitted to the network.
+ + Payload Wrapping + + ### How Data Is Unpacked, Verified and Then Aggregated On-Chain Firstly, the appended data packages are extracted from the call data. Then, security steps are taken including verifying if the signature was created by a trusted provider and validating the timestamp, confirming the information is correct. Afterward, for each requested data feed RedStone calculates the number of received unique signers, extracts the value for each unique signer, and calculates the aggregated value. The middle value of all the values (median), is the default value that is provided. This logic is executed in the on-chain environment and its execution has been optimized using a low-level assembly code to reduce gas consumption to the absolute minimum. To increase the security of the RedStone Oracle system, we've created the on-chain aggregation mechanism. This mechanism adds an additional requirement of ensuring a minimum number of distinct data feeds are relied on. The values from different providers are then aggregated before returning to a consumer contract. By default, RedStone uses the median value calculation for aggregation. This way, even if a small subset of providers are corrupt (e.g. 2 of 10), it does not significantly affect the aggregated value. From de0ee5d0bea0ef881a60c1407c8699512f46e82f Mon Sep 17 00:00:00 2001 From: hatskier Date: Mon, 7 Oct 2024 19:27:27 +0200 Subject: [PATCH 2/4] docs: added lombard details --- docs/get-started/lombard.md | 55 +++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 docs/get-started/lombard.md diff --git a/docs/get-started/lombard.md b/docs/get-started/lombard.md new file mode 100644 index 00000000..57c117dc --- /dev/null +++ b/docs/get-started/lombard.md @@ -0,0 +1,55 @@ +--- +sidebar_position: 10 +sidebar_label: "⛓️‍💥 Lombard LBTC feed" +--- + +# ⛓️‍💥 Lombard LBTC feed + +The document below explains how RedStone calculates and delivers the on-chain fundamental value of the LBTC token. + +## Lombard protocol and LBTC + +### Overview + +Lombard BTC Staking Protocol is a platform that allows users to stake Bitcoin (BTC) in a trustless and secure manner. This process is powered by the Babylon Network, which integrates Bitcoin's security with the flexibility of Proof-of-Stake (PoS) systems. The Lombard protocol introduces LBTC, a liquid staking token (LST) that is backed 1:1 by BTC on the Bitcoin blockchain. The LBTC token is designed as an omni-chain token, meaning it can be minted and transferred across multiple EVM chains. Currently, LBTC minting is enabled on the Ethereum network, and the Lombard team is actively working to expand support for new chains. + +### Minting LBTC + +To mint LBTC, a user first stakes BTC by sending it to a specific address on the Bitcoin blockchain controlled by the Lombard protocol. At this stage, the BTC is locked, but LBTC is not yet minted. After the required blockchain confirmations (typically six, which takes about an hour), the user submits cryptographic proof with the target chain (currently Ethereum) and the amount of LBTC to mint. Finally, the LBTC is minted directly into the user's wallet on the selected EVM chain. + +### Burning LBTC + +To burn LBTC and unstake BTC, a user initiates the burn by sending LBTC back to the Lombard protocol on the EVM chain. Once burned, the BTC on the Bitcoin blockchain enters a 7-day withdrawal period for security reasons. After this period, the unstaked BTC is released and transferred to the user's designated Bitcoin address​. + +## How RedStone delivers LBTC/BTC fundamental price + +### Overview + +RedStone delivers LBTC/BTC fundamental price using both [pull](./models/redstone-pull.mdx) and [push](./models/redstone-push.md) models. The price is calculated as a ratio between BTC controlled by the Lombard protocol and the total supply of LBTC tokens across all supported chains. There are also adjustments for **total_unclaimed_lbtc** (LBTC tokens that are not yet minted but already have correlated BTC tokens deposited to the Lombard protocol) and **total_btc_unstakes_pending** (BTC tokens in the 7-day withdrawal period). + +Currently, the value has an upper cap of 1, meaning a healthy value is 1, indicating the protocol’s stability. A value of 1 also signifies that there are as many or more BTC held by the protocol than there are LBTC tokens in circulation, ensuring full or over-collateralization. For example, if half of the BTC tokens disappear from wallets controlled by Lombard, but the LBTC supply remains the same, the ratio would drop to 0.5. Once Lombard starts generating yield, this cap will be removed, and the value will gradually increase, similar to reward bearing LST tokens like wstETH or pufETH. + +### Calculating LBTC/BTC fundamental price (pull model) + +The following algorithm runs on each oracle node independently, every 20 minutes: + +1. Fetch the list of BTC addresses from the Lombard API +1. Fetch the list of BTC addresses from the Lombard API + 1. Verify the ownership of the address using Lombard’s root public key + 1. Save the address in the oracle-node persistent storage +1. Calculate **total_btc_balance**, by summing up Bitcoin balances for all the addresses from the oracle-node persistent storage +1. Calculate **total_btc_unstakes_pending** using the Lombard API. If the value is negative, 0 is used +1. Calculate **total_supply_on_evms**, by summing up total supplies of the LBTC token on all supported EVM chains +1. Calculate **total_unclaimed_lbtc** using the Lombard API. If the value is negative, 0 is used +1. Calculate the final ratio using the following formula: **(total_btc_balance - total_btc_unstakes_pending) / (total_supply_on_evms + total_unclaimed_lbtc)** +1. Save the value with the timestamp to persistent oracle node storage +1. Calculate median of the values saved in the persistent oracle node storage within the last 6 hours. This step is important to increase robustness and security and ignore temporary spikes caused by delays between chains. +1. Sign the value using private key and broadcast it to the RedStone distributed data layer (DDL) + + +### Saving LBTC fundamental price on Ethereum (push model) + +The value from the pull model is saved on the Ethereum network. It’s available in the following contract: [0xb415eAA355D8440ac7eCB602D3fb67ccC1f0bc81](https://etherscan.io/address/0xb415eAA355D8440ac7eCB602D3fb67ccC1f0bc81) through the standard AggregatorV3Interface interface. It's being updated at least every 24 hours or more often, if the on-chain value deviates from current value by more than 1%. + + +- [Link to the feed details in the RedStone App](https://app.redstone.finance/app/feeds/ethereum-mainnet/lbtc_fundamental/) From 216470436a2b437015f9f40b6dcdf4d50d28acb4 Mon Sep 17 00:00:00 2001 From: hatskier Date: Mon, 7 Oct 2024 19:29:49 +0200 Subject: [PATCH 3/4] fix: linter issues resolved --- docs/data-providers/deploy.md | 32 ++++++++++++++--------------- docs/data-providers/introduction.md | 11 ++++++---- docs/get-started/lombard.md | 6 ++---- 3 files changed, 24 insertions(+), 25 deletions(-) diff --git a/docs/data-providers/deploy.md b/docs/data-providers/deploy.md index 9fe653e9..d93128fd 100644 --- a/docs/data-providers/deploy.md +++ b/docs/data-providers/deploy.md @@ -5,7 +5,7 @@ sidebar_label: "Deploy" # Deploy -In this section you will see how to run a RedStone node using Docker Compose. +In this section you will see how to run a RedStone node using Docker Compose. :::caution For production deployments consider using more sofisticated tools, e.g. Kubernetes. @@ -18,11 +18,10 @@ For production deployments consider using more sofisticated tools, e.g. Kubernet - At least 30 GB of storage (mainly for logs) - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) -:::tip +:::tip The hardware requirements for running a RedStone node are quite low, but you should reserve a healthy margin. The more resources you spare when provisioning your machine, the better it will perform and the less likely it will be to run into issues. ::: - ### Docker Compose Example Here's a simple example of a `docker-compose.yml` file that you can use to run the RedStone oracle node locally. Copy this configuration and adjust it to your needs. @@ -38,7 +37,7 @@ services: - public_network - private_network volumes: - - redstone-oracle-node:/oracle-node-level-db + - redstone-oracle-node:/oracle-node-level-db environment: OVERRIDE_DIRECT_CACHE_SERVICE_URLS: '["https://httpbin.org/anything"]' OVERRIDE_MANIFEST_USING_FILE: ./manifests/dev/dev.json @@ -59,8 +58,8 @@ services: - private_key secrets: - private_key: - file: private_key.txt + private_key: + file: private_key.txt volumes: redstone-oracle-node: @@ -69,14 +68,14 @@ networks: public_network: driver: bridge private_network: - internal: true # This ensures the network is private + internal: true # This ensures the network is private ``` ### Services #### RedsStone KMS (Key Management Service) -RedStone KMS' sole purpose is to handle all operations on your private key. It signs the data fetched by the oracle node and returns the evm address. This should be the only service that has access to your private key. Use the [RedStone KMS](https://gallery.ecr.aws/y7v2w8b2/kms) Docker image. +RedStone KMS' sole purpose is to handle all operations on your private key. It signs the data fetched by the oracle node and returns the evm address. This should be the only service that has access to your private key. Use the [RedStone KMS](https://gallery.ecr.aws/y7v2w8b2/kms) Docker image. | Param | Description | Example value | | ------------------------ | ---------------------------------------------------------------------------------------------------- | ----------------------------------------------- | @@ -84,21 +83,20 @@ RedStone KMS' sole purpose is to handle all operations on your private key. It s | **KMS_PRIVATE_KEY** | Hex encoded key. Alternative way of passing the key | `KMS_PRIVATE_KEY=0xYOUR_PRIVATE_KEY` | | **KMS_ADDRESS** | Bind address | `KMS_ADDRESS=0.0.0.0:4499` | -#### RedStone Node -RedStone oracle node will fetch data from main public APIs, sign them with your private keys and broadcast to the streamr network and redstone Data Distribution Layer (DDL). -It should be configured using environment variables. +#### RedStone Node +RedStone oracle node will fetch data from main public APIs, sign them with your private keys and broadcast to the streamr network and redstone Data Distribution Layer (DDL). +It should be configured using environment variables. -| Param | Description | Example value | -| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | -| **ENABLE_REMOTE_SIGNER** | Delegate signing to a remote signer. Only this image has access to your ECDSA private key | `ENABLE_REMOTE_SIGNER=true` | -| REMOTE_SIGNER_URL | This is where Redstone's signer is listening. We recommend using a colocation e.g. in Kubernetes oracle-node and signer should be run in the same POD. By default `http://localhost:4499`.key | `REMOTE_SIGNER_URL=http://localhost:4499` | +| Param | Description | Example value | +| -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | +| **ENABLE_REMOTE_SIGNER** | Delegate signing to a remote signer. Only this image has access to your ECDSA private key | `ENABLE_REMOTE_SIGNER=true` | +| REMOTE_SIGNER_URL | This is where Redstone's signer is listening. We recommend using a colocation e.g. in Kubernetes oracle-node and signer should be run in the same POD. By default `http://localhost:4499`.key | `REMOTE_SIGNER_URL=http://localhost:4499` | | **OVERRIDE_DIRECT_CACHE_SERVICE_URLS** | Your personal private URLs of gateways to the RedStone Data Distribution Layer (DDL). For running a local node you can simply put `OVERRIDE_DIRECT_CACHE_SERVICE_URLS=["https://httpbin.org/anything"]`. But for production node running you should [request them](https://redstone.finance/discord) from the RedStone team. | `OVERRIDE_DIRECT_CACHE_SERVICE_URLS=["https://xxx.yyy.secret-url-1.com","https://zzz.aaa.secret-url-2.com"]` | | **OVERRIDE_MANIFEST_USING_FILE** | Path to your manifest file. Manifest is a public JSON file that defines the provider's obligation regarding the data that they provide. It sets fetching interval, tokens, sources and other public technical details for the provided data. You can check available manifests [here.](https://github.com/redstone-finance/redstone-oracles-monorepo/tree/main/packages/oracle-node/manifests) | `OVERRIDE_MANIFEST_USING_FILE=./manifests/dev/dev.json` | -| **LEVEL_DB_LOCATION** | Path to the level DB. Each RedStone oracle node relies on a single-level DB. It is used to store recently fetched values from the last 15 minutes. These values are used for checking value deviations, filtering outliers and preventing price manipulation attacks.

You don't need to create a Level DB instance manually, it will be created automatically at the specified path during the first node launch. | `LEVEL_DB_LOCATION=/oracle-node-level-db` | +| **LEVEL_DB_LOCATION** | Path to the level DB. Each RedStone oracle node relies on a single-level DB. It is used to store recently fetched values from the last 15 minutes. These values are used for checking value deviations, filtering outliers and preventing price manipulation attacks.

You don't need to create a Level DB instance manually, it will be created automatically at the specified path during the first node launch. | `LEVEL_DB_LOCATION=/oracle-node-level-db` | | **ENABLE_REMOTE_SIGNER** | Switch on signing with RedStone KMS | `ENABLE_REMOTE_SIGNER=true` | :::tip Custom local manifest If you want to run oracle-node from Docker with your custom manifest you should [mount the manifest file](https://docs.docker.com/storage/bind-mounts/) from your local system to the docker container and update the `OVERRIDE_MANIFEST_USING_FILE` env variable. ::: - diff --git a/docs/data-providers/introduction.md b/docs/data-providers/introduction.md index b7bd69fc..a4c4d109 100644 --- a/docs/data-providers/introduction.md +++ b/docs/data-providers/introduction.md @@ -2,29 +2,32 @@ sidebar_position: 1 sidebar_label: "Introduction" --- + # Introduction RedStone is one of the leading oracle providers on the market, but with great power comes great responsibility. In order to minimize the (already small) risk of price manipulation RedStone is allowing other, verified and trusted companies to send oracle data to its gateways. This way we increase safety, transparency and improve data quality. -We call those companies **External Oracle Providers** (EOP). - +We call those companies **External Oracle Providers** (EOP). ## Becoming External Oracle Provider Not every company may become an External Oracle Provider (EOP). Institutions need to meet some legal and technical conditions. The whole process may be divided into 3 steps - legal analysis, joining staging and joining production environments. -### Legal Analysis +### Legal Analysis + At the end of this stage EOP understands what are the legal consequences of malicious manipulation of rates. EOP also knows how the compensation mechanism works. ### Staging + Now that all legal matters are out of the way EOP will start setting up its own Oracle Node. At this point RedStone’s and EOP’s dev ops contact and set it all up. First EOP connects to staging environment where we monitor in detail how stable is the deployment. We record downtime as well as the provided prices - we do everything we can to catch problems before EOP is moved to production. This stage lasts for as long as it is needed, at least 30 days. RedStone will be in constant contact with EOP's team to inform about the problems so they may be fixed right away. ### Production + After EOP runs on Staging and both parties agree that it's ok - we move to Production. RedStone continues to monitor EOP, but now data is available in our production gateways. :::info Data availability RedStone consumers will be able to use the data published by your node [on all supported chains.](/docs/get-started/supported-chains) You can learn more about the RedStone Oracles architecture [here.](../get-started/selecting-redstone-model.md) -::: \ No newline at end of file +::: diff --git a/docs/get-started/lombard.md b/docs/get-started/lombard.md index 57c117dc..9ce619b0 100644 --- a/docs/get-started/lombard.md +++ b/docs/get-started/lombard.md @@ -35,8 +35,8 @@ The following algorithm runs on each oracle node independently, every 20 minutes 1. Fetch the list of BTC addresses from the Lombard API 1. Fetch the list of BTC addresses from the Lombard API - 1. Verify the ownership of the address using Lombard’s root public key - 1. Save the address in the oracle-node persistent storage + 1. Verify the ownership of the address using Lombard’s root public key + 1. Save the address in the oracle-node persistent storage 1. Calculate **total_btc_balance**, by summing up Bitcoin balances for all the addresses from the oracle-node persistent storage 1. Calculate **total_btc_unstakes_pending** using the Lombard API. If the value is negative, 0 is used 1. Calculate **total_supply_on_evms**, by summing up total supplies of the LBTC token on all supported EVM chains @@ -46,10 +46,8 @@ The following algorithm runs on each oracle node independently, every 20 minutes 1. Calculate median of the values saved in the persistent oracle node storage within the last 6 hours. This step is important to increase robustness and security and ignore temporary spikes caused by delays between chains. 1. Sign the value using private key and broadcast it to the RedStone distributed data layer (DDL) - ### Saving LBTC fundamental price on Ethereum (push model) The value from the pull model is saved on the Ethereum network. It’s available in the following contract: [0xb415eAA355D8440ac7eCB602D3fb67ccC1f0bc81](https://etherscan.io/address/0xb415eAA355D8440ac7eCB602D3fb67ccC1f0bc81) through the standard AggregatorV3Interface interface. It's being updated at least every 24 hours or more often, if the on-chain value deviates from current value by more than 1%. - - [Link to the feed details in the RedStone App](https://app.redstone.finance/app/feeds/ethereum-mainnet/lbtc_fundamental/) From 8fbdf8f5f9ef039dd41fab80d56c7b945bce4bfe Mon Sep 17 00:00:00 2001 From: hatskier Date: Tue, 8 Oct 2024 12:19:53 +0200 Subject: [PATCH 4/4] chore: fix after review --- docs/get-started/lombard.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/get-started/lombard.md b/docs/get-started/lombard.md index 9ce619b0..7ae0a114 100644 --- a/docs/get-started/lombard.md +++ b/docs/get-started/lombard.md @@ -34,7 +34,7 @@ Currently, the value has an upper cap of 1, meaning a healthy value is 1, indica The following algorithm runs on each oracle node independently, every 20 minutes: 1. Fetch the list of BTC addresses from the Lombard API -1. Fetch the list of BTC addresses from the Lombard API +1. For each new (not processed by RedStone previously) address 1. Verify the ownership of the address using Lombard’s root public key 1. Save the address in the oracle-node persistent storage 1. Calculate **total_btc_balance**, by summing up Bitcoin balances for all the addresses from the oracle-node persistent storage