From 2eb1a642b69ecdd80933601b5823d9170dc6b09b Mon Sep 17 00:00:00 2001 From: Ethan Frey Date: Wed, 28 Jun 2023 22:44:50 +0200 Subject: [PATCH 01/43] Update top-level documentation --- docs/README.md | 21 +++++++++++------ docs/UseCases.md | 60 +++++++++++++++++++++++++++++++++++------------- 2 files changed, 58 insertions(+), 23 deletions(-) diff --git a/docs/README.md b/docs/README.md index b87354e4..95575300 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,9 +6,8 @@ simplifications used for MVP (testnet) or v1 (production-ready, feature-limited) as footnotes in the documents. ```mermaid +%%{init: {'theme': 'forest'}}%% flowchart TD - %%{init: {'theme': 'forest'}}%% - subgraph Osmosis A{{$OSMO}} -- User Deposit --> B(Vault); B -- $OSMO --> C(Local Staker); @@ -25,18 +24,21 @@ flowchart TD end subgraph Juno + I(Price Oracle) -- price feed --> M; + J(Price Oracle) -- price feed --> N; + M(Osmosis Converter) -- virtual stake --> O(Virtual Staking 1); N(Akash Converter) -- virtual stake --> P(Virtual Staking 2); O & P -- $JUNO --> Q[Native Staking]; end - G -. IBC .-> R; + G -. IBC .-> T; subgraph Stargaze - R{{Osmosis Receiver}} -- virtual stake --> S(Virtual Staking); - S -- $STARS --> T[Native Staking]; + S(Price Oracle) -- price feed --> T; + T{{Osmosis Converter}} -- virtual stake --> U(Virtual Staking); + U -- $STARS --> V[Native Staking]; end - ``` You can get a good overview of the whole system flow in the above diagram. @@ -94,6 +96,9 @@ This max cap can be updated by governance as needed, so if a lot more (or less) tokens are staked, the max cap of the providers can be updated with a gov vote to keep them in reasonable limits. +Please [look at the use cases](./UseCases.md) to see the use of max cap in some example scenarios +to gain a better intuition of how it works. + ### Failure Modes There are three main failure modes to consider: @@ -174,7 +179,9 @@ such a condition without the Validators fault. It would require hacks deep into (not the custom ABCI app) and a governance upgrade to direct the validators to use it, and we generally consider this unlikely (it has never been observed in 4+ years of the Cosmos). -## Definitions +## Glossary + +Some common terms are defined here, which may be used throughout the documentation. - **Pairing** - a trust relationship between two chains, such that one promises to lock up slashable stake, while the other leverages this promise to issue validation power in the dPoS system. diff --git a/docs/UseCases.md b/docs/UseCases.md index 422e16e7..d44eeb47 100644 --- a/docs/UseCases.md +++ b/docs/UseCases.md @@ -1,5 +1,17 @@ # Use Cases +We assume each chain has a native staking token denom, with some quantity (could be 1 or 1,000,000,000). +This is the initial token to start the chain with. Each provider chain that connects gets a maximum of X virtual tokens, defined by consumer govenance +when authorizing the new provider. The ratio between the amount of native tokens and the max cap +of virtual tokens on each provider is a key element in defining the various security models. + +To make these diagrams easier to read, I will normalize each chain has 100 native tokens, and label the +connections on how many virtual tokens they are authorized to mint. Thus, a connection with 100 could +exert the same amount of voting power as all native stakers. A connection with 10 could exert 10% of the +power of native stakers, and a connection of 1000 could exert 10 times the power of native stakers. + +(Note this is not clear percentages. 40 / (100 + 40) = 28.5% of the total power in the hands of that provider) + ## Sibling Chains Two chains of similar size want to support each other. @@ -9,8 +21,8 @@ larger -> smaller has more weight than smaller -> larger. ```mermaid flowchart LR %%{init: {'theme': 'forest'}}%% - A(Juno) -- 40% --> B(Star); - B -- 20% --> A; + A(Juno) -- 40 --> B(Star); + B -- 20 --> A; ``` ## Full Mesh @@ -27,14 +39,19 @@ so the weights are proportional to their relative market caps. ```mermaid flowchart LR %%{init: {'theme': 'forest'}}%% - A(OSMO) == 50% ==> B(Juno); - A == 50% ==> C(Akash); - B -- 20% --> C; - C -- 20% --> B; - B -- 10% --> A; - C -- 10% --> A; + A(OSMO) == 60 ==> B(Juno); + A == 60 ==> C(Akash); + B -- 20 --> C; + C -- 20 --> B; + B -- 10 --> A; + C -- 10 --> A; ``` +You could analyze Juno in this example: +100 native, 60 from Osmosis, 20 from Akash = 180 total. +Osmosis hits the 1/3 threshold exactly, while native tokens still hold the majority in the governance votes. +Does that make sense, should this be adjusted? + ## DAOs migrating to own chain A number of Juno DAOs launching their own chains. They want to inherit most of their security from Juno, @@ -43,11 +60,13 @@ but keep governance to their own token. ```mermaid flowchart TD %%{init: {'theme': 'forest'}}%% - Juno -- 75%, no gov --> DAO1; - Juno -- 75%, no gov --> DAO2; - Juno -- 75%, no gov --> DAO3; + Juno -- 300, no gov --> DAO1; + Juno -- 300, no gov --> DAO2; + Juno -- 300, no gov --> DAO3; ``` +Note < 1/3 power in the native token, so all PoS security relies on Juno (while all governance security relies on the DAO token) + ## Almost Replicated Security Mesh Security is not ICSv1 "Replicated Security". We do not map validators from provider to consumer, but rather delegators. @@ -62,9 +81,14 @@ of the chain, including gov votes. The end effect is the new chain is almost 100 ```mermaid flowchart TD %%{init: {'theme': 'forest'}}%% - Juno -- 99%, gov --> Eve; + Juno -- 10000, gov --> Eve; ``` +The native token is just used in bootstrapping, to make setup and deployment simple. +However, at around 1% of the power, it quickly becomes irrelevant, and makes this a close approximation of "replicated security". +You could just reduce the native supply to 1 solo token and then this gets like 99.9999% replicated security, but with a much +clearer setup phase. + ## Credibly Neutral Common Good There are some items that should be neutral or independent of multiple chains, @@ -74,8 +98,12 @@ control the staking and governance, even without any native staking power. ```mermaid flowchart TD %%{init: {'theme': 'forest'}}%% - A(OSMO) --> |25%| N[Name Service]; - B(Juno) -- 25% --> N; - C(Stargaze) -- 25% --> N; - D(Cosmos Hub) -- 25% --> N; + A(OSMO) --> |10000| N[Name Service]; + B(Juno) -- 10000 --> N; + C(Stargaze) -- 10000 --> N; + D(Cosmos Hub) -- 10000 --> N; ``` + +All participating chains have the same power. The native token is almost irrelevant (100 / 40100 = 0.25% of the power) +and can be ignored. It is essential, however, for bootstrapping and getting up the chain until all the provider chains have +connected and sufficient amount of cross-stakers have joined from those chains. From 6d5887b844f68e0fe724310b676bc14d039f044b Mon Sep 17 00:00:00 2001 From: Ethan Frey Date: Wed, 28 Jun 2023 22:45:00 +0200 Subject: [PATCH 02/43] Start updating provider docs --- docs/provider/DAOs.md | 95 +++++++++++++++++++++++++++++++ docs/provider/Provider.md | 115 ++++---------------------------------- docs/provider/Vault.md | 5 +- 3 files changed, 109 insertions(+), 106 deletions(-) create mode 100644 docs/provider/DAOs.md diff --git a/docs/provider/DAOs.md b/docs/provider/DAOs.md new file mode 100644 index 00000000..3d74fd31 --- /dev/null +++ b/docs/provider/DAOs.md @@ -0,0 +1,95 @@ +# DAO DAO Extension + +After discussing this general diagram, we realized there is some value in +a simplified version of this, which may also be a great starting place to +testing out UX without the complications of IBC. DAOs have their own +token, governance, staking, and reward contracts. We can compare them to +low-cap chains embedded in a host chain. Let's look at two ways of using DAOs locally + +This document holds some brainstorm idea of how one might reuse much of the +design but customize some pieces to allow DAOs partially secured by native +tokens, or let huge DAOs provide security to the chain they run on. +None of this is fleshed out or on the road map, but is here to give +an idea on the flexibility of Mesh Security and may inspire someone +to develop it further + +## Bootstrapping DAOs + +When a new DAO launches, it often wants to accomplish two things: + +1. Ensure a reasonable security for the DAO (regardless of low market cap) +2. Airdrop some tokens to native stakers. + +By using a variant of mesh security, they can do both, acting as a +consumer of security from the native staking tokens to provide a solid +base security amount. And also, providing a share of their rewards +to those $JUNO cross-stakers, effectively airdropping those tokens +(at zero cost to the recipient) for providing some initial security. +It would look like this: + +```mermaid +flowchart LR + %%{init: {'theme': 'forest'}}%% + A{{$JUNO}} -- User Deposit --> B(Vault); + B -- $JUNO --> C(Local Staker); + C -- Stake --> D[Native Staking] + B -- Lien --> E(External Staker Neta); + E -- virtual $NETA --> F(Neta Staking contract); +``` + +Note that this requires the exact same Vault and Local Staker +as the real use case, and uses the same External Staker interface. +The "Neta Staking" contract is already built and by building out +this "External Staker Neta" adapter contract, we can work through +all the design issues in a local environment (where we can easy get +coverage with `cw-multi-test`), as well as start building out a +single-chain demo UI, where we can start discovering and resolving +the UX issues in explaining such a system. + +This may force a bit more abstraction in the system, as +"external staker Neta" doesn't take a validator as an argument, +it just stakes. Without delegation, this also brings up many questions +about governance power and such, which may be easier to prototype +in DAO contracts than modifying the Cosmos SDK staking module. + +**Recommendation** Once the MVP Vault is built, it would be good to assign +one contract dev to work out this External Staker implementation to +some standard DAO DAO staking contract (can be a "hacked" version that +just holds a lot of the DAO token, like we did in HackWasm Medellin). +This will unblock frontend developers and allow us to get much quicker +feedback on UX issues in such a system, while the backend engineers +are working with the complexities of IBC and staking virtual tokens in +the native SDK staking module. + +We don't develop this to production quality, but use it +as a proof-of-concept to get some hands on feedback on how to deal +with various issues here like different unbonding periods and +what to do about governance power. + +## Mega DAOs + +If a DAO has a market cap approaching the TVL staked in the native token, this +becomes a dangerous situation for the DAO as the security provisioned by the chain +is insufficient to protect it from attacks. We could turn this model around and allow +the DAO token to "externally stake" on the local staking contract. + +```mermaid +flowchart LR + %%{init: {'theme': 'forest'}}%% + A{{$WYND}} -- User Deposit --> B(Vault); + B -- $WYND --> C(Local Staker); + C -- Stake --> D(WYND DAO Staking) + B -- Lien --> E(External Staker Juno); + E -- virtual stake --> F(Virtual Staking); + F -- $JUNO --> G[Native Staking]; +``` + +Note this would require a different implementation for vault (to handle cw20), +and likely a different "local staker" interface (you don't select validators, but rather unbonding time). +The "External Staker JUNO" would be similar to the normal [Receiver model](../consumer/Receiver.md) +and we will need a full implementation of the [Consumer side](../consumer/Consumer.md) +implemented on the same chain. + +**Recommendation** We do not build this either as MVP or v1, and view later if it makes +sense at all. However, we should consider this use case in our designs to ensure our interfaces +and invariants make this possible. diff --git a/docs/provider/Provider.md b/docs/provider/Provider.md index 8823d6b4..34990c16 100644 --- a/docs/provider/Provider.md +++ b/docs/provider/Provider.md @@ -8,8 +8,8 @@ staking the native token locally, as well as staking "virtual tokens" on multiple external chains connected via IBC: ```mermaid +%%{init: {'theme': 'forest'}}%% flowchart LR - %%{init: {'theme': 'forest'}}%% subgraph Osmosis A{{$OSMO}} -- User Deposit --> B(Vault); B -- $OSMO --> C(Local Staker); @@ -20,7 +20,6 @@ flowchart LR E -. IBC .-> F{{Juno Consumer}}; G -. IBC .-> H{{Stars Consumer}}; - ``` ## Flows @@ -42,7 +41,7 @@ for more. The external stakers allow us to cross-stake the native vault token on other chains (or other DAOs) that use a different native -staking token, but have opt-ed in to accepting a portion +staking token, but have opted in to accepting a portion of their security from this vault in exchange for a portion of their rewards. In the basic model, these accept a lien from the vault and will communicate with a consumer interface @@ -50,7 +49,7 @@ to inform how much stake is locked to which validator and to receive rewards. [Read more about external staking](./ExternalStaking.md). -The connection to the consumer is generally over IBC and the consumer is +The connection to the consumer is over IBC and the consumer is responsible for converting these "virtual tokens" into delegations in the native staking module. Note that the consumer must first opt-in to accept the provider's tokens and can place multiple restrictions and limits @@ -66,22 +65,20 @@ The question arises as to what influence the cross-staked user can have on chain For MVP, all these delegations provide full governance power to the validator that was selected, but the cross-staker may not directly vote on any of these issues (they inherit the validator's vote). -**For MVP, no override of votes** - -For local staking (in the native token), the end goal is the cross-staker has the -same governance rights as if they had staked directly and can override -the validator's voice if they request. However, this is relatively complex when -one local staking contract hold delegations from many staker to the same validator, -and takes careful design with weighted votes and probably something -like cron cat to trigger this. -**We aim to have full participation in local votes by v2** +For local staking (in the native token), the cross-staker has the +same governance rights as if they had staked directly. They can vote +on governance proposals to override the validator's voice if they desire. +They may also restake with the same restrictions as normal (once per +unbonding period), and naturally unstake when they want. For external staking, the cross-staker will never be able to override the vote, as they are not expected to be very active in local governance on these external protocols. (If they want to participate, they can take the cross-staking rewards and delegate those tokens directly to get a voice.) +**Desired goal by v2:** + There will be two supported configurations for external staking. Either the cross-staked tokens provides governance voting power to the validator in the same proportion that it provides Tendermint voting power. @@ -89,96 +86,8 @@ Or the cross-staked tokens only provide Tendermint voting power (security) without granting more governance power to that validator. There are [use cases](../UseCases.md) for each configuration. -**By v2, we will be able to configure if cross-staked tokens provide governance power to the validator** - -## DAO DAO Extension - -After discussing this general diagram, we realized there is some value in -a simplified version of this, which may also be a great starting place to -testing out UX without the complications of IBC. DAOs have their own -token, governance, staking, and reward contracts. We can compare them to -low-cap chains embedded in a host chain. Let's look at two ways of using DAOs locally - -### Bootstrapping DAOs - -When a new DAO launches, it often wants to accomplish two things: - -1. Ensure a reasonable security for the DAO (regardless of low market cap) -2. Airdrop some tokens to native stakers. - -By using a variant of mesh security, they can do both, acting as a -consumer of security from the native staking tokens to provide a solid -base security amount. And also, providing a share of their rewards -to those $JUNO cross-stakers, effectively airdropping those tokens -(at zero cost to the recipient) for providing some initial security. -It would look like this: - -```mermaid -flowchart LR - %%{init: {'theme': 'forest'}}%% - A{{$JUNO}} -- User Deposit --> B(Vault); - B -- $JUNO --> C(Local Staker); - C -- Stake --> D[Native Staking] - B -- Lien --> E(External Staker Neta); - E -- virtual $NETA --> F(Neta Staking contract); -``` - -Note that this requires the exact same Vault and Local Staker -as the real use case, and uses the same External Staker interface. -The "Neta Staking" contract is already built and by building out -this "External Staker Neta" adapter contract, we can work through -all the design issues in a local environment (where we can easy get -coverage with `cw-multi-test`), as well as start building out a -single-chain demo UI, where we can start discovering and resolving -the UX issues in explaining such a system. - -This may force a bit more abstraction in the system, as -"external staker Neta" doesn't take a validator as an argument, -it just stakes. Without delegation, this also brings up many questions -about governance power and such, which may be easier to prototype -in DAO contracts than modifying the Cosmos SDK staking module. - -**Recommendation** Once the MVP Vault is built, it would be good to assign -one contract dev to work out this External Staker implementation to -some standard DAO DAO staking contract (can be a "hacked" version that -just holds a lot of the DAO token, like we did in HackWasm Medellin). -This will unblock frontend developers and allow us to get much quicker -feedback on UX issues in such a system, while the backend engineers -are working with the complexities of IBC and staking virtual tokens in -the native SDK staking module. - -We don't develop this to production quality, but use it -as a proof-of-concept to get some hands on feedback on how to deal -with various issues here like different unbonding periods and -what to do about governance power. - -### Mega DAOs - -If a DAO has a market cap approaching the TVL staked in the native token, this -becomes a dangerous situation for the DAO as the security provisioned by the chain -is insufficient to protect it from attacks. We could turn this model around and allow -the DAO token to "externally stake" on the local staking contract. - -```mermaid -flowchart LR - %%{init: {'theme': 'forest'}}%% - A{{$WYND}} -- User Deposit --> B(Vault); - B -- $WYND --> C(Local Staker); - C -- Stake --> D(WYND DAO Staking) - B -- Lien --> E(External Staker Juno); - E -- virtual stake --> F(Virtual Staking); - F -- $JUNO --> G[Native Staking]; -``` - -Note this would require a different implementation for vault (to handle cw20), -and likely a different "local staker" interface (you don't select validators, but rather unbonding time). -The "External Staker JUNO" would be similar to the normal [Receiver model](../consumer/Receiver.md) -and we will need a full implementation of the [Consumer side](../consumer/Consumer.md) -implemented on the same chain. - -**Recommendation** We do not build this either as MVP or v1, and view later if it makes -sense at all. However, we should consider this use case in our designs to ensure our interfaces -and invariants make this possible. +It is unclear if this is possible without forking the governance module +and design work is [tracked in this issue](https://github.com/osmosis-labs/mesh-security-sdk/issues/48) ## Optional Extensions diff --git a/docs/provider/Vault.md b/docs/provider/Vault.md index 21bc2271..c36da763 100644 --- a/docs/provider/Vault.md +++ b/docs/provider/Vault.md @@ -17,9 +17,8 @@ then one could safely use the same collateral to provide security to 20 chains, if every validator that used that collateral double-signed, there would still be enough stake to slash to cover that security promise. -As discussed in the higher-level description of the provider design, about extending -this [concept to local DAOs](./Provider.md#dao-dao-extension), -there may be many different implementations of both the _Local Staking_ concept as well +Looking at [extending the concept of mesh security to local DAOs](./DAOs.md), +we see there may be many different implementations of both the _Local Staking_ concept as well as the _External Staking_ concept. However, we must define standard interfaces here that can plug into the Vault. From 1a4a40d5d3e51803f85c84b05177acf0cf2a7df3 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 30 Jun 2023 09:36:03 +0200 Subject: [PATCH 03/43] Fix syntax / links --- docs/README.md | 28 ++++++++++++++-------------- docs/UseCases.md | 17 +++++++++-------- docs/provider/DAOs.md | 22 +++++++++++----------- docs/provider/Provider.md | 24 ++++++++++++------------ 4 files changed, 46 insertions(+), 45 deletions(-) diff --git a/docs/README.md b/docs/README.md index 95575300..750a65ed 100644 --- a/docs/README.md +++ b/docs/README.md @@ -45,7 +45,7 @@ You can get a good overview of the whole system flow in the above diagram. The design should allow one chain to provide security to multiple chains, while at the same time receiving security from multiple chains. -A key to understanding the design, is that the whole system is _delegator-centeric_ +A key to understanding the design, is that the whole system is _delegator-centric_ not _validator-centric_. This means that we don't try to match the same validators on multiple chains, or even subsets, but rather focus on leveraging the security provided by staking tokens to secure validators on multiple chains. This provides a way to @@ -65,9 +65,9 @@ Addressing some common points people raise up, which are hidden in the docs. There are many questions if this isn't "fractional reserve banking" or such. This does use the same collateral to back multiple claims (staking), but -the [final invariant in the vault](https://github.com/osmosis-labs/mesh-security/blob/main/contracts/vault/README.md#invariants) +the [final invariant in the vault](https://github.com/osmosis-labs/mesh-security/blob/main/contracts/provider/vault/README.md#invariants) ensures there is sufficient collateral to cover the maximum loss -(eg. if all local and cross-staked validators double-sign). +(e.g. if all local and cross-staked validators double-sign). If the double slash penalty is 5%, you can safely cross stake 20x. If it were to be raised to say 40%, you could only safely cross stake 2.5x This is more like insurance companies holding reserves for the worse cases expected losses, @@ -76,9 +76,9 @@ not the total value of all property insured. ### Power Limits on Remote Chains Another common concern is whether there are effective limits in the power a remote -chain can exert over one chain. Would it be possible for a higher cap chain (eg $ATOM) +chain can exert over one chain. Would it be possible for a higher cap chain (e.g. $ATOM) to take over more than one-third, or even two-thirds or the power of a smaller cap chain -(eg $STARS) it is cross-staking on. +(e.g. $STARS) it is cross-staking on. Clearly the consumer chain wants to put some limits. The first limit is the [discount applied during the conversion](https://github.com/osmosis-labs/mesh-security/blob/main/docs/consumer/Converter.md#price-normalization). @@ -110,7 +110,7 @@ There are three main failure modes to consider: #### Huge Price Swing This is the most likely scenario. If the provider price rises too much, it may suddenly have -disproportionate power over the affairs of the consumer. If the provier price falls too much, +disproportionate power over the affairs of the consumer. If the provider price falls too much, in the period until the price oracle is updated, there may be less remote collateral than the virtual stake. @@ -135,7 +135,7 @@ max cap limit preventing total takeover, so it just provides overly high rewards #### Byzantine Provider Chain The worst case is the provider claims to deposit millions of tokens on one malicious validator, -while not actually locking any collateral on it's own chain. This is similar to the case of +while not actually locking any collateral on its own chain. This is similar to the case of a hostile community on the provider chain selecting validators against the interest of the community on the consumer chain. @@ -154,10 +154,10 @@ perceived stability. If a Consumer Chain goes Byzantine (or starts some mob-rule governance proposals), it can try to damage stake on the Provider chain. There are several ways it can try to do so: -- Withholding all rewards to said Provider -- Removing all voting power from said Provider -- Refusing to unlock the virtual stake of the Provider -- Unfairly slashing virtual stake from the Provider +- Withholding all rewards to said Provider. +- Removing all voting power from said Provider. +- Refusing to unlock the virtual stake of the Provider. +- Unfairly slashing virtual stake from the Provider. The first two are temporary and can be seen in the case when the consumer no longer trusts the provider and sets "max cap" to zero. This is a temporary effect but must be acknowledged as @@ -165,7 +165,7 @@ a possible risk, which is loss of benefits, but not loss of collateral. The third point is impossible, as the unlock period is implemented between the external staking contracts and the vault on the provider side. Nothing on the Consumer can lock up stake longer. -And if a Provider feels they have unfairly withhold benefits (first two points), they could +And if a Provider feels they have unfairly withheld benefits (first two points), they could make a governance vote to allow immediate unbonding of all cross-stake to that consumer. The last point is a bit trickier. We will not fully define slashing until v1, but the design is @@ -201,7 +201,7 @@ Some common terms are defined here, which may be used throughout the documentati based on misbehavior of that validator. - **Unbonding period** - The time delay between initiating unbonding and having free access to the underlying collateral. Once this time has passed after unstaking, all claims on the underlying - collateral are released and + collateral are released. - **Rewards** - Block rewards are issued to validators in the native token of the consumer chain. A portion of these rewards goes to the stakers and is collected cross-chain. - **Slashing** - If a validator misbehaves, the tokens delegated to it, which provided the @@ -212,7 +212,7 @@ Some common terms are defined here, which may be used throughout the documentati prevented from returning. Tokens staked to it would be partially slashed and should be unstaked as soon as possible, as they will receive no more rewards. Stake to a jailed validator still must wait the unbonding period to be liquid. -- **Latency** - Time delay from an action being initiated and the effects being reflected in +- **Latency** - Time delay from an action being initiated and its effects being reflected in another contract or chain. This doesn't refer to the unbonding period, but rather the delay between initiating bonding or unbonding on the provider and the equivalent action occurring on the consumer. diff --git a/docs/UseCases.md b/docs/UseCases.md index d44eeb47..a91d34c8 100644 --- a/docs/UseCases.md +++ b/docs/UseCases.md @@ -1,16 +1,16 @@ # Use Cases We assume each chain has a native staking token denom, with some quantity (could be 1 or 1,000,000,000). -This is the initial token to start the chain with. Each provider chain that connects gets a maximum of X virtual tokens, defined by consumer govenance -when authorizing the new provider. The ratio between the amount of native tokens and the max cap -of virtual tokens on each provider is a key element in defining the various security models. +This is the initial token to start the chain with. Each provider chain that connects gets a maximum of X virtual tokens, +defined by consumer governance when authorizing the new provider. The ratio between the amount of native tokens +and the max cap of virtual tokens on each provider is a key element in defining the various security models. To make these diagrams easier to read, I will normalize each chain has 100 native tokens, and label the connections on how many virtual tokens they are authorized to mint. Thus, a connection with 100 could exert the same amount of voting power as all native stakers. A connection with 10 could exert 10% of the power of native stakers, and a connection of 1000 could exert 10 times the power of native stakers. -(Note this is not clear percentages. 40 / (100 + 40) = 28.5% of the total power in the hands of that provider) +(Note that these are not clear percentages. 40 / (100 + 40) = 28.5% of the total power in the hands of that provider). ## Sibling Chains @@ -52,7 +52,7 @@ You could analyze Juno in this example: Osmosis hits the 1/3 threshold exactly, while native tokens still hold the majority in the governance votes. Does that make sense, should this be adjusted? -## DAOs migrating to own chain +## DAOs migrating to their own chain A number of Juno DAOs launching their own chains. They want to inherit most of their security from Juno, but keep governance to their own token. @@ -65,13 +65,14 @@ flowchart TD Juno -- 300, no gov --> DAO3; ``` -Note < 1/3 power in the native token, so all PoS security relies on Juno (while all governance security relies on the DAO token) +Note that less than 1/3 power is in the native token, so all PoS security relies on Juno (while all governance security relies +on the DAO token). ## Almost Replicated Security Mesh Security is not ICSv1 "Replicated Security". We do not map validators from provider to consumer, but rather delegators. -And the power is the subset of staked tokens that opt-in, so will always be lower than full stake. By design we always require -a native staking token in the consumer chain, but we can approximate "replciated security" for the "fully owned subsidiary" +And the power is the subset of staked tokens that opt-in, so will always be lower than full stake. By design, we always require +a native staking token in the consumer chain, but we can approximate "replicated security" for the "fully owned subsidiary" use case. You can launch a chain with a governance token with minimal distribution to bootstrap the chain. Then accept another chain as a diff --git a/docs/provider/DAOs.md b/docs/provider/DAOs.md index 3d74fd31..ff639694 100644 --- a/docs/provider/DAOs.md +++ b/docs/provider/DAOs.md @@ -1,23 +1,23 @@ # DAO DAO Extension -After discussing this general diagram, we realized there is some value in -a simplified version of this, which may also be a great starting place to +After discussing this general diagram, we realized that there is some value in +a simplified version of this, which may also be a great starting place for testing out UX without the complications of IBC. DAOs have their own token, governance, staking, and reward contracts. We can compare them to -low-cap chains embedded in a host chain. Let's look at two ways of using DAOs locally +low-cap chains embedded in a host chain. Let's look at two ways of using DAOs locally. This document holds some brainstorm idea of how one might reuse much of the design but customize some pieces to allow DAOs partially secured by native tokens, or let huge DAOs provide security to the chain they run on. -None of this is fleshed out or on the road map, but is here to give +None of this is fleshed out or on the roadmap, but is here to give an idea on the flexibility of Mesh Security and may inspire someone -to develop it further +to develop it further. ## Bootstrapping DAOs When a new DAO launches, it often wants to accomplish two things: -1. Ensure a reasonable security for the DAO (regardless of low market cap) +1. Ensure a reasonable security for the DAO (regardless of low market cap). 2. Airdrop some tokens to native stakers. By using a variant of mesh security, they can do both, acting as a @@ -52,10 +52,10 @@ it just stakes. Without delegation, this also brings up many questions about governance power and such, which may be easier to prototype in DAO contracts than modifying the Cosmos SDK staking module. -**Recommendation** Once the MVP Vault is built, it would be good to assign +**Recommendation**: Once the MVP Vault is built, it would be good to assign one contract dev to work out this External Staker implementation to some standard DAO DAO staking contract (can be a "hacked" version that -just holds a lot of the DAO token, like we did in HackWasm Medellin). +just holds a lot of the DAO token, like we did in HackWasm Medellín). This will unblock frontend developers and allow us to get much quicker feedback on UX issues in such a system, while the backend engineers are working with the complexities of IBC and staking virtual tokens in @@ -86,10 +86,10 @@ flowchart LR Note this would require a different implementation for vault (to handle cw20), and likely a different "local staker" interface (you don't select validators, but rather unbonding time). -The "External Staker JUNO" would be similar to the normal [Receiver model](../consumer/Receiver.md) -and we will need a full implementation of the [Consumer side](../consumer/Consumer.md) +The "External Staker JUNO" would be similar to the normal [Receiver model](../consumer/Receiver.md), and +we will need a full implementation of the [Consumer side](../consumer/Consumer.md) implemented on the same chain. -**Recommendation** We do not build this either as MVP or v1, and view later if it makes +**Recommendation**: We do not build this either as MVP or v1, and view later if it makes sense at all. However, we should consider this use case in our designs to ensure our interfaces and invariants make this possible. diff --git a/docs/provider/Provider.md b/docs/provider/Provider.md index 34990c16..2fd51273 100644 --- a/docs/provider/Provider.md +++ b/docs/provider/Provider.md @@ -33,7 +33,7 @@ which they can withdraw assuming there are no outstanding liens on that amount. Each vault contains _exactly one_ denom and has _exactly one_ local staking module. This local staker can stake the vault token with the native staking module. It actually accepts the original token, which makes -it different than external stakers, which accept liens. By depositing in the vault +it different from external stakers, which accept liens. By depositing in the vault and staking in the local staker, I will have achieved the same effect (and get the same rewards) as directly staking... but I can now use my balance for more. @@ -45,16 +45,16 @@ staking token, but have opted in to accepting a portion of their security from this vault in exchange for a portion of their rewards. In the basic model, these accept a lien from the vault and will communicate with a consumer interface -to inform how much stake is locked to which validator and +to inform how much stake is locked to which validator, and to receive rewards. [Read more about external staking](./ExternalStaking.md). The connection to the consumer is over IBC and the consumer is responsible for converting these "virtual tokens" into delegations -in the native staking module. Note that the consumer must first opt-in to +in their native staking module. Note that the consumer must first opt in to accept the provider's tokens and can place multiple restrictions and limits -on to how much power can be granted into any external chain. -[Read more about consumers](../consumers/Consumer.md). +on how much power can be granted to any external chain. +[Read more about consumers](../consumer/Consumer.md). ## Stakers and Governance @@ -75,26 +75,26 @@ unbonding period), and naturally unstake when they want. For external staking, the cross-staker will never be able to override the vote, as they are not expected to be very active in local governance on these external protocols. (If they want to participate, they can take the -cross-staking rewards and delegate those tokens directly to get a voice.) +cross-staking rewards and delegate those tokens directly to get a voice). **Desired goal by v2:** There will be two supported configurations for external staking. Either the cross-staked tokens provides governance voting power to the validator in the same proportion that it provides Tendermint voting power. -Or the cross-staked tokens only provide Tendermint voting power (security) +Or, the cross-staked tokens only provide Tendermint voting power (security) without granting more governance power to that validator. There are [use cases](../UseCases.md) for each configuration. -It is unclear if this is possible without forking the governance module -and design work is [tracked in this issue](https://github.com/osmosis-labs/mesh-security-sdk/issues/48) +It is unclear if this is possible without forking the governance module, +and design work is [tracked in this issue](https://github.com/osmosis-labs/mesh-security-sdk/issues/48). ## Optional Extensions These won't be included in MVP and require more modifications to the core Cosmos SDK modules, which makes them more risky and -more difficult to port to other chains. But could be considered +more difficult to port to other chains. But, could be considered as chain-specific extensions. -- Enable moving bonded tokens directly into the vault? (Custom SDK change) -- Allow depositing vesting tokens to the vault? (deeper SDK change) +- Enable moving bonded tokens directly into the vault? (custom SDK change). +- Allow depositing vesting tokens to the vault? (deeper SDK change). From 754bc938f2682025c8f12353820af7b7a825291c Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 30 Jun 2023 10:25:29 +0200 Subject: [PATCH 04/43] Add ExternalStaking stub --- docs/provider/ExternalStaking.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 docs/provider/ExternalStaking.md diff --git a/docs/provider/ExternalStaking.md b/docs/provider/ExternalStaking.md new file mode 100644 index 00000000..41fa80ee --- /dev/null +++ b/docs/provider/ExternalStaking.md @@ -0,0 +1,10 @@ +# External Staking + +An external staking contract connects to a [Vault](./Vault.md). +It manages staking and staking the vault's (virtual) tokens to the remote protocol, +and releases the virtual tokens when finished unbonding. + +It also manages distribution of rewards coming from the remote (consumer) chain, +and transferring of those rewards to remote recipients on the consumer chain. + +**TODO** flesh this out From 8ff32db43178198244dccf9922ccb183e5d43c5b Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 30 Jun 2023 11:06:06 +0200 Subject: [PATCH 05/43] Fix Vault doc syntax --- docs/provider/Vault.md | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/provider/Vault.md b/docs/provider/Vault.md index c36da763..97c0a681 100644 --- a/docs/provider/Vault.md +++ b/docs/provider/Vault.md @@ -4,11 +4,11 @@ The entry point of Mesh Security is the **Vault**. This is where a potential staker can provide collateral in the form of native tokens, with which he or she wants to stake on multiple chains. -Connected to the _Vault_ contract, is exactly one [local Staking contract](./LocalStaking.md) +Connected to the _Vault_ contract, is exactly one [Local Staking contract](./LocalStaking.md) which can delegate the actual token to the native staking module. It also can connect to an -arbitrary number of [external staking contracts](./ExternalStaking.md) which can make use -of said collateral as "virtual stake" to use in an external staking system (that doesn't -use the vault token as collateral). +arbitrary number of [External Staking contracts](./ExternalStaking.md) which can make use +of said collateral as "virtual stake" to use in an external staking system (one that doesn't +directly use the vault token as collateral). The key here is that we can safely use the same collateral for multiple protocols, as the maximum slashing penalty is significantly @@ -18,8 +18,8 @@ if every validator that used that collateral double-signed, there would still be stake to slash to cover that security promise. Looking at [extending the concept of mesh security to local DAOs](./DAOs.md), -we see there may be many different implementations of both the _Local Staking_ concept as well -as the _External Staking_ concept. However, we must define +we see that there may be many different implementations of both the _Local Staking_ concept and +the _External Staking_ concept. However, we must define standard interfaces here that can plug into the Vault. We define this interface as a _Creditor_ (as it accepts liens). @@ -30,9 +30,9 @@ TODO: refine this - **Native Token** - The native staking token of this blockchain. More specifically, the token in which all collateral is measured. -- **Slashable Collateral** - `Liens(user).map(|x| x.amount * x.slashable).sum()` -- **Maximum Lien** - `Liens(user).map(|x| x.amount).max()` -- **Free Collateral** - `Collateral(user) - max(SlashableCollateral(user), MaximumLien(user))` +- **Slashable Collateral** - `Liens(user).map(|x| x.amount * x.slashable).sum()`. +- **Maximum Lien** - `Liens(user).map(|x| x.amount).max()`. +- **Free Collateral** - `Collateral(user) - max(SlashableCollateral(user), MaximumLien(user))`. ## Design Decisions @@ -41,10 +41,10 @@ created, and this contract address cannot be changed. The _vault_ contract doesn't require the _External Stakers_ to be pre-registered. Each user can decide which external staker it trusts with their tokens. (We will provide guidance in the UI to only -show "recommended" externals, but do not enforce on the contract level, if someone wants to build their own UI) +show "recommended" externals, but do not enforce at the contract level, if someone wants to build their own UI). The _vault_ contract enforces the maximum amount a given Creditor can slash to whatever was -agreed when making the lien. +agreed upon when making the lien. The _vault_ contract will only release a lien when requested by the creditor. (No auto-release override). @@ -58,7 +58,7 @@ The _vault_ contract may have a parameter to limit slashable collateral or maxim 100% of the size of the collateral. This makes handling a small slash condition much simpler. The _creditor_ is informed of a new lien and may reject it by returning an error -(eg if the slashing percentage is too small, or if the total credit would be too high). +(e.g. if the slashing percentage is too small, or if the total credit would be too high). The _creditor_ may slash the collateral up to the agreed upon amount when it was lent out. @@ -71,15 +71,15 @@ it is much less clear than the corresponding code. ### State -- Collateral: `User -> Amount` -- Liens: `User -> {Creditor, Amount, Slashable}[]` -- Credit `Creditor -> Amount` +- Collateral: `User -> Amount`. +- Liens: `User -> {Creditor, Amount, Slashable}[]`. +- Credit `Creditor -> Amount`. ### Invariants -- `SlashableCollateral(user) <= Collateral(user)` - for all users -- `MaximumLien(user) <= Collateral(user)` - for all users -- `Liens(user).map(|x| x.creditor).isUnique()` - for all users +- `SlashableCollateral(user) <= Collateral(user)` - for all users. +- `MaximumLien(user) <= Collateral(user)` - for all users. +- `Liens(user).map(|x| x.creditor).isUnique()` - for all users. ### Transitions From 663dcb312e71b649cdecc376b70f3849724b0637 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Tue, 4 Jul 2023 10:20:26 +0200 Subject: [PATCH 06/43] Improve Vault docs --- contracts/provider/vault/src/contract.rs | 4 +- docs/provider/LocalStaking.md | 2 +- docs/provider/Vault.md | 64 +++++++++++++++--------- 3 files changed, 42 insertions(+), 28 deletions(-) diff --git a/contracts/provider/vault/src/contract.rs b/contracts/provider/vault/src/contract.rs index f506b943..6c36ca75 100644 --- a/contracts/provider/vault/src/contract.rs +++ b/contracts/provider/vault/src/contract.rs @@ -51,7 +51,7 @@ pub struct VaultContract<'a> { pub local_staking: Item<'a, LocalStaking>, /// All liens in the protocol /// - /// Liens are indexed with (user, creditor), as this pair has to be unique + /// Liens are indexed with (user, lien_holder), as this pair has to be unique pub liens: Map<'a, (&'a Addr, &'a Addr), Lockable>, /// Per-user information pub users: Map<'a, &'a Addr, Lockable>, @@ -685,7 +685,7 @@ impl VaultContract<'_> { /// Updates the local stake for unstaking from any contract /// /// The unstake (both local and remote) is always called by the staking contract - /// (aka lienholder), so the `sender` address is used for that. + /// (aka lien_holder), so the `sender` address is used for that. fn unstake(&self, ctx: &mut ExecCtx, owner: String, amount: Coin) -> Result<(), ContractError> { let denom = self.config.load(ctx.deps.storage)?.denom; ensure!(amount.denom == denom, ContractError::UnexpectedDenom(denom)); diff --git a/docs/provider/LocalStaking.md b/docs/provider/LocalStaking.md index ecf8c13a..95d2787e 100644 --- a/docs/provider/LocalStaking.md +++ b/docs/provider/LocalStaking.md @@ -1,6 +1,6 @@ # Local Staking -A local staking contract connects to a [Vault](./Vault.md). Unlike other creditors, it actually +A local staking contract connects to a [Vault](./Vault.md). Unlike external stakers, it actually accepts the native token along with the lien. It manages staking the vault tokens to the native protocol and returns them when finished unbonding. diff --git a/docs/provider/Vault.md b/docs/provider/Vault.md index 97c0a681..af38434b 100644 --- a/docs/provider/Vault.md +++ b/docs/provider/Vault.md @@ -22,17 +22,25 @@ we see that there may be many different implementations of both the _Local Staki the _External Staking_ concept. However, we must define standard interfaces here that can plug into the Vault. -We define this interface as a _Creditor_ (as it accepts liens). +We define this interface as a _Lien Holder_ (as it accepts liens, i.e. embargable amounts). ## Definitions -TODO: refine this - +- **Vault** - The main contract that holds all the collateral and liens. - **Native Token** - The native staking token of this blockchain. More specifically, the token in which all collateral is measured. -- **Slashable Collateral** - `Liens(user).map(|x| x.amount * x.slashable).sum()`. -- **Maximum Lien** - `Liens(user).map(|x| x.amount).max()`. -- **Free Collateral** - `Collateral(user) - max(SlashableCollateral(user), MaximumLien(user))`. +- **Local Staking** - A contract that can delegate the native token to the native staking module. +- **External Staking** - A contract that can use the collateral as "virtual stake" in an external staking system. +- **User** - A user of the vault that provides collateral, to either the local staking or an external staking system. +- **Lien** - A promise to provide some amount of collateral to a lien holder. +- **Lien Holder** - A contract that accepts liens and can slash the collateral. +- **Collateral** - The total amount of collateral, in the native token, for a given user. +- **Slashable Collateral** - The total amount of collateral that can be slashed, in the native token, for a given user. + `Liens(user).map(|x| x.amount * x.slashable).sum()`. +- **Maximum Lien** - The maximum lien of all liens a user has. + `Liens(user).map(|x| x.amount).max()`. +- **Free Collateral** - The total collateral a user has, minus either their slashable collateral or their maximum lien (whatever is bigger). + `Collateral(user) - max(SlashableCollateral(user), MaximumLien(user))`. ## Design Decisions @@ -43,13 +51,13 @@ The _vault_ contract doesn't require the _External Stakers_ to be pre-registered which external staker it trusts with their tokens. (We will provide guidance in the UI to only show "recommended" externals, but do not enforce at the contract level, if someone wants to build their own UI). -The _vault_ contract enforces the maximum amount a given Creditor can slash to whatever was +The _vault_ contract enforces the maximum amount a given Lien Holder can slash to whatever was agreed upon when making the lien. -The _vault_ contract will only release a lien when requested by the creditor. (No auto-release override). +The _vault_ contract will only release a lien when requested by the lien holder. (No auto-release override). -The _vault_ contract may force-reduce the size of the lien only in the case of slashing by another creditor. -The creditors must be able to handle this case properly. +The _vault_ contract may force-reduce the size of the lien only in the case of slashing by another lien holder. +The lien holders must be able to handle this case properly. The _vault_ contract ensures the user's collateral is sufficient to service all the liens made on said collateral. @@ -57,12 +65,12 @@ made on said collateral. The _vault_ contract may have a parameter to limit slashable collateral or maximum lien to less than 100% of the size of the collateral. This makes handling a small slash condition much simpler. -The _creditor_ is informed of a new lien and may reject it by returning an error +The _lien holder_ is informed of a new lien, and may reject it by returning an error (e.g. if the slashing percentage is too small, or if the total credit would be too high). -The _creditor_ may slash the collateral up to the agreed upon amount when it was lent out. +The _vault_ may slash the collateral associated to a given lien holder, up to the agreed upon amount when it was lent out. -The _creditor_ should release the lien once the user terminates any agreement with the creditor. +The _vault_ should release the lien once the lien holder terminates any agreement with the user. ## Implementation @@ -71,36 +79,40 @@ it is much less clear than the corresponding code. ### State -- Collateral: `User -> Amount`. -- Liens: `User -> {Creditor, Amount, Slashable}[]`. -- Credit `Creditor -> Amount`. +- Config: General contract configuration. +- LocalStaking: Local staking info. +- Liens: All liens in the protocol. Liens are indexed with (user, lien_holder), as this pair has to be unique. +- Users: Per-user information. Collateral, max lien, total slashable amount, total used collateral and free collateral. +- Txs: Pending txs information. ### Invariants - `SlashableCollateral(user) <= Collateral(user)` - for all users. - `MaximumLien(user) <= Collateral(user)` - for all users. -- `Liens(user).map(|x| x.creditor).isUnique()` - for all users. +- `Liens(user).map(|x| x.lien_holder).isUnique()` - for all users. ### Transitions -**Provide Collateral** +**Provide Collateral (i.e. bond)** Any user may deposit native tokens to the vault contract, thus increasing their collateral as stored in this contract. -**Withdraw Collateral** +**Withdraw Collateral (i.e. unbond)** Any user may withdraw any _Free Collateral_ credited to their account. Their collateral is reduced by this amount and these native tokens are immediately transferred to their account. -**Provide Lien** +**Provide Lien (i.e. remote staking)** + +Promise collateral as slashable to some lien holder. The vault has to guarantee that that promise can be fulfilled +(i.e. that the slashable amount is always available for slashing). +Args `(lien_holder, amount, slashable)`. -TODO. Promise collateral as slashable to some creditor. -Args `(creditor, amount, slashable)`. -This is updated locally +This is updated locally to the vault. -**Release Lien** +**Release Lien (i.e. remote unstaking)** TODO @@ -108,7 +120,9 @@ TODO TODO -- Increase Slashing(user, creditor)? +- Increase Slashing(user, lien_holder)? + +TODO ## Footnotes From 37eba0f48cd8e32c34fef6a9307952e525565ead Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Tue, 4 Jul 2023 11:00:52 +0200 Subject: [PATCH 07/43] Update / complete Vault doc Transitions --- docs/provider/Vault.md | 50 ++++++++++++++++++++++++++++++++---------- 1 file changed, 38 insertions(+), 12 deletions(-) diff --git a/docs/provider/Vault.md b/docs/provider/Vault.md index af38434b..75ddfc6d 100644 --- a/docs/provider/Vault.md +++ b/docs/provider/Vault.md @@ -74,8 +74,9 @@ The _vault_ should release the lien once the lien holder terminates any agreemen ## Implementation -**TODO** translate the below into Rust code. After writing this in text, I realize -it is much less clear than the corresponding code. +- [Vault](../../contracts/provider/vault/src/contract.rs). +- [Local Staking](../../contracts/provider/native-staking/src/contract.rs). +- [External Staking](../../contracts/provider/external-staking/src/contract.rs). ### State @@ -93,36 +94,61 @@ it is much less clear than the corresponding code. ### Transitions -**Provide Collateral (i.e. bond)** +**Provide Collateral (i.e. `bond`)** Any user may deposit native tokens to the vault contract, thus increasing their collateral as stored in this contract. -**Withdraw Collateral (i.e. unbond)** +**Withdraw Collateral (i.e. `unbond`)** Any user may withdraw any _Free Collateral_ credited to their account. Their collateral is reduced by this amount and these native tokens are immediately transferred to their account. -**Provide Lien (i.e. remote staking)** +**Provide Lien (i.e. `stake_local`)** -Promise collateral as slashable to some lien holder. The vault has to guarantee that that promise can be fulfilled -(i.e. that the slashable amount is always available for slashing). -Args `(lien_holder, amount, slashable)`. +This sends actual tokens to the local staking contract for native staking. +Native staking for a user is handled through a proxy contract (i.e. `native_staking_proxy`), +which is owned by the native staking contract, and instantiated for each user. + +Available collateral and related quantities are updated locally to the vault. + +**Provide Lien (i.e. `stake_remote`)** + +Promises collateral as slashable to some lien holder. It is the responsibility of the vault to guarantee that +that promise can be fulfilled (i.e. that the associated slashable amount is always available for eventual slashing). This is updated locally to the vault. -**Release Lien (i.e. remote unstaking)** +**Release Local Stake (i.e. `release_local_stake`)** + +Local unstaking is initiated by the user on their corresponding native-staking-proxy contract (i.e. native-staking-proxy `unstake` handler). +This is because unstaking requires an unbonding period to be enforced. Only after that unbonding period is over, the user's native-staking-proxy +contract allows the user to withdraw the unbonded amounts (i.e. `withdraw_unbonded`). And only then it sends a message to the vault contract, +to release the associated "lien" (Properly, a claim. The implementation uses "lien" indistinctly, for simplicity) (i.e. `release_local_stake`). + +**Release Lien (i.e. `release_cross_stake`)** + +Remote unstaking is initiated by the user on the external-staking contract (i.e. external-staking `unstake` handler). +This is because unstaking requires an unbonding period to be enforced. Only after that unbonding period is over, the external-staking +contract allows the user to withdraw the unbonded amounts (i.e. `withdraw_unbonded`). And only then it sends a message to the vault contract, +to release the associated lien (i.e. `release_cross_stake`). + +**Commit Tx (i.e. `commit_tx`)** +Though this is a public handler, it is only meant to be called by external-staking contracts. This finalises the remote staking process +successfully, and updates the vault state accordingly. -TODO +**Rollback Tx (i.e. `rollback_tx`)** +Though this is a public handler, it is only meant to be called by external-staking contracts. This aborts the remote staking process +in case of error, and rollbacks the vault state accordingly. **Slash** -TODO +TODO: Slashing is not part of MVP, and will be implemented in a future version of mesh-security. - Increase Slashing(user, lien_holder)? -TODO +TODO: Increase slashing is not part of MVP, and will be implemented in a future version of mesh-security. ## Footnotes From b703c8d90392b4b102f65a5b9db733c4f541c942 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Wed, 5 Jul 2023 09:31:51 +0200 Subject: [PATCH 08/43] Update Local Staking Transitions --- docs/provider/LocalStaking.md | 74 +++++++++++++++++++++++++++++++++-- 1 file changed, 70 insertions(+), 4 deletions(-) diff --git a/docs/provider/LocalStaking.md b/docs/provider/LocalStaking.md index 95d2787e..35928222 100644 --- a/docs/provider/LocalStaking.md +++ b/docs/provider/LocalStaking.md @@ -1,7 +1,73 @@ -# Local Staking +# Local Staking (i.e. Native Staking) -A local staking contract connects to a [Vault](./Vault.md). Unlike external stakers, it actually -accepts the native token along with the lien. It manages staking the vault tokens to the native +A local / native staking contract connects to a [Vault](./Vault.md). Unlike external stakers, it actually +accepts the native token along with a claim on it. It manages staking the vault tokens to the native protocol and returns them when finished unbonding. -**TODO** flesh this out +# Description + +Native staking is composed of two contracts: a staking contract and a staking proxy contract, +which is instantiated on behalf of each user. This is so to give each user the ability to +manage their own funds, and perform actions associated with them (i.e. unstaking, voting, etc). + +# Transitions + +## Native Staking Contract + +**Stake (i.e. `receive_stake`)** + +Receives stake (`info.funds`) from the vault contract on behalf of the user, and performs the action +specified in the accompanying `msg`. +`msg` is custom to each implementation of the staking contract, and opaque to the vault. +Typically, it will be a `StakeMsg` message containing the validator to stake to. + +This will be staked to the native protocol of the blockchain (through a native-staking-proxy contract), +and the vault will have a claim on the staked tokens. + +**Unstake (i.e. `release_proxy_stake`)** + +This accepts tokens sent back from the native-staking-proxy contract (through `info.funds`). +The native-staking contract can determine which user they belong to via an internal map. +It will then send those tokens back to the vault, and release the associated claim. + +## Native Staking Proxy Contract + +**Stake (i.e. `stake`)** + +Stakes the tokens from `info.funds` to the given validator. Can only be called by the parent contract, +i.e. the native-staking contract. + +This performs the actual staking action, and holds the user's funds. + +**Restake (i.e. `restake`)** + +Re-stakes the given amount from the one validator to another on behalf of the calling user. +Returns an error if the user doesn't have enough stake. + +**Vote (i.e. `vote`)** + +Vote with the user's stake (over all delegations). + +**Weighted Vote (i.e. `vote_weighted`)** + +Vote with the user's stake (over all delegations), +with a given weight. + +**Withdraw Rewards (i.e. `withdraw_rewards`)** + +If the caller has any delegations, withdraw all rewards from those delegations and +send the tokens to the calling user. + +**Unstake (i.e. `unstake`)** + +Unstakes the given amount from the given validator on behalf of the calling user. +Returns an error if the user doesn't have such a stake. + +After the unbonding period, it will allow the user to claim the tokens (returning +them to the vault). + +**Release Unbonded (i.e. `release_unbonded`)** + +Releases any tokens that have fully unbonded from a previous `unstake`. +The funds will go back to the parent (the native-staking contract) via `release_proxy_stake`. +Errors if the proxy doesn't have any liquid tokens. From e25bf9f23fe2cb26167c2cdfb16cc52b5f4dc2c8 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Wed, 5 Jul 2023 10:06:06 +0200 Subject: [PATCH 09/43] Update External Staking transitions / API --- .../provider/external-staking/src/contract.rs | 2 +- .../provider/external-staking/src/msg.rs | 4 +-- docs/provider/ExternalStaking.md | 30 ++++++++++++++++++- 3 files changed, 32 insertions(+), 4 deletions(-) diff --git a/contracts/provider/external-staking/src/contract.rs b/contracts/provider/external-staking/src/contract.rs index 7f7716fe..b1852962 100644 --- a/contracts/provider/external-staking/src/contract.rs +++ b/contracts/provider/external-staking/src/contract.rs @@ -512,7 +512,7 @@ impl ExternalStakingContract<'_> { } } - /// Withdraws all released tokens to the sender. + /// Withdraws all of their released tokens to the calling user. /// /// Tokens to be claimed have to be unbond before by calling the `unbond` message, and /// their unbonding period must have passed. diff --git a/contracts/provider/external-staking/src/msg.rs b/contracts/provider/external-staking/src/msg.rs index e8479606..f8017599 100644 --- a/contracts/provider/external-staking/src/msg.rs +++ b/contracts/provider/external-staking/src/msg.rs @@ -65,13 +65,13 @@ pub struct StakeInfo { pub stake: Uint128, } -/// Aggregated mutiple stakes response +/// Aggregated multiple stakes response #[cw_serde] pub struct StakesResponse { pub stakes: Vec, } -/// Message to be send as `msg` field on `receive_virtual_staking` +/// Message to be sent as `msg` field on `receive_virtual_staking` #[cw_serde] pub struct ReceiveVirtualStake { pub validator: String, diff --git a/docs/provider/ExternalStaking.md b/docs/provider/ExternalStaking.md index 41fa80ee..f0e10318 100644 --- a/docs/provider/ExternalStaking.md +++ b/docs/provider/ExternalStaking.md @@ -7,4 +7,32 @@ and releases the virtual tokens when finished unbonding. It also manages distribution of rewards coming from the remote (consumer) chain, and transferring of those rewards to remote recipients on the consumer chain. -**TODO** flesh this out +# Transitions + +**Stake (i.e. `receive_virtual_stake`)** + +Receives virtual stake from the vault contract on behalf of the user, and performs the action +specified in the accompanying `msg`. +`msg` is custom to each implementation of the external staking contract, and opaque to the vault. +Typically, it will be a `ReceiveVirtualStakeMsg` message containing the remote validator to stake to. + +This will be staked to the native protocol of the remote/external blockchain. +The vault will hold a lien on the remotely staked tokens, which allows for +multiple remote staking of the same funds. + +**Unstake (i.e. `unstake`)** + +Schedules tokens for release, adding them to the pending unbonds. After the +unbonding period passes, funds are ready to be released, which is accomplished +with a `withdraw_unbonded` call by the user. + +**Withdraw Unbonded (i.e. `withdraw_unbonded`)** + +Withdraws all released tokens to the calling user. + +Tokens to be claimed have to be unbond before, by calling the `unstake` message and +waiting for the unbonding period. + +**Withdraw Rewards (i.e. `withdraw_rewards`)** + +Withdraws the rewards that are the result of staking via a given external validator. From 5008f5c12d73b988cba88012aebc200a0411e184 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Wed, 5 Jul 2023 10:14:32 +0200 Subject: [PATCH 10/43] Improve Consumer syntax --- docs/consumer/Consumer.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index 6cd8f02f..1982ecb3 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -5,7 +5,7 @@ some trusted providers and uses those to update the local staking weights. It then provides rewards to those providers in exchange for the security promise it receives. -This is the other half of [the provider flow](../provider/Provider.md) +This is the other half of [the provider flow](../provider/Provider.md). ```mermaid flowchart LR @@ -38,17 +38,17 @@ This first is convert the token based on a price oracle. The second step is to a which captures both the volatility of the remote asset, as well as a general preference for local/native staking. -This is described [more in depth under Converter](./Converter.md#staking-flow). +This is described more in depth under [Converter](./Converter.md#staking-flow). ## Virtual Staking Once the Converter has normalized the foreign stake into the native staking units, -it calls into the associated ["Virtual Staking" contract](./VirtualStaking.md) in order -to convert this into real stake without owning actual tokens. This contract must be -authorized to have extra power in a native Cosmos SDK module to do this impact, and has +it calls into the associated [VirtualStaking](./VirtualStaking.md) contract in order +to convert this into real stake, without owning actual tokens. This contract must be +authorized to have extra power in a native Cosmos SDK module to have this impact, and has a total limit of tokens it can stake. It performs 3 high level tasks: -- Staking "virtual tokens" as requested by the Converter (up to a limit) -- Periodically withdrawing rewards and sending them to the Converter +- Staking "virtual tokens" as requested by the Converter (up to a limit). +- Periodically withdrawing rewards and sending them to the Converter. - Unstaking "virtual tokens" as requested by the Converter. This must be immediate and avoid the "7 concurrent unbonding" limit on the `x/staking` module to be properly usable. From 4b778e5c861d61fa821206274d5fefdaf31e000a Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Wed, 5 Jul 2023 13:46:31 +0200 Subject: [PATCH 11/43] Improve Converter syntax --- docs/consumer/Converter.md | 70 +++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index a91aee6b..a3ea9a77 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -9,27 +9,29 @@ The converter is connected to the Provider chain via IBC and handles the various ## Setup +TODO: Update this. + When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Stake Converter on the consumer chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this connection is established, Consumer governance can authorize this Stake Converter with some ability to mint -on the ["Virtual Staking" contract](./VirtualStaking.md). +on the [Virtual Staking](./VirtualStaking.md) contract. When we deploy the Stake Converter, we must also configure the address of the -["Virtual Staking" contract](./VirtualStaking.md) that it will use to stake tokens. +[Virtual Staking](./VirtualStaking.md) contract that it will use to stake tokens. In addition, we must define a price feed on setup. (see [Price Normalization / Price Feeds](#price-feeds) below) ## Staking Flow -Once the connection is established, the provider can send various "virtual stake" messages to the converter, -which is responsible for processing them and normalizing for the local "virtual staking" module. These -packets are sent via a dedicated channel between the provider chain and the consumer chain to ensure -there are no other security assumptions (3rd party modules) involved in sending this critical staking +Once the connection is established, the Provider can send various "virtual stake" messages to the Converter, +which is responsible for processing them and normalizing amounts for the local "virtual staking" module. These +packets are sent via a dedicated channel between the Provider chain and the Consumer chain, to ensure +that there are no other security assumptions (3rd party modules) involved in sending this critical staking info. -By itself, a Converter cannot impact the local staking system. It must connect to the [Virtual Staking system](./VirtualStaking.md), -which will convert the "virtual stake" into actual stake in the dPoS system, and return the rewards as well. This -document focuses on the flow from IBC packets to the virtual stake. +By itself, a Converter cannot impact the local staking system. It must connect to the [Virtual Staking](./VirtualStaking.md) +contract, which will convert the "virtual stake" into actual stake in the dPoS system, and return the rewards as well. +This document focuses on the flow from IBC packets to the virtual stake. ### Price normalization @@ -38,26 +40,26 @@ local staking tokens. The first step is simply doing a price conversion. This is done via a [Price Feed](#price-feeds), which is defined on setup and can call into arbitrary logic depending on the chain. (For example, -if we are sent 1000 JUNO, we convert to 1200 OSMO based on some price feed) +if we are sent 1000 JUNO, we convert to 1200 OSMO based on some price feed). The second step is to apply a discount. This discount reduces the value of the cross-stake to a value below what we would get from the pure currency conversion above. This has two purposes: the first is to provide a margin of error when the price deviates far from the TWAP, so -the cross-stake is not overvalued above native staking; the second is to encourage local staking over remote staking. Looking at the +that the cross-stake is not overvalued above native staking; the second is to encourage local staking over remote staking. Looking at the asset's historical volatility can provide a good estimate for the first step, as a floor for minimum discount. Beyond that, consumer chain tokenomics and governance design is free to increase the discount as they feel beneficial. -In this case, let's assume a discount of 40%. A user on the provider chain cross-stakes 100 PROV. We end up with a weight of +In this case, let's assume a discount of 40%. A user on the Provider chain cross-stakes 100 PROV. We end up with a weight of `100 PROV * 18 CONS/PROV * (1 - 0.4) = 1080 CONS` -Thus this cross-stake will trigger the converter to request the virtual staking module to stake 1080 CONS. +Thus, this cross-stake will trigger the Converter to request the virtual staking module to stake 1080 CONS. The discount is stored in the Converter contract and can only be updated by the admin (on-chain governance). -**Important** When we calculate the virtual stake (eg 1080 CONS in the example above), those +**Important** When we calculate the virtual stake (e.g. 1080 CONS in the example above), those tokens will be staked as if they were native CONS tokens. They have the same influence on the validator's voting power, and will receive the same rewards. The only difference is that they -can never be withdrawn and slashing is managed remotely on the Provider chain. +can never be withdrawn, and slashing is managed remotely on the Provider chain. ### Price Feeds @@ -66,56 +68,60 @@ trustable price feed. Since this logic may be chain dependent, we don't want to contract, but rather allow chains to plug in their custom price feed without modifying any of the complex logic required to properly stake. -There are many possible price feed implementations, a few of the main ones we consider are: +There are many possible price feed implementations. A few of the main ones we consider are: **Gov-defined feed.** This is a simple contract that stores a constant price value, which is always -returns when asked for the price. On-chain governance can send a vote to update this price -value when needed. **This is good for mocks, or new chaina with no solid price feed and wanting a stable peg** +returned when asked for the price. On-chain governance can send a vote to update this price +value when needed. **This is good for mocks, or a new chain with no solid price feed, +and wanting a stable peg**. -**Local Oracle** If there is a DEX on the consumer chain with sufficient liquidity and volume +**Local Oracle** If there is a DEX on the Consumer chain with sufficient liquidity and volume on this asset pair (local staking - remote staking), then we can use that for a price feed. Assuming it keeps a proper TWAP oracle on the pair, we sample this every day and can get the average price over the last day, which is quite hard to manipulate for such a long time. -**This is good for an established chain with solid DEX infrastructure, like Osmosis or Juno** +**This is good for an established chain with solid DEX infrastructure, like Osmosis or Juno**. **Remote Oracle** More dynamic than the gov-defined feed, but less secure than the local Oracle, we can do an IBC query on a DEX on another chain to find the price feed. This works like the -Local Oracle, except the DEX being queries lives on eg. Osmosis. Note that it introduces another -security dependency, as if the DEX chain goes Byzantine, it could impact the security of the consumer +Local Oracle, except the DEX being queried lives on e.g. Osmosis. Note that it introduces another +security dependency, as if the DEX chain goes Byzantine, it could impact the security of the Consumer chain. **This is a better option if the local staking token has a liquid market, but there is -no established DEX on the chain itself (like Stargaze).** +no established DEX on the chain itself (like Stargaze)**. The actual logic giving the price feed is located in an Oracle contract (configured upon init). -We recommend using an (eg daily) TWAP on a DEX with good liquidity - ideally on the consumer chain, but this implementation is left up -to the particular chain it is being deployed on. With this TWAP we convert eg. 1 PROV to 18 CONS. +We recommend using an (e.g. daily) TWAP on a DEX with good liquidity - ideally on the consumer chain, +but this implementation is left up to the particular chain it is being deployed on. + +With this TWAP we convert e.g. 1 PROV to 18 CONS. ### Virtual Staking -Each Converter is connected 1:1 with a [Virtual Staking Contract](./VirtualStaking.md). This contract +Each Converter is connected 1:1 with a [Virtual Staking](./VirtualStaking.md) contract. This contract manages the stake and has limited permissions to [call into a native SDK module](./GoModule.md) to mint "virtual tokens" and stake them, as well as immediately unbonding them. The contract ensures the delegations are properly distributed. -The Converter simply tells the virtual staking contract it wishes to bond/unbond N tokens +The Converter simply tells the Virtual Staking contract it wishes to bond/unbond N tokens and that contract manages all minting of tokens and distribution among multiple validators. -We dig more into the mechanics of the virtual staking contract in the +We dig more into the mechanics of the Virtual Staking contract in the [Virtual Staking](./VirtualStaking.md) document. ## Rewards Flow -Once per epoch, the virtual staking module will trigger rewards. This will send a number of +Once per epoch, the Virtual Staking module will trigger rewards. This will send a number of messages to the Converter, specifying which validators the rewards belong to, along with the native reward tokens themselves. +TODO: Update this. The Converter will then [transfer all these tokens via ICS20](../ibc/Overview.md) to the corresponding `External Staking` contract on the Provider chain, and send a message over the standard IBC channel to inform the `External Staking` contract how to distribute them. -(If we get callbacks on ics20, we send the metadata only after tokens have arrived. Until then -(for MVP), we send them concurrently and hope) +(If we get callbacks on ICS20, we send the metadata only after tokens have arrived. Until then +(for MVP), we send them concurrently and hope). ## Unstaking Flow The Converter can also unstake some tokens. These will be held in escrow on the Provider and are susceptible to slashing upon proper evidence submission. Since the virtual stake is, well, "virtual" and slashing has no impact, the delegation numbers can be immediately reduced -on the consumer's native staking module. +on the Consumer's native staking module. From 0dbefc7ea11480107f6993930d56c97651bee3d4 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Wed, 5 Jul 2023 14:00:28 +0200 Subject: [PATCH 12/43] Update Converter Setup section --- docs/consumer/Converter.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index a3ea9a77..11a7b524 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -9,18 +9,21 @@ The converter is connected to the Provider chain via IBC and handles the various ## Setup -TODO: Update this. +We must first instantiate a Price Feed contract(see [Price Normalization / Price Feeds](#price-feeds) below), +then the Converter contract. +The Converter will then instantiate a Virtual Staking contract to work with it, +as they both need references to each other. +In addition to a reference to the Price Feed contract and the code id of the Virtual Staking contract, +the Converter also needs the discount ratio, the remote denomination, and the admin of the Virtual Staking +contract, which is taken as an explicit argument, and normally will be the same admin of the Converter. + +TODO: Update IBC Setup -When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Stake Converter on the consumer +When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Converter on the Consumer chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this -connection is established, Consumer governance can authorize this Stake Converter with some ability to mint +connection is established, Consumer governance can authorize this Converter with some ability to mint on the [Virtual Staking](./VirtualStaking.md) contract. -When we deploy the Stake Converter, we must also configure the address of the -[Virtual Staking](./VirtualStaking.md) contract that it will use to stake tokens. -In addition, we must define a price feed on setup. -(see [Price Normalization / Price Feeds](#price-feeds) below) - ## Staking Flow Once the connection is established, the Provider can send various "virtual stake" messages to the Converter, From c95c174660eef13fdb50b64eb346e799c00764ad Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Thu, 6 Jul 2023 13:16:08 +0200 Subject: [PATCH 13/43] Update Converted doc --- docs/consumer/Converter.md | 33 +++++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index 11a7b524..e20c9209 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -1,4 +1,4 @@ -# Stake Converter +# Converter Overview The Stake Converter is on the consumer side and is connected to an External Staker on the Provider side. This handles the normalization of the external tokens and _converts_ them into "Virtual Stake". @@ -10,20 +10,37 @@ The converter is connected to the Provider chain via IBC and handles the various ## Setup We must first instantiate a Price Feed contract(see [Price Normalization / Price Feeds](#price-feeds) below), -then the Converter contract. -The Converter will then instantiate a Virtual Staking contract to work with it, -as they both need references to each other. -In addition to a reference to the Price Feed contract and the code id of the Virtual Staking contract, -the Converter also needs the discount ratio, the remote denomination, and the admin of the Virtual Staking -contract, which is taken as an explicit argument, and normally will be the same admin of the Converter. +and a Virtual Staking contract must be stored on chain. So that we can get its code id. -TODO: Update IBC Setup +Then the Converter contract is instantiated, which takes the address of the Price Feed contract, and the Code Id +of the Virtual Staking contract. +The Converter will then instantiate a Virtual Staking contract to work with it, as they both need references +to each other. +In addition to the Price Feed contract address and the code id of the Virtual Staking contract, +the Converter also needs the discount ratio, the remote denomination, and the admin of the Virtual Staking +contract, which is taken as an explicit argument, and normally will be the same admin of the Converter (but +could be a different one). This (wasm) admin is very important, as it is the only one who can migrate the Virtual +Staking contract if needed. When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Converter on the Consumer chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this connection is established, Consumer governance can authorize this Converter with some ability to mint on the [Virtual Staking](./VirtualStaking.md) contract. +When IBC connection is established from the Converter, the Provider side [External Staking](../provider/ExternalStaking.md) +contract must be already instantiated with the proper IBC channel information (i.e. proper connection id +and port id information in the `AuthorizedEndpoint` struct, set as part of their `InstantiateMsg`). +See the [Provider](../provider/Provider.md) Setup for more information. + +## Validator Updates Flow + +The Converter contract on the Provider chain will send validator updates to the Consumer chain via IBC packets, +so that the External Staking contract on the Provider can know if a given remote validator is active or not. + +These packets are sent on IBC connection establishment. +TODO: Send validator updates dynamically, so that the Provider chain is kept up-to-date with the validator set on +the Consumer chain. + ## Staking Flow Once the connection is established, the Provider can send various "virtual stake" messages to the Converter, From 472357d29b558f00f243717a52558edcecb0efc0 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Thu, 6 Jul 2023 13:46:41 +0200 Subject: [PATCH 14/43] Add Provider Setup section --- docs/provider/ExternalStaking.md | 2 +- docs/provider/Provider.md | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/provider/ExternalStaking.md b/docs/provider/ExternalStaking.md index f0e10318..ffbcb1c7 100644 --- a/docs/provider/ExternalStaking.md +++ b/docs/provider/ExternalStaking.md @@ -7,7 +7,7 @@ and releases the virtual tokens when finished unbonding. It also manages distribution of rewards coming from the remote (consumer) chain, and transferring of those rewards to remote recipients on the consumer chain. -# Transitions +## Transitions **Stake (i.e. `receive_virtual_stake`)** diff --git a/docs/provider/Provider.md b/docs/provider/Provider.md index 2fd51273..13609938 100644 --- a/docs/provider/Provider.md +++ b/docs/provider/Provider.md @@ -22,6 +22,34 @@ flowchart LR G -. IBC .-> H{{Stars Consumer}}; ``` +## Setup + +## Contracts setup + +After the Consumer side is instantiated(See [Consumer Setup](../consumer/Consumer.md#setup)), +we need to instantiate the contracts on the Provider side. + +The Vault contract is instantiated with the code id of the Local Staking contract, +and a initialization message for it which includes the code id of the Native Staking Proxy contract, +and a max slashing setting for local staking. +It also requires the local denom, which is the denom of the native staking token on the Provider chain. + +After that, the External Staking contract can be instantiated. This also requires the local denom, the +address of the Vault contract, the unbonding period of the remote chain, the rewards denom (which is the +denom of the staking token on the remote chain), and a max slashing setting for external staking. +It also requires remote contact information, which are the IBC connection and port ids for +connecting to the remote (Consumer) chain over IBC. + +### IBC + +After the contracts setup is done, IBC setup needs to be done, again starting from the Consumer side. +See [Consumer IBC](../consumer/Consumer.md#ibc). + +This may involve some test setup, involving by example [ibctesting](https://pkg.go.dev/github.com/incubus-network/fanfury-sdk/ibctesting) +or a similar framework, or a relayer configuration, which would require the External Staking contract address on the Provider and the Converter +address on the Consumer. +The exact setup of IBC is out of scope for this document. + ## Flows The first action the user must undertake is depositing From 0f77b419e5a0e0afcf55e42e1057603207bc820a Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Thu, 6 Jul 2023 13:47:03 +0200 Subject: [PATCH 15/43] Move Consumer Setup section to Consumer.md --- docs/consumer/Consumer.md | 29 +++++++++++++++++++++++++++++ docs/consumer/Converter.md | 25 ------------------------- 2 files changed, 29 insertions(+), 25 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index 1982ecb3..84eb8987 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -27,6 +27,35 @@ flowchart LR B -. IBC .-> K; ``` +## Setup + +### Contracts Setup + +We must first instantiate a Price Feed contract(see [Price Normalization / Price Feeds](./Converter.md#price-feeds)), +and a [Virtual Staking](./VirtualStaking.md) contract must be stored on chain. So that we can get its code id. + +Then the Converter contract is instantiated, which takes the address of the Price Feed contract, and the Code Id +of the Virtual Staking contract. +The Converter will then instantiate a Virtual Staking contract to work with it, as they both need references +to each other. +In addition to the Price Feed contract address and the code id of the Virtual Staking contract, +the Converter also needs the discount ratio, the remote denomination, and the admin of the Virtual Staking +contract, which is taken as an explicit argument, and normally will be the same admin of the Converter (but +could be a different one). This (wasm) admin is very important, as it is the only one who can migrate the Virtual +Staking contract if needed. + +### IBC + +When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Converter on the Consumer +chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this +connection is established, Consumer governance can authorize this Converter with some ability to mint +on the [Virtual Staking](./VirtualStaking.md) contract. + +When IBC connection is established from the Converter, the Provider side [External Staking](../provider/ExternalStaking.md) +contract must be already instantiated with the proper IBC channel information (i.e. proper connection id +and port id information in the `AuthorizedEndpoint` struct, set as part of their `InstantiateMsg`). +See the [Provider](../provider/Provider.md) Setup for more information. + ## Converting Foreign Stake Not all providers are treated equally. (And this is a good thing) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index e20c9209..c131505d 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -7,31 +7,6 @@ which handles the actual issuance. The converter is connected to the Provider chain via IBC and handles the various packets coming from it. -## Setup - -We must first instantiate a Price Feed contract(see [Price Normalization / Price Feeds](#price-feeds) below), -and a Virtual Staking contract must be stored on chain. So that we can get its code id. - -Then the Converter contract is instantiated, which takes the address of the Price Feed contract, and the Code Id -of the Virtual Staking contract. -The Converter will then instantiate a Virtual Staking contract to work with it, as they both need references -to each other. -In addition to the Price Feed contract address and the code id of the Virtual Staking contract, -the Converter also needs the discount ratio, the remote denomination, and the admin of the Virtual Staking -contract, which is taken as an explicit argument, and normally will be the same admin of the Converter (but -could be a different one). This (wasm) admin is very important, as it is the only one who can migrate the Virtual -Staking contract if needed. - -When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Converter on the Consumer -chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this -connection is established, Consumer governance can authorize this Converter with some ability to mint -on the [Virtual Staking](./VirtualStaking.md) contract. - -When IBC connection is established from the Converter, the Provider side [External Staking](../provider/ExternalStaking.md) -contract must be already instantiated with the proper IBC channel information (i.e. proper connection id -and port id information in the `AuthorizedEndpoint` struct, set as part of their `InstantiateMsg`). -See the [Provider](../provider/Provider.md) Setup for more information. - ## Validator Updates Flow The Converter contract on the Provider chain will send validator updates to the Consumer chain via IBC packets, From f2b5a39ede8bb15127ed1bdc00288aeab5817e3e Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Thu, 6 Jul 2023 14:18:52 +0200 Subject: [PATCH 16/43] Update README (remove TODO) --- docs/README.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/docs/README.md b/docs/README.md index 750a65ed..57686a41 100644 --- a/docs/README.md +++ b/docs/README.md @@ -222,19 +222,18 @@ Below are links to detailed documents on various sub-systems: [Provider](./provider/Provider.md): -- [Vault](./provider/Vault.md) -- [Local Staking](./provider/LocalStaking.md) -- [External Staking](./provider/ExternalStaking.md) -- TODO - Rust interfaces +- [Vault](./provider/Vault.md). +- [Local Staking](./provider/LocalStaking.md). +- [External Staking](./provider/ExternalStaking.md). [Consumer](./consumer/Consumer.md): -- [Converter](./consumer/Converter.md) -- [Virtual Staking](./consumer/VirtualStaking.md) -- SDK Changes +- [Converter](./consumer/Converter.md). +- [Virtual Staking](./consumer/VirtualStaking.md). +- SDK Changes. [IBC Protocol](./ibc/Overview.md): -- [Cross-Chain Staking](./ibc/Staking.md) -- [Reward Flow](./ibc/Rewards.md) -- [Handling Slashing Evidence](./ibc/Slashing.md) +- [Cross-Chain Staking](./ibc/Staking.md). +- [Reward Flow](./ibc/Rewards.md). +- [Handling Slashing Evidence](./ibc/Slashing.md). From ff3e1c412a8bd6eb43db7888bd993d5955e55094 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Thu, 6 Jul 2023 14:30:21 +0200 Subject: [PATCH 17/43] Update Rewards Flow section --- docs/consumer/Converter.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index c131505d..63a8f13a 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -103,16 +103,19 @@ We dig more into the mechanics of the Virtual Staking contract in the ## Rewards Flow -Once per epoch, the Virtual Staking module will trigger rewards. This will send a number of +Once per epoch, the Virtual Staking module will trigger rewards. This will generate a number of messages to the Converter, specifying which validators the rewards belong to, along with the -native reward tokens themselves. - -TODO: Update this. -The Converter will then [transfer all these tokens via ICS20](../ibc/Overview.md) to the -corresponding `External Staking` contract on the Provider chain, and send a message over the -standard IBC channel to inform the `External Staking` contract how to distribute them. -(If we get callbacks on ICS20, we send the metadata only after tokens have arrived. Until then -(for MVP), we send them concurrently and hope). +amounts of rewards. The Converter will send vouchers of the reward amounts to the Exteral Staking contract +over IBC. The actual tokens will be kept on the Converter, for later distribution. + +The External Staking contract on the Consumer chain will receive the vouchers of the amounts per +validator, and will then inform in turn to the Converter of the proper distribution of rewards per user. + +This is done in this way because while the Consumer side knows the rewards per validator, the information +about which delegators are staking with which validator is only known on the Provider side. Thus, +the Provider must inform the Consumer of the proper distribution of rewards. + +The Converter will then send the actual rewards to their respective owners. ## Unstaking Flow From 2bc1bacd87d71468eeb5d3966ea1c071d653a8e4 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Thu, 6 Jul 2023 14:35:42 +0200 Subject: [PATCH 18/43] Add Rebalancing Flow section --- docs/consumer/Converter.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index 63a8f13a..7f52b360 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -117,6 +117,16 @@ the Provider must inform the Consumer of the proper distribution of rewards. The Converter will then send the actual rewards to their respective owners. +## Rebalancing Flow + +Once per epoch, the Virtual Staking module will check if a rebalancing of staking amounts is required. +This can happen once the max staking cap is reached on the Consumer. In this case, the Virtual Staking +module will trigger a rebalancing, which will generate a number of messages for bonding/unbonding +of amounts for each validator. + +TODO: The current implementation does not consider changes to the validator set, and rebalancing may +(repeatedly) fail if any validator was slashed or fell out of the active set. + ## Unstaking Flow The Converter can also unstake some tokens. These will be held in escrow on the Provider and From d48a4d29301e6c76a78b8deed8093571d30bfdbc Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 13:51:31 +0200 Subject: [PATCH 19/43] Update IBC Control Channel doc --- docs/consumer/Consumer.md | 2 + docs/ibc/ControlChannel.md | 75 ++++++++++++++++++++++---------------- 2 files changed, 46 insertions(+), 31 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index 84eb8987..acf5716b 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -56,6 +56,8 @@ contract must be already instantiated with the proper IBC channel information (i and port id information in the `AuthorizedEndpoint` struct, set as part of their `InstantiateMsg`). See the [Provider](../provider/Provider.md) Setup for more information. +Also, see [IBC Deployment](../ibc/ControlChannel.md#deployment) for more information on how the IBC connection is established. + ## Converting Foreign Stake Not all providers are treated equally. (And this is a good thing) diff --git a/docs/ibc/ControlChannel.md b/docs/ibc/ControlChannel.md index 1e3438b0..f96c7b81 100644 --- a/docs/ibc/ControlChannel.md +++ b/docs/ibc/ControlChannel.md @@ -1,14 +1,14 @@ # Control Channel This document describes the "control channel", which is a direct IBC channel between the -`external-staking` contract on the provider side and the `converter` contract -on the consumer side. This is used to send messages about bonding and unbonding, +`external-staking` contract on the Provider side and the `converter` contract +on the Consumer side. This is used to send messages about bonding and unbonding, and any other metadata about the protocol (like validators). -It is **not** used to [send reward tokens](./Rewards.md), which must be sent over -the commonly accepted ICS20 interface, so they are fungible after receipt. +It is also used to send reward vouchers(./Rewards.md), which represent amounts of rewards +that have been earned by delegators from the Provider chain, and are redeemable on the Consumer chain. -It is also **not** used to [handle slashing](./Slashing.md), as there are concerns +It is **not** used to [handle slashing](./Slashing.md), as there are concerns a malicious state machine would lie, so we demand original evidence of Tendermint misbehavior on the provider chain. @@ -28,18 +28,21 @@ digging into the sub-protocols below. ## Validator Metadata Syncing -The provider sends packets for the original validator set and for every update. +The Consumer sends packets for the original validator set. + +TODO: Validator set updates are not yet supported. Only the original validator set is sync +after IBC connection establishment at the moment. We use a CRDT-based algorithm to maintain a consistent view of the validator set regardless of the order the packets received, such that any permutation of the same set of messages will result in the same state on the provider. These operations are `commutative` and `idempotent`. -The [full description of the algorithm](./ValidatorSet.md) is quite lengthy and defined in its own page. +The [full description of the algorithm](./Validators.md) is quite lengthy and defined in its own page. ## Virtual Staking Protocol -The provider sends messages to stake and unstake virtual tokens to various validators -on the consumer chain. This must be done in such a way that it is robust in face +The Provider sends messages to stake and unstake virtual tokens to various validators +on the Consumer chain. This must be done in such a way that it is robust in face of reordering and errors upon an unordered channel. The [full description of the algorithm](./Staking.md) is quite lengthy and defined in its own page. @@ -63,26 +66,36 @@ the converter contract must be instantiated before the `external-staking` contra The channel is initiated by a relayer, with party A being the appropriate `converter` contract on the consumer chain, and party B being the `external-staking` contract. -The general process (assuming a vault is already established on the provider) is: +The general process (assuming a vault is already established on the Provider) is: -1. Instantiate price feed, converter, virtual staking contracts on the consumer chain -2. Instantiate external staking contract on the provider chain (referencing IBC port of the converter) -3. Create IBC channel from provider to consumer -4. Apply to consumer governance to provide a virtual staking max cap to the associated virtual staking contract, so that this connection may have voting power. +1. Instantiate price feed, converter, virtual staking contracts on the Consumer chain. +2. Instantiate external staking contract on the Provider chain (referencing IBC port of the converter). +3. Create IBC channel from Provider to Consumer. +4. Apply to Consumer governance to provide a virtual staking max cap to the associated virtual staking contract, + so that this connection may have voting power. ### Handshake -Opening the channel is a 4 step process. It must be initiated by the consumer side. - -1. Start with `OpenInit` from converter to the (connection, port) of the external staking. The version SHOULD be set to the highest mesh-security version it supports (see below), and the channel ordering MUST be "unordered". It MUST error if it has a previously established channel. -2. The external staking contract receives `OpenTry`. The channel ordering MUST be "unordered", and the version protocol MUST be `mesh-security`. It performs version negotiation as defined below. It MUST error if the (connection, port) being proposed is not the one it was initialized with. It MUST error if it has a previously established channel. -3. The converter receives `OpenAck` and MUST verify the version protocol is `mesh-security`. It MUST verify the version is not higher than the one it proposed, and not lower than the oldest version it supports. If successful, it stores the new channel details locally. -4. The external staking contract receives `OpenConfirm`. Everything has been verified on all sides, and there can be no errors here. It stores the new channel details locally. - -Closing a channel is currently not well defined. It is expected that the channel will remain open, as it is unordered. If the channel is closed, -both sides must mark the channel as closed locally, and error on any attempt to send IBC packets. The channel may be re-opened by repeating -the initial process, with both sides validating the re-open was from the same (connection, port) as the original channel. When that handshake is -completed, they can replace the closed channel from storage with the new open channel. +Opening the channel is a 4-step process. It must be initiated by the Consumer side. + +1. Start with `OpenInit` from converter to the (connection, port) of the external staking. The version + SHOULD be set to the highest mesh-security version it supports (see below), and the channel ordering + MUST be "unordered". It MUST error if it has a previously established channel. +2. The external staking contract receives `OpenTry`. The channel ordering MUST be "unordered", + and the version protocol MUST be `mesh-security`. It performs version negotiation as defined below. + It MUST error if the (connection, port) being proposed is not the one it was initialized with. + It MUST error if it has a previously established channel. +3. The converter receives `OpenAck` and MUST verify the version protocol is `mesh-security`. + It MUST verify the version is not higher than the one it proposed, and not lower than the oldest version + it supports. If successful, it stores the new channel details locally. +4. The external staking contract receives `OpenConfirm`. Everything has been verified on all sides, + and there can be no errors here. It stores the new channel details locally. + +Closing a channel is currently not well-defined. It is expected that the channel will remain open, as it is unordered. +If the channel is closed, both sides must mark the channel as closed locally, and error on any attempt to send IBC packets. +The channel may be re-opened by repeating the initial process, with both sides validating the re-open +was from the same (connection, port) as the original channel. When that handshake is completed, they can replace +the closed channel from storage with the new open channel. ### Version Negotiation @@ -107,7 +120,7 @@ error if the major version is higher than its known versions (for now, anything The consumer should respond with the highest version it supports, but no higher than that proposed by the provider. -At the end, the version is the highest version supported by both sides and they may freely make +At the end, the version is the highest version supported by both sides, and they may freely make use of any features added up to that version. This document describes version `1.0.0` of the protocol, but additions may be added in the future (which must be linked to from this section). @@ -115,18 +128,18 @@ the protocol, but additions may be added in the future (which must be linked to Note the entire protocol is designed around syncing an initial state and sending a stream of diffs, such that `State = Initial + Sum(Diffs)`. This applies to both the validator set -as well as the total amounts staked. +and the total amounts staked. If we reorder the diffs, it is possible to get a different result, so we need to be careful about relying on Unordered channels. Imagine Stake 100 and Unstake 50. If Unstake goes first, it would return an error, yet the Stake would apply properly, leaving a total of 100 not 50. Furthermore, if a packet is dropped, and the diff is not applied, the two sides will -get out of sync, where eg the Provider believes there is 500k staked to a given validator, +get out of sync, where e.g. the Provider believes there is 500k staked to a given validator, while the Consumer believes there is 400k. At the same time, Ordered channels are fragile, in that a single timed-out or undeliverable packet will render the channel useless forever. We must make sure to use extremely large timeouts (say 7 days) to handle the case of a prolonged chain halt. We must also ensure that the -receiving contract always returns Acks with errors on failure, and never panics. -A contract panic will abort the tx containing the IbcPacketReceiveMsg as of wasmd 0.40 -(MSV for Mesh Security) +receiving contract always returns ACKs with errors on failure, and never panics. +A contract panic will abort the tx containing the IbcPacketReceiveMsg, as of wasmd 0.40 +(MSV for Mesh Security). From 500cce3de4327eba326f66b8f0430b2f13241193 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 13:52:08 +0200 Subject: [PATCH 20/43] Update converter IBC comments --- contracts/consumer/converter/src/ibc.rs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/contracts/consumer/converter/src/ibc.rs b/contracts/consumer/converter/src/ibc.rs index 2537de2d..f9bb6e8c 100644 --- a/contracts/consumer/converter/src/ibc.rs +++ b/contracts/consumer/converter/src/ibc.rs @@ -30,14 +30,14 @@ const DEFAULT_VALIDATOR_TIMEOUT: u64 = 24 * 60 * 60; const DEFAULT_REWARD_TIMEOUT: u64 = 60 * 60; pub fn packet_timeout_validator(env: &Env) -> IbcTimeout { - // No idea about their blocktime, but 24 hours ahead of our view of the clock + // No idea about their block time, but 24 hours ahead of our view of the clock // should be decently in the future. let timeout = env.block.time.plus_seconds(DEFAULT_VALIDATOR_TIMEOUT); IbcTimeout::with_timestamp(timeout) } pub fn packet_timeout_rewards(env: &Env) -> IbcTimeout { - // No idea about their blocktime, but 1 hour ahead of our view of the clock + // No idea about their block time, but 1 hour ahead of our view of the clock // should be decently in the future. let timeout = env.block.time.plus_seconds(DEFAULT_REWARD_TIMEOUT); IbcTimeout::with_timestamp(timeout) @@ -196,7 +196,7 @@ pub fn ibc_packet_receive( } #[cfg_attr(not(feature = "library"), entry_point)] -/// We get acks on sync state without much to do. +/// We get ACKs on sync state without much to do. /// If it succeeded, take no action. If it errored, we can't do anything else and let it go. /// We just log the error cases so they can be detected. pub fn ibc_packet_ack( From eb9ed3275de78fc7673dfddff03b80276f6bd7c7 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 14:03:06 +0200 Subject: [PATCH 21/43] Update IBC Overview doc --- docs/ibc/Overview.md | 51 ++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/docs/ibc/Overview.md b/docs/ibc/Overview.md index 773b1596..87b05ca4 100644 --- a/docs/ibc/Overview.md +++ b/docs/ibc/Overview.md @@ -9,7 +9,7 @@ we abstracted away when focusing on [Providers](../provider/Provider.md) and [Consumers](../consumer/Consumer.md). Fundamentally there are these two flows, although slashing -is also done cross chain, but outside of IBC (by external observers). +is also done cross chain, but outside IBC (by external observers). For the time being, we focus on the two IBC data flows and discuss cross-chain slashing in another section. @@ -27,8 +27,8 @@ As mentioned in [the consumer section](../consumer/Consumer.md), we need to establish a trust relationship between the Provider side and the Consumer side. This is done in multiple stages. I refer to "Admin" here in most of the setup, which can be any address that can make calls on two chains. -It is responsible for proper configuration, but revokes all it's power before the provider-consumer pairing has any -voting power (permitted to virtually stake). It can be a normal account, a ledger, a cosmos sdk native multisig, +It is responsible for proper configuration, but revokes all its power before the Provider-Consumer pairing has any +voting power (permitted to virtually stake). It can be a normal account, a ledger, a Cosmos SDK native multisig, a DAO using ICA to act on two chains. In theory this could be on-chain governance, but that would be unwieldy. I recommend using the approach of a single actor deploying (for simplicity), then using governance to authorize @@ -36,28 +36,30 @@ the connection once it is all configured (for security). Establish a channel (allow list): -2. Admin deploys _Converter_ and _Virtual Staking_ on Consumer side, both referencing each other. -3. Admin deploys _External Staker_ on Provider side, registering `(connection, port)` from _Converter_ contract from (1) -4. Admin updates _Converter_ (1) to allow the `(connection, port)` from (2) -5. Anyone establishes a new channel between _Converter_ and _External Staker_. - Both contracts ensure the other side matches the stored `(connection, port)` and refused the channel otherwise. -6. Neither contract accept any more channels. They only accept one channel and only what matches their config. +1. Admin deploys _Converter_ and _Virtual Staking_ on the Consumer side, both referencing each other. +2. Admin deploys _External Staker_ on the Provider side, registering `(connection, port)` + from _Converter_ contract from (1). +3. Admin updates _Converter_ (1) to allow the `(connection, port)` from (2). +4. Anyone establishes a new channel between _Converter_ and _External Staker_. + Both contracts ensure the other side matches the stored `(connection, port)` and refuse the channel otherwise. +5. Neither contract accept any more channels. They only accept one channel and only what matches their config. Now that we have a channel and know which contract is talking to whom, we need to authorize them: -1. Admin removes their admin rights on both contracts (or passes to chain governance) -2. Due diligence performed on configuration of both contracts and the channel. Parties from both chains can verify the contract code and configuration. +1. Admin removes their admin rights on both contracts (or passes to chain governance). +2. Due diligence performed on configuration of both contracts and the channel. Parties from both chains + can verify the contract code and configuration. 3. Consumer chain governance votes to authorize this _Virtual Staking_ contract to have special privileges on the staking system (see [Virtual Staking](../consumer/VirtualStaking.md)). 4. Authorization on the Provider chain [is not required](https://github.com/CosmWasm/mesh-security/blob/begin-architecture/docs/provider/Vault.md#design-decisions), but the default cross-stake frontend application should add the _External Staker_ to the recommended list. -Once this has been completed, everything is set up and the token holders on the provider side -can now safely cross-stake on the consumer side with one click. +Once this has been completed, everything is set up and the token holders on the Provider side +can now safely cross-stake on the Consumer side with one click. Note that the majority of the setup is permissionless. And it just requires one governance vote on the -consumer side to authorize ability to stake virtual tokens, which is essential for any security guarantees. +Consumer side to authorize the ability to stake virtual tokens, which is essential for any security guarantees. No forks, no complicated processes... just one proposal. ## Protocol design @@ -69,23 +71,20 @@ principle form of communication, called the [Control Channel](./ControlChannel.md), and is the core piece of the Mesh Security IBC protocol. -We send reward tokens over a standard ics20 channel so that -they are fungible with the native staking token currently sent -between the chains (it will have the same denom as what's listed on the DEX). -Besides moving the rewards, we need to inform the _External Staker_ that -it has received rewards so it can distribute them to validators. -We will do that via ICA, or by making use of the ICS20 memo field to pass this info. -Read more about the [Cross-Chain Rewards Protocol](./Rewards.md) -in more depth. +We send reward vouchers over the Control Channel from the converter contract +in the Consumer side to the External Staking contract on the Provider side, so that +they can be distributed by the Provider side. The Provider side +indicates in turn to the Consumer side which users should receive rewards. +Read more about the [Cross-Chain Rewards Protocol](./Rewards.md). -Finally, slashing is not in-protocol, so a malicious consumer chain +Finally, slashing is not in-protocol, so a malicious Consumer chain cannot slash an honest delegator. Rather, an external observer must -submit evidence of a double sign on the consumer chain directly to -the provider chain. With that direct proof, it will take action to slash +submit evidence of a double sign on the Consumer chain directly to +the Provider chain. With that direct proof, it will take action to slash the appropriate stake. We explain this design more in "Trust Assumptions" below. Read more about the mechanics of the [Slashing Evidence Handling](./Slashing.md). ## Trust Assumptions The [top-level introduction covers many concerns](../README.md#common-concerns) -about possible exploits in the trust relationship between consumer and provider. +about possible exploits in the trust relationship between Consumer and Provider. From 5efd348c33e7be6cd17eeaf1b0bc43a7d2664d93 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 14:07:33 +0200 Subject: [PATCH 22/43] Improve Serializability doc syntax --- docs/ibc/Serializability.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/ibc/Serializability.md b/docs/ibc/Serializability.md index e1ac4ded..fc69d9a9 100644 --- a/docs/ibc/Serializability.md +++ b/docs/ibc/Serializability.md @@ -31,7 +31,7 @@ It goes on and explains Continuing with an example of [such a case that you can read](https://www.postgresql.org/docs/current/transaction-iso.html#XACT-SERIALIZABLE). -The key note here is that, in the interest of performance, the database _does not_ actually run them +The key point here is that, in the interest of performance, the database _does not_ actually run them serially, not use locking to ensure key portions are run serially. Instead, it detects any conflict, and if present, aborts the transaction with an error (to be retried later). Such a definition may be useful when speeding up local changes, but it is not possible when the transaction consists of sub-transactions on different blockchains, as we cannot roll them both back on failure. @@ -68,7 +68,7 @@ locks, or special handling of the packet and ack ordering. One way to guarantee is to create a [data structure that is a CRDT](#crdts) (link to section below). Often that is not possible (we have some global invariants that may never be broken), and we need to look further. But if it is possible to use CRDTs, you can always guarantee serializability, without -the need for locks or any other book-keeping. +the need for locks or any other bookkeeping. Note that Incrementing and Decrementing a counter is a good example of a commutative operation, but once we start adding some global invariants, like "counter must never go below 0", this is @@ -107,7 +107,7 @@ significant communication between the nodes, and is not suitable for an IBC syst For our purpose, we can look at the basic requirements of [Atomic Commits](https://en.m.wikipedia.org/wiki/Atomic_commit): -> In the field of computer science, an atomic commit is an operation that applies a set of distinct changes as a single operation. If the changes are applied, then the atomic commit is said to have succeeded. If there is a failure before the atomic commit can be completed, then all of the changes completed in the atomic commit are reversed. This ensures that the system is always left in a consistent state +> In the field of computer science, an atomic commit is an operation that applies a set of distinct changes as a single operation. If the changes are applied, then the atomic commit is said to have succeeded. If there is a failure before the atomic commit can be completed, then all the changes completed in the atomic commit are reversed. This ensures that the system is always left in a consistent state This can be implemented in a two-chain IBC protocol with the following approach: @@ -146,7 +146,7 @@ that fails depends on the ordering of the operations, thus rendering it no longe and there is a limit to the value of the counter (not user defined), then we can prove we will never hit the said limits and this would be commutative. However, since we usually enforce `value > 0` on blockchains, this would rarely work. -Other types that are well defined and may be useful to IBC protocols are "grow-only set", "two-phase-set" (once removed, it can never enter), +Other types that are well-defined and may be useful to IBC protocols are "grow-only set", "two-phase-set" (once removed, it can never enter), "last write wins" (based on some trusted timestamp). These are mathematical definitions and can be implemented in a variety of ways. For example "grow-only set" could be an "append-only vector" where we keep it sorted and remove duplicates. @@ -162,10 +162,10 @@ Note that commit ordering is essential here. If we start transaction A, B, C on concurrently, we still want to treat them as if they were committed in order. That all of A's processing is done before B starts. If they are completely independent and don't touch the same data, then they can safely run concurrently. If they depend on (or would interfere with) each other, then we must fail the later transactions before sending an IBC packet. -It can be retried after A is committed and we can safely determine the result. +It can be retried after A is committed, and we can safely determine the result. -Note this extends both to the order of processing of A', B', C' on the receiving chain, as well as the order of ACKs -arriving on the sending chain. On top of this, we have to ensure that no other transactions conflict with any open +Note this extends both to the order of processing of A', B', C' on the receiving chain, and the order of ACKs +arriving at the sending chain. On top of this, we have to ensure that no other transactions conflict with any open IBC transactions. Transaction A is "open" from the time the first logic is run on the sending chain (which will send the IBC message) until the ACK is fully processed and committed on the sending chain. This will span several blocks, possibly hours in the case of timeouts. @@ -198,7 +198,7 @@ In some cases where the business logic is very complex and hard to model commuta sub-system (nothing universal like bank), this may be the best approach. However, it is very limiting and should be avoided if possible. **Note** Unlike a typical DB, we cannot wait on a lock, as a transaction must be processed sequentially (and quickly). -Instead we return an error on any transaction that cannot acquire a lock, pushing the responsibility to the client to retry later. +Instead, we return an error on any transaction that cannot acquire a lock, pushing the responsibility to the client to retry later. This also ensures we can never get deadlock (just very poor performance if we lock too much). ## Forcing Commutativity @@ -222,10 +222,10 @@ We can say that ICS20 implementation does something like this. As the only invar it preemptively moves the tokens out of the users account to an escrow. This doesn't require any further lock on the user's account, but ensures no other transaction will be possible to execute that would render the user's balance below 0 if this was committed. Thus, any other transaction that would be commutative with this (commit or rollback) can be executed concurrently, but any other -transaction that would conflict with this (eg. spend the same tokens, and only be valid if the ICS20 transfer gets an error ack), +transaction that would conflict with this (e.g. spend the same tokens, and only be valid if the ICS20 transfer gets an error ack), will immediately fail. -ICS20 is an extremely simple case and you don't need such theory to describe decrementing one counter. However, assume there was not +ICS20 is an extremely simple case, and you don't need such theory to describe decrementing one counter. However, assume there was not only a min token balance (0) but some max, say 1 million. Then ICS20 would not work, as you could escrow 500k tokens, then receive 600k tokens, and if the ICS20 received an error ACK, it would be impossible to validly roll this back (another example of non-commutative, or conflicting, operations). @@ -250,7 +250,7 @@ With some clever reasoning, we could possibly enforce such value ranges without After discussions with other developers, we feel that Value Ranges could be a very valuable approach, offering the same guarantees of commutativity as locking, but with much less impact on the user's experience. It doesn't require developers to reason about every workflow, but rather, like the locking approach, enforces constraints in the data structures themselves. -This is much less error prone, and the same data structures that would be affected by locking are the same ones that would +This is much less error-prone, and the same data structures that would be affected by locking are the same ones that would be affected by value ranges. We will focus on Locking for MVP and look forward to develop this further for the V1 release, with plenty of time to discuss the From 47357f57f2631a6ddfef02c753aa11a34ba227e5 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 14:09:10 +0200 Subject: [PATCH 23/43] Improve Slashing doc syntax --- docs/ibc/Slashing.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/ibc/Slashing.md b/docs/ibc/Slashing.md index 6b1be735..710e526f 100644 --- a/docs/ibc/Slashing.md +++ b/docs/ibc/Slashing.md @@ -27,7 +27,7 @@ if the **entire chain** has gone Byzantine, which is to say that there are two v light client proofs for the same height, and over 1/3 of the validators have double-signed. At such a point, the light client will halt and require governance intervention to -be restored. No packets or acks on any channel between those two chains will be +be restored. No packets or ACKs on any channel between those two chains will be allowed until the governance intervention is complete. Mesh Security should detect such a state and likewise freeze the relevant `external-staking` @@ -54,10 +54,11 @@ as [well as the signature](https://github.com/cometbft/cometbft/blob/v0.37.1/pro The basic checks are: -1. Ensure both votes are by the same validator, same height, and same round, and same vote type (precommit) -2. Ensure the Block IDs of the two votes are different -3. Look up the validator's public key from the validator address (stored in `external-staking`) and ensure this is a valid validator on the consumer chain -4. Finally, [verify the signature on both votes](https://github.com/cometbft/cometbft/blob/v0.37.1/evidence/verify.go#L211-L219) using the public key and the chain-id of the consumer chain (this must be set up in the `external-staking` contract) +1. Ensure both votes are by the same validator, same height, and same round, and same vote type (pre-commit). +2. Ensure the Block IDs of the two votes are different. +3. Look up the validator's public key from the validator address (stored in `external-staking`) and ensure this is a valid validator on the consumer chain. +4. Finally, [verify the signature on both votes](https://github.com/cometbft/cometbft/blob/v0.37.1/evidence/verify.go#L211-L219) + using the public key and the chain-id of the consumer chain (this must be set up in the `external-staking` contract) We can also add some consistency checks, like "this evidence has not been seen before", which is equivalent to "this validator has not been tombstoned yet", and maybe some limit on age of From 2322a0235716846b4d944ff4af036d0fc44c208a Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 21:43:38 +0200 Subject: [PATCH 24/43] Update docs/UseCases.md Co-authored-by: Alexander Peters --- docs/UseCases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/UseCases.md b/docs/UseCases.md index a91d34c8..1758176b 100644 --- a/docs/UseCases.md +++ b/docs/UseCases.md @@ -1,7 +1,7 @@ # Use Cases We assume each chain has a native staking token denom, with some quantity (could be 1 or 1,000,000,000). -This is the initial token to start the chain with. Each provider chain that connects gets a maximum of X virtual tokens, +This is the initial token to start the chain with. Each provider chain that connects may mint up to a maximum of X virtual tokens, defined by consumer governance when authorizing the new provider. The ratio between the amount of native tokens and the max cap of virtual tokens on each provider is a key element in defining the various security models. From 290da7a347f32cd9319105db670927b5deea897b Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 21:45:14 +0200 Subject: [PATCH 25/43] Update docs/consumer/Consumer.md Co-authored-by: Alexander Peters --- docs/consumer/Consumer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index acf5716b..470953af 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -46,7 +46,7 @@ Staking contract if needed. ### IBC -When we [deploy the contracts](../ibc/Overview.md#deployment), we connect the Converter on the Consumer +After we [deployed the contracts](../ibc/Overview.md#deployment), an IBC channel can be setup between Converter on the Consumer chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this connection is established, Consumer governance can authorize this Converter with some ability to mint on the [Virtual Staking](./VirtualStaking.md) contract. From c1b1b67af2a26463a2fba1276800f60eb153abdf Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 21:48:03 +0200 Subject: [PATCH 26/43] Update docs/ibc/ControlChannel.md Co-authored-by: Alexander Peters --- docs/ibc/ControlChannel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ibc/ControlChannel.md b/docs/ibc/ControlChannel.md index f96c7b81..a12daa42 100644 --- a/docs/ibc/ControlChannel.md +++ b/docs/ibc/ControlChannel.md @@ -30,7 +30,7 @@ digging into the sub-protocols below. The Consumer sends packets for the original validator set. -TODO: Validator set updates are not yet supported. Only the original validator set is sync +TODO: Validator set updates are not yet supported. Only the original validator set is sent once after IBC connection establishment at the moment. We use a CRDT-based algorithm to maintain a consistent view of the validator set regardless From cb3a9a48c395e60e3eff3f2b999625300201baf8 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Fri, 7 Jul 2023 21:49:00 +0200 Subject: [PATCH 27/43] Update docs/consumer/Converter.md Co-authored-by: Alexander Peters --- docs/consumer/Converter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index 7f52b360..a6fd72f4 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -111,7 +111,7 @@ over IBC. The actual tokens will be kept on the Converter, for later distributio The External Staking contract on the Consumer chain will receive the vouchers of the amounts per validator, and will then inform in turn to the Converter of the proper distribution of rewards per user. -This is done in this way because while the Consumer side knows the rewards per validator, the information +This is done in this way because while the Consumer side does not know anything about individual stakers. This is stored on the provider side with the distribution model. about which delegators are staking with which validator is only known on the Provider side. Thus, the Provider must inform the Consumer of the proper distribution of rewards. From 09b0ce9dd1edc3c88610790adaf8004a8873899b Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 09:58:06 +0200 Subject: [PATCH 28/43] Add note about required wasmd version --- docs/consumer/Consumer.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index 470953af..6a858d6c 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -44,6 +44,8 @@ contract, which is taken as an explicit argument, and normally will be the same could be a different one). This (wasm) admin is very important, as it is the only one who can migrate the Virtual Staking contract if needed. +**Note**: wasmd v0.41+ is required, for gov authority propagation to work. + ### IBC After we [deployed the contracts](../ibc/Overview.md#deployment), an IBC channel can be setup between Converter on the Consumer From 15a1c51d626f81e797ef612455c881940579f8ef Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:01:28 +0200 Subject: [PATCH 29/43] Rewrite Consumer contracts requirements section --- docs/consumer/Consumer.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index 6a858d6c..f63da911 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -36,15 +36,18 @@ and a [Virtual Staking](./VirtualStaking.md) contract must be stored on chain. S Then the Converter contract is instantiated, which takes the address of the Price Feed contract, and the Code Id of the Virtual Staking contract. -The Converter will then instantiate a Virtual Staking contract to work with it, as they both need references +It will then instantiate a Virtual Staking contract to work with it, as they both need references to each other. -In addition to the Price Feed contract address and the code id of the Virtual Staking contract, -the Converter also needs the discount ratio, the remote denomination, and the admin of the Virtual Staking -contract, which is taken as an explicit argument, and normally will be the same admin of the Converter (but -could be a different one). This (wasm) admin is very important, as it is the only one who can migrate the Virtual -Staking contract if needed. +The Converter also needs: + - Discount ratio. + - Remote denomination. + - Admin of the Virtual Staking contract. -**Note**: wasmd v0.41+ is required, for gov authority propagation to work. +The admin is taken as an explicit argument, and normally will be the same admin of the Converter (but +could be different). The (wasm) admin is important, as it is the only one who can migrate the Virtual +Staking contract without a chain upgrade. + +**Note**: wasmd v0.41+ is required for gov authority propagation to work. ### IBC From b469a8ef6510f7bfa17e0fe8492aa5ea19fc28db Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:06:29 +0200 Subject: [PATCH 30/43] Clarify gov auth to mint / burn --- docs/consumer/Consumer.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index f63da911..f0761177 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -53,8 +53,8 @@ Staking contract without a chain upgrade. After we [deployed the contracts](../ibc/Overview.md#deployment), an IBC channel can be setup between Converter on the Consumer chain with an [External Staking](../provider/ExternalStaking.md) contract on the Provider. Once this -connection is established, Consumer governance can authorize this Converter with some ability to mint -on the [Virtual Staking](./VirtualStaking.md) contract. +connection is established, Consumer chain governance needs to authorize the [Virtual Staking](./VirtualStaking.md) contract +to mint virtual tokens. The Consumer contract orchestrates the mint / burn. When IBC connection is established from the Converter, the Provider side [External Staking](../provider/ExternalStaking.md) contract must be already instantiated with the proper IBC channel information (i.e. proper connection id From 1cefda61d576177a78d3e0e1048a5ccfc1d9aa38 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:10:01 +0200 Subject: [PATCH 31/43] Clarify IBC conn / channel setup --- docs/consumer/Consumer.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index f0761177..b022eb3e 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -56,8 +56,9 @@ chain with an [External Staking](../provider/ExternalStaking.md) contract on the connection is established, Consumer chain governance needs to authorize the [Virtual Staking](./VirtualStaking.md) contract to mint virtual tokens. The Consumer contract orchestrates the mint / burn. -When IBC connection is established from the Converter, the Provider side [External Staking](../provider/ExternalStaking.md) -contract must be already instantiated with the proper IBC channel information (i.e. proper connection id +When the IBC connection is established between chains, a channel can be setup between the Converter on the Consumer side +and the [External Staking](../provider/ExternalStaking.md) contract on the Provider. +The External Staking contract must be already instantiated with the proper IBC channel information (i.e. proper connection id and port id information in the `AuthorizedEndpoint` struct, set as part of their `InstantiateMsg`). See the [Provider](../provider/Provider.md) Setup for more information. From 957b070549b3067b36ef9472324e0d94c4c5ec86 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:16:20 +0200 Subject: [PATCH 32/43] Clarify connection vs channel establishment --- docs/consumer/Converter.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index a6fd72f4..2d24c4e1 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -9,10 +9,10 @@ The converter is connected to the Provider chain via IBC and handles the various ## Validator Updates Flow -The Converter contract on the Provider chain will send validator updates to the Consumer chain via IBC packets, +The Converter contract on the Provider chain will send validator information to the Consumer chain via IBC packets, so that the External Staking contract on the Provider can know if a given remote validator is active or not. +These packets are currently being sent on IBC channel ACK only (i.e. as part of IBC channel establishment between both contracts). -These packets are sent on IBC connection establishment. TODO: Send validator updates dynamically, so that the Provider chain is kept up-to-date with the validator set on the Consumer chain. From 9224e3295e5f08fe6c7c671aad3927429e489ae9 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:37:17 +0200 Subject: [PATCH 33/43] Clarify Consumer / Provider flows --- docs/consumer/Converter.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index 2d24c4e1..e129af13 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -18,7 +18,7 @@ the Consumer chain. ## Staking Flow -Once the connection is established, the Provider can send various "virtual stake" messages to the Converter, +Once the IBC connection and channel are established, the Provider can send various "virtual stake" messages to the Converter, which is responsible for processing them and normalizing amounts for the local "virtual staking" module. These packets are sent via a dedicated channel between the Provider chain and the Consumer chain, to ensure that there are no other security assumptions (3rd party modules) involved in sending this critical staking @@ -105,23 +105,28 @@ We dig more into the mechanics of the Virtual Staking contract in the Once per epoch, the Virtual Staking module will trigger rewards. This will generate a number of messages to the Converter, specifying which validators the rewards belong to, along with the -amounts of rewards. The Converter will send vouchers of the reward amounts to the Exteral Staking contract +amounts of rewards. The Converter will send reward amounts to the External Staking contract over IBC. The actual tokens will be kept on the Converter, for later distribution. -The External Staking contract on the Consumer chain will receive the vouchers of the amounts per -validator, and will then inform in turn to the Converter of the proper distribution of rewards per user. +The External Staking contract will receive the amounts per validator, +and will inform the Converter of the distribution of rewards per user. -This is done in this way because while the Consumer side does not know anything about individual stakers. This is stored on the provider side with the distribution model. +Say, there are 10 rewards for validator 1, and 20 rewards for validator 2. The External Staking +contract will inform the Converter that 5 rewards go to user X, 15 rewards go to user Y, and 10 rewards +go to user Z. + +This is done this way because while the Consumer side does not know anything about individual stakers. This is stored on the provider side with the distribution model. about which delegators are staking with which validator is only known on the Provider side. Thus, the Provider must inform the Consumer of the proper distribution of rewards. -The Converter will then send the actual rewards to their respective owners. +The Converter will then send the actual rewards in the chain's native token +to their respective owners on the Consumer chain. ## Rebalancing Flow Once per epoch, the Virtual Staking module will check if a rebalancing of staking amounts is required. This can happen once the max staking cap is reached on the Consumer. In this case, the Virtual Staking -module will trigger a rebalancing, which will generate a number of messages for bonding/unbonding +module will trigger a rebalancing, which will generate a number of messages for bonding / unbonding of amounts for each validator. TODO: The current implementation does not consider changes to the validator set, and rebalancing may From 912c4657258a8b67201d848bd34aea84ba7afa3b Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:40:25 +0200 Subject: [PATCH 34/43] Remove refs to 'vouchers' for clarity --- docs/ibc/ControlChannel.md | 4 ++-- docs/ibc/Overview.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/ibc/ControlChannel.md b/docs/ibc/ControlChannel.md index a12daa42..576b17e5 100644 --- a/docs/ibc/ControlChannel.md +++ b/docs/ibc/ControlChannel.md @@ -5,8 +5,8 @@ This document describes the "control channel", which is a direct IBC channel bet on the Consumer side. This is used to send messages about bonding and unbonding, and any other metadata about the protocol (like validators). -It is also used to send reward vouchers(./Rewards.md), which represent amounts of rewards -that have been earned by delegators from the Provider chain, and are redeemable on the Consumer chain. +It is also used to send reward amounts(./Rewards.md) that have been earned by delegators +from the Provider chain, and are redeemable on the Consumer chain. It is **not** used to [handle slashing](./Slashing.md), as there are concerns a malicious state machine would lie, so we demand original evidence of Tendermint diff --git a/docs/ibc/Overview.md b/docs/ibc/Overview.md index 87b05ca4..feadc471 100644 --- a/docs/ibc/Overview.md +++ b/docs/ibc/Overview.md @@ -71,7 +71,7 @@ principle form of communication, called the [Control Channel](./ControlChannel.md), and is the core piece of the Mesh Security IBC protocol. -We send reward vouchers over the Control Channel from the converter contract +We send reward amounts over the Control Channel from the Converter contract in the Consumer side to the External Staking contract on the Provider side, so that they can be distributed by the Provider side. The Provider side indicates in turn to the Consumer side which users should receive rewards. From 0dafad826088086fa8eee5f03fd80c305ca0a855 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 10:59:52 +0200 Subject: [PATCH 35/43] Add empty version variant --- docs/ibc/ControlChannel.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/ibc/ControlChannel.md b/docs/ibc/ControlChannel.md index 576b17e5..e34bc0ab 100644 --- a/docs/ibc/ControlChannel.md +++ b/docs/ibc/ControlChannel.md @@ -79,8 +79,9 @@ The general process (assuming a vault is already established on the Provider) is Opening the channel is a 4-step process. It must be initiated by the Consumer side. 1. Start with `OpenInit` from converter to the (connection, port) of the external staking. The version - SHOULD be set to the highest mesh-security version it supports (see below), and the channel ordering - MUST be "unordered". It MUST error if it has a previously established channel. + SHOULD be set to the highest mesh-security version it supports (see below), or left empty to let + the contract handle it. The channel ordering MUST be "unordered". It MUST error if it has a previously + established channel. 2. The external staking contract receives `OpenTry`. The channel ordering MUST be "unordered", and the version protocol MUST be `mesh-security`. It performs version negotiation as defined below. It MUST error if the (connection, port) being proposed is not the one it was initialized with. From 01cdd4623d219f6f3a5a19e71a44877deccb24d6 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 11:00:08 +0200 Subject: [PATCH 36/43] Clarify admin roles / perms --- docs/ibc/Overview.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/ibc/Overview.md b/docs/ibc/Overview.md index feadc471..adacf7a9 100644 --- a/docs/ibc/Overview.md +++ b/docs/ibc/Overview.md @@ -47,7 +47,8 @@ Establish a channel (allow list): Now that we have a channel and know which contract is talking to whom, we need to authorize them: -1. Admin removes their admin rights on both contracts (or passes to chain governance). +1. Admin removes their admin rights on both contracts. Or passes to chain governance; it would be + unwise to not have any admin, as this would prevent on-chain migrations. 2. Due diligence performed on configuration of both contracts and the channel. Parties from both chains can verify the contract code and configuration. 3. Consumer chain governance votes to authorize this _Virtual Staking_ contract to have special From 0ce7fac27ba35921fddf457f23476e3ce1257cb8 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 12:11:05 +0200 Subject: [PATCH 37/43] General syntax corrections --- docs/ibc/Serializability.md | 18 ++--- docs/ibc/Staking.md | 138 ++++++++++++++++++------------------ docs/ibc/Validators.md | 32 ++++----- 3 files changed, 93 insertions(+), 95 deletions(-) diff --git a/docs/ibc/Serializability.md b/docs/ibc/Serializability.md index fc69d9a9..cdd47d8a 100644 --- a/docs/ibc/Serializability.md +++ b/docs/ibc/Serializability.md @@ -64,7 +64,7 @@ the details of the actual transactions being executed. > A more general definition of conflicting operations (also for complex operations, which may each consist of several "simple" read/write operations) requires that they are noncommutative (changing their order also changes their combined result). Each such operation needs to be atomic by itself (using proper system support) in order to be considered an operation for a commutativity check. For example, read–read operations are commutative (unlike read–write and the other possibilities) and thus read–read is not a conflict. Another more complex example: the operations increment and decrement of a counter are both write operations (both modify the counter), but do not need to be considered conflicting (write-write conflict type) since they are commutative (thus increment–decrement is not a conflict; e.g., already has been supported in the old IBM's IMS "fast path"). The idea of "commutative operations" gets very attractive for our case. In this case, we need no -locks, or special handling of the packet and ack ordering. One way to guarantee communativity +locks, or special handling of the packet and ACK ordering. One way to guarantee commutativity is to create a [data structure that is a CRDT](#crdts) (link to section below). Often that is not possible (we have some global invariants that may never be broken), and we need to look further. But if it is possible to use CRDTs, you can always guarantee serializability, without @@ -113,8 +113,8 @@ This can be implemented in a two-chain IBC protocol with the following approach: 1. The sending chain A completes its sub-transaction and maintains all needed locks (or other structs) to guarantee it can commit or rollback the state transitions as needed. -2. The receiving chain B processes its sub-transaction and returns a success or error ack. -3. If the ack is a success, chain A commits its state transitions. If it is an error, chain A +2. The receiving chain B processes its sub-transaction and returns a success or error ACK. +3. If the ACK is a success, chain A commits its state transitions. If it is an error, chain A rolls back its state transitions. This builds on the existing IBC infrastructure and is the reason why ACKs were introduced @@ -176,7 +176,7 @@ resistent to deadlock. We would do the following: 1. Start Tx on Sending Chain: Acquire all read/write locks on all data that will be touched. This is the "growing" phase of the lock. 2. Process Packet on Receiving Chain: Acquire all read/write locks on all data, process data, release all locks. This goes from the "growing" phase to the "shrinking" phase. -3. Process Ack on the Sending Chain: Process ack, and release all locks. This is the "shrinking" phase. +3. Process ACK on the Sending Chain: Process ACK, and release all locks. This is the "shrinking" phase. Note that blockchains actually process all transactions sequentially (this is one of their main purposes), so we can simplify this by considering any non-IBC transaction to get and release locks on all data it touches during its execution. @@ -187,11 +187,11 @@ _data read during Phase 3 will need a read lock from Phase 1_ With that, we can simplify to: 1. Start Tx on Sending Chain: Acquire all locks on all data that may be touched in Phase 3, but don't write anything. -2. Process Packet on Receiving Chain: Process data, and return success or error ack. -3. Process Ack on the Sending Chain: Commit or rollback based on ack. Only can read/write data held under lock from step 1 +2. Process Packet on Receiving Chain: Process data, and return success or error ACK. +3. Process ACK on the Sending Chain: Commit or rollback based on ACK. Only can read/write data held under lock from step 1 If we modelled ICS20 like this, it would require us to hold a lock on the account balance of the sender (at least the keys holding the -denom being sent) from the original send until the ack. This would not interfere with any other processes, +denom being sent) from the original send until the ACK. This would not interfere with any other processes, but that user could not send tokens locally, stake those tokens, receive local tokens, or even properly query his balance (as it is undefined). In some cases where the business logic is very complex and hard to model commutatively, and the keys under lock are only used by this one @@ -222,7 +222,7 @@ We can say that ICS20 implementation does something like this. As the only invar it preemptively moves the tokens out of the users account to an escrow. This doesn't require any further lock on the user's account, but ensures no other transaction will be possible to execute that would render the user's balance below 0 if this was committed. Thus, any other transaction that would be commutative with this (commit or rollback) can be executed concurrently, but any other -transaction that would conflict with this (e.g. spend the same tokens, and only be valid if the ICS20 transfer gets an error ack), +transaction that would conflict with this (e.g. spend the same tokens, and only be valid if the ICS20 transfer gets an error ACK), will immediately fail. ICS20 is an extremely simple case, and you don't need such theory to describe decrementing one counter. However, assume there was not @@ -235,7 +235,7 @@ or conflicting, operations). One idea to implement this would be to not just store one value (the balance), but a range of values, covering possible values if all in-flight transactions were to succeed or fail. Normal transactions (send 100 tokens) would update both values and error if either one violated some invariant (too high or too low). When an IBC transaction is initiated, it would execute eg. "Maybe(200)", which would -attempt to decrease the min by 200 but leave the max unchanged (requiring this range to remain valid). When the IBC ack is received, +attempt to decrease the min by 200 but leave the max unchanged (requiring this range to remain valid). When the IBC ACK is received, we would either `Commit(200)` or `Rollback(200)`, which would once again bring min and max to the same value (which one depends on Commit or Rollback). This approach would only work by values not used by complex formulas, such as balances of an AMM (we can't calculate prices of ranges), diff --git a/docs/ibc/Staking.md b/docs/ibc/Staking.md index e2acc455..e98a5f69 100644 --- a/docs/ibc/Staking.md +++ b/docs/ibc/Staking.md @@ -2,15 +2,15 @@ The staking protocol has two basic operations: -- Stake X tokens on Validator V - `S(V, X)` -- Unstake X tokens from Validator V - `U(V, X)` +- Stake X tokens on Validator V - `S(V, X)`. +- Unstake X tokens from Validator V - `U(V, X)`. We want to ensure that at any point in time, when all in-flight messages would be resolved, both the -provider and the consumer chain have the same view of the staking state. (**TODO** Consider the +Provider and the Consumer chain have the same view of the staking state. (**TODO** Consider the effects of slashing, not included in this document currently). -We also want to guarantee that the provider chain always maintains sufficient staked tokens -in the vault to cover all virtual staking actions currently outstanding on the provider chain. +We also want to guarantee that the Provider chain always maintains sufficient staked tokens +in the vault to cover all virtual staking actions currently outstanding on the Provider chain. As [mentioned before](./ControlChannel.md#channel-ordering), we wish to use an unordered channel, and therefore must bring a degree of understanding of [Serializability](./Serializability.md) @@ -18,27 +18,25 @@ to this protocol. ## Delegation Syncing -The consumer side wants to maintain the same counter per validator as the provider side. +The Consumer side wants to maintain the same counter per validator as the Provider side. It manages a delegation count per validator - `D[V]`, with the following rules: -- Uninitialized is equivalent to `D[V] = 0` -- `S(V, X)` => `D[V] += X`, return success ack +- Uninitialized is equivalent to `D[V] = 0`. +- `S(V, X)` => `D[V] += X`, return success ACK. - `U(V, X)` => - - if `D[V] < X`, return error ack - - else `D[V] -= X`, return success ack + - if `D[V] < X`, return error ACK, + - else `D[V] -= X`, return success ACK. This is a pretty straightforward counter with a lower bound of 0, along with an increment and decrement counters. -The only requirements to ensure this stays in sync is that the provider can successfully commit the same -changes upon a success ack, and is able to revert them (unstake) on an error ack. - -**TODO** Do we need text / code / explanation here? +The only requirements to ensure this stays in sync is that the Provider can successfully commit the same +changes upon a success ACK, and is able to revert them (unstake) on an error ACK. ## Provider Side Design -Beyond this delegation list, the provider must maintain much more information. -The delegation to a given validator must be mapped to a user on the provider chain. +Beyond this delegation list, the Provider must maintain much more information. +The delegation to a given validator must be mapped to a user on the Provider chain. This user must also have a sufficient lien on the vault to cover the delegation. And the vault must have sufficient funds to cover the lien. @@ -47,43 +45,43 @@ and the `vault`. It includes possible conflicts with non-IBC transactions, like collateral from the vault that would be needed to cover the lien for an in-flight delegation. Let us start analysing the protocol, but viewing all the state transitions that must be made on proper -success acks. These same state transitions must always be reverted without problem upon receiving error acks. +success ACKs. These same state transitions must always be reverted without problem upon receiving error ACKs. ### Identifying Potential Conflicts A staking operation would have the following steps and checks: -- Send a lien from `vault` to `external-staking` contract - - Ensure there is sufficient collateral to cover max(lien) - - Ensure there is sufficient collateral in sum(potential slashes) - - Increase the lien for that given user on the `external-staking` contract -- Add a delegation in `external-staking` contract - - Increase stake count for (user, validator) - - Increase total stake on the validator - - Increase the user's shares in the reward distribution -- Send an IBC packet to inform the Consumer - - Guarantee we can commit all above on success - - Guarantee we can rollback all above on error +- Send a lien from `vault` to `external-staking` contract. + - Ensure there is sufficient collateral to cover max(lien). + - Ensure there is sufficient collateral in sum(potential slashes). + - Increase the lien for that given user on the `external-staking` contract. +- Add a delegation in `external-staking` contract. + - Increase stake count for (user, validator). + - Increase total stake on the validator. + - Increase the user's shares in the reward distribution. +- Send an IBC packet to inform the Consumer. + - Guarantee we can commit all above on success. + - Guarantee we can roll back all above on error. An unstaking operation would have the following steps and checks: -- Remove a delegation in `external-staking` contract - - Ensure stake count (user, validator) is set and greater than desired unstake amount - - Ensure total stake on the validator is set and greater than desired unstake amount (should always be true if above is true) - - Decrease stake count for (user, validator) - - Decrease total stake on the validator - - Decrease the user's shares in the reward distribution +- Remove a delegation in `external-staking` contract. + - Ensure stake count (user, validator) is set and greater than desired unstake amount. + - Ensure total stake on the validator is set and greater than desired unstake amount (should always be true if above is true). + - Decrease stake count for (user, validator). + - Decrease total stake on the validator. + - Decrease the user's shares in the reward distribution. - Add an entry to the (user, validator) "pending unbonding" to withdraw said tokens after the unbonding period. -- Send an IBC packet to inform the Consumer - - Guarantee we can commit all above on success - - Guarantee we can rollback all above on error +- Send an IBC packet to inform the Consumer. + - Guarantee we can commit all above on success. + - Guarantee we can roll back all above on error. Possible keys with conflicts are: -- In `vault` - `collateral(user)` and `lien(user, external-staking)` -- In `external-staking` - `stake(user, validator)`, `total-stake(validator)`, `reward-shares(user)`, `pending-unbonding(user, validator)` +- In `vault` - `collateral(user)` and `lien(user, external-staking)`. +- In `external-staking` - `stake(user, validator)`, `total-stake(validator)`, `reward-shares(user)`, `pending-unbonding(user, validator)`. -### Identifying Potential Commutatibility +### Identifying Potential Commutability The general design should be to write all changes only on a successful ACK, but hold any locks needed to ensure those writes will not fail in any condition. Using the approach of [Value Ranges](./Serializability.md#value-ranges), let us analyze @@ -91,10 +89,10 @@ what needs to be minimally enforced here. For staking, we update: -- `vault::lien(user, external-staking)` => `Maybe(+X)` -- `external-staking::stake(user, validator)` => `Maybe(+X)` -- `external-staking::total-stake(validator)` => `Maybe(+X)` -- `external-staking::reward-shares(validator, user)` => `Maybe(+X)` +- `vault::lien(user, external-staking)` => `Maybe(+X)`. +- `external-staking::stake(user, validator)` => `Maybe(+X)`. +- `external-staking::total-stake(validator)` => `Maybe(+X)`. +- `external-staking::reward-shares(validator, user)` => `Maybe(+X)`. The lower three have no upper limit and thus can never fail, so those operations are always commutative with others valid operations. `vault::lien` has an upper value, and thus applying `Commit(+X)` could have a conflict with another transaction, so we must "lock" that @@ -102,11 +100,11 @@ value. For unstaking, we update: -- `external-staking::stake(user, validator)` => `Maybe(-X)` -- `external-staking::total-stake(validator)` => `Maybe(-X)` -- `external-staking::reward-shares(validator, user)` => `Maybe(-X)` -- `external-staking::total-shares(validator)` => `Maybe(-X)` -- `external-staking::pending-unbonding(user, validator)` => `Append((X, T))` +- `external-staking::stake(user, validator)` => `Maybe(-X)`. +- `external-staking::total-stake(validator)` => `Maybe(-X)`. +- `external-staking::reward-shares(validator, user)` => `Maybe(-X)`. +- `external-staking::total-shares(validator)` => `Maybe(-X)`. +- `external-staking::pending-unbonding(user, validator)` => `Append((X, T))`. We have already seen that appending to an ordered/sorted list is commutative with other valid operations, so we can consider that as commutative. The other three are all decrements, and may well conflict with another concurrent operation, such as another unstake, @@ -116,11 +114,11 @@ as `Commit(-X)` could potentially fail. From the above, we have a list of all the values that could possibly cause conflicts with other transactions and thus must be locked: -- `vault::lien(user, external-staking)` -- `external-staking::stake(user, validator)` -- `external-staking::total-stake(validator)` -- `external-staking::reward-shares(validator, user)` -- `external-staking::total-shares(validator)` +- `vault::lien(user, external-staking)`. +- `external-staking::stake(user, validator)`. +- `external-staking::total-stake(validator)`. +- `external-staking::reward-shares(validator, user)`. +- `external-staking::total-shares(validator)`. We can simplify this list by making use of our knowledge of how various states are derived from each other: @@ -134,8 +132,8 @@ which is sufficient to guarantee the invariants of the other staking values, wit We end up with two keys that need to write locks, both of which only affect a single user: -- `vault::lien(user, external-staking)` -- `external-staking::stake(user, validator)` +- `vault::lien(user, external-staking)`. +- `external-staking::stake(user, validator)`. Note that a write-lock will prevent reading of the value from any other transaction. That means the same user offering a lien on another validator, which iterates over all liens to find the sum of max slashing, will be blocked until the first transaction completes. @@ -150,27 +148,27 @@ exact lock (we must store the value pending in vault to not trust the external-s In this case, we look to approximate the lock by a series of writes that guarantee the invariant is maintained. -- Preparing Phase: `vault::lien(user, external-staking) += X` -- Commit: do nothing (no message) -- Rollback: (call release_lien) `vault::lien(user, external-staking) -= X` +- Preparing Phase: `vault::lien(user, external-staking) += X`. +- Commit: do nothing (no message). +- Rollback: (call release_lien) `vault::lien(user, external-staking) -= X`. The principle questions we need to answer to prove this is safe is: - Can we guarantee that the rollback will never fail? - Is there any transaction that would be valid after the preparing phase, but not after the rollback? -For the first one, since the increment is held by our `external-staking` contract, by correct rules of not releasing until the IBC Ack, +For the first one, since the increment is held by our `external-staking` contract, by correct rules of not releasing until the IBC ACK, we can guarantee that the lien is still held and able to be decremented. For the second one, the case is some transition that would be effectively using a "Phantom Read" to be valid. I will review the possible transactions on the vault: -- `bond` - increases collateral -- `unbond` - decreases collateral after comparing to max(liens) -- `stake_remote` - sends call to stake after comparing to max(liens) -- `stake_local` - sends call to stake after comparing to max(liens) -- `release_local_stake` - reduces lien and adjusts max_lien, slashable -- `release_remote_stake` - reduces lien and adjusts max_lien, slashable +- `bond` - increases collateral. +- `unbond` - decreases collateral after comparing to max(liens). +- `stake_remote` - sends call to stake after comparing to max(liens). +- `stake_local` - sends call to stake after comparing to max(liens). +- `release_local_stake` - reduces lien and adjusts max_lien, slashable. +- `release_remote_stake` - reduces lien and adjusts max_lien, slashable. Many of these will return errors if the lien is too large to permit said operation. But via inspection, none would succeed with a larger lien that would not with a smaller one. However, this is not a proof, @@ -188,10 +186,10 @@ enum Op { ``` One issue here is that the Consumer doesn't care about the user who made the staking action, only the validator. -However, when processing the ack, the Provider will need to know which user performed the action. +However, when processing the ACK, the Provider will need to know which user performed the action. We could store this as local state in the Provider, indexed by the packet sequence, but simply including an extra field in the packet is much simpler. The Consumer should ignore the user field, but the Provider will receive this original -Packet along with the ack and be able to properly commit/rollback the staking action. +Packet along with the ACK and be able to properly commit/rollback the staking action. ### Consumer-Side Logic @@ -217,7 +215,7 @@ match op { ### Provider-Side Staking -This is more complex logic over multiple contracts, so I will only give a high-level overview here rather than the Rust psuedo-code: +This is more complex logic over multiple contracts, so I will only give a high-level overview here rather than the Rust pseudocode: **TODO** See [open question above](#idea-approximating-locks) for a discussion on how to possibly avoid locking on staking. @@ -261,4 +259,4 @@ do_stake(); ### Error correction -**TODO** Ideas about using values in success acks to double check the state matches expectations and flag possible errors +**TODO** Ideas about using values in success ACKs to double check the state matches expectations and flag possible errors diff --git a/docs/ibc/Validators.md b/docs/ibc/Validators.md index c332c15a..eeaacddb 100644 --- a/docs/ibc/Validators.md +++ b/docs/ibc/Validators.md @@ -4,7 +4,7 @@ It is important for the provider to know the proper validators on the consumer c Both in order to limit the delegations to valid targets _before_ creating an IBC message, but also in order to track tendermint public keys to be able to slash properly. -We define the "Validator Subprotocol" as a way to sync this information. It uses a CRDT-like +We define the "Validator Sub-protocol" as a way to sync this information. It uses a CRDT-like design to maintain consistency in the face of arbitrary reordering. And retries in order to ensure dropped packets are eventually received. @@ -12,7 +12,7 @@ to ensure dropped packets are eventually received. All validator change messages are initiated from the consumer side. The provider is responsible for guaranteeing any packet with a success ACK is written to state. -Given all successful acks have been committed, the consumer maintains enough +Given all successful ACKs have been committed, the consumer maintains enough information to sync outstanding changes and guarantee the Provider will eventually have a proper view of the dynamic validator set. @@ -29,7 +29,7 @@ stream of `AddValidators` and `RemoveValidators` messages. As new validators are added to the active set, the consumer will send an `AddValidators` message with their information. We do not signal when a validator is removed from the active -set as long as it is still valid to delegate to them (ie. they have not been tombstoned). +set as long as it is still valid to delegate to them (i.e. they have not been tombstoned). When a validator is tombstoned, the consumer will send a `RemoveValidators` message with the address of that validator. Once it has been removed, it can never be added again. @@ -40,12 +40,12 @@ _Note: sending these updates as a stream (rather than polling for the whole list As you see from the message types, we are using an operation-based CRDT design. This requires that all operations are commutative. We also guarantee they are idempotent, -although that is not strictly required given IBC's "exactly once" delivery. +although that is not strictly required, given IBC's "exactly once" delivery. In this section, we consider the operations that compose IBC packets: -- `A(x, p)` - Add validator `x` with pubkey `p` to the validator set -- `R(x)` - Remove validator `x` from the validator set +- `A(x, p)` - Add validator `x` with pubkey `p` to the validator set. +- `R(x)` - Remove validator `x` from the validator set. We do not consider Tendermint key rotation in this section, but describe it as an addition in [the following section](#validator-key-rotation). @@ -53,9 +53,9 @@ This section is sufficient to support the basic operations available today. We wish to maintain a validator set `V` on the Provider with the following properties: -- If no packet has been received for a given `x`, `x` is not in `V` -- If `R(x)` has been received, `x` is not in `V` -- If `A(x, p)` has been received, but no `R(x)`, `x` is in `V` with pubkey `p` +- If no packet has been received for a given `x`, `x` is not in `V`. +- If `R(x)` has been received, `x` is not in `V`. +- If `A(x, p)` has been received, but no `R(x)`, `x` is in `V` with pubkey `p`. ### Basic Implementation @@ -102,8 +102,8 @@ let new_state: State = match (old_state, op) { ### Proof of correctness The basic implementation without public key rotation is another expression of the same algorithm -used in [`2P-Set`](). This is a proven CRDT, and if we implement it to match the spec, we have a -guarantee of commutability. +used in [`2P-Set`](). +This is a proven CRDT, and if we implement it to match the spec, we have a guarantee of commutability. ## Validator Key Rotation @@ -112,19 +112,19 @@ to support. This protocol is designed to handle Tendermint key rotation, such th address may be associated with multiple public keys over the course of its existence. This feature is not implemented in Cosmos SDK yet, but long-requested and the Mesh Security -protocol should be future proof. (Note: we do not handle changing the `valoper` address as +protocol should be future-proof. (Note: we do not handle changing the `valoper` address as we use that as the unique identifier). We wish the materialized state to have the following properties: We wish to maintain a validator set `V` on the Provider with the following properties: -- If no packet has been received for a given `x`, `x` is not in `V` -- If `R(x)` has been received, `x` is not in `V` +- If no packet has been received for a given `x`, `x` is not in `V`. +- If `R(x)` has been received, `x` is not in `V`. - If at least one `A(x, _, _)` has been received, but no `R(x)`, `x` is in `V` with: - - A set of all pubkeys, along with the block height they were first active from. + - A set of all pubkeys, along with the block height they were first active from (We may represent this as a sorted list without duplicates, but that is a mathematically - equivalent optimization) + equivalent optimization). ### Proof of correctness From 7b10d8396fa4551ad672a335a78b5609ecae78a8 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 12:29:04 +0200 Subject: [PATCH 38/43] Flesh out the Rewards doc --- docs/ibc/Rewards.md | 88 +++++++++++++++++++++++---------------------- 1 file changed, 46 insertions(+), 42 deletions(-) diff --git a/docs/ibc/Rewards.md b/docs/ibc/Rewards.md index 5a75dfc2..8db26447 100644 --- a/docs/ibc/Rewards.md +++ b/docs/ibc/Rewards.md @@ -1,27 +1,30 @@ # Cross-Chain Rewards Protocol -The cross-chain rewards protocol is used to send staking rewards from the consumer chain -to the provider chain, where it will be distributed among the virtual delegators. +The cross-chain rewards protocol is used to send staking rewards from the Consumer chain +to the Provider chain, where it will be distributed among the virtual delegators. + +**Note**: But see [Possible Implementations](#possible-implementations) below for alternatives +to this approach. In particular, rewards can be distributed on the Consumer chain directly. ## Channel Requirements -The original reward token is the native staking token of the consumer chain, -and it should be transferred to the provider chain using the most commonly accepted +The original reward token is the native staking token of the Consumer chain, +and it should be transferred to the Provider chain using the most commonly accepted token bridge, to ensure fungibility. Generally this will be the ICS20 channel between `transfer` ports on a standard accepted channel. However, if there is another bridge that is more commonly used, it may be used instead. -The consumer side should take care of both the _transfer_ of the reward token -as well as _distributing_ the tokens to the `external-staking` contract. -This means, the provider side just sees it as a local method execution +The Consumer side should take care of both the _transfer_ of the reward token +and _distributing_ the tokens to the `external-staking` contract. +This means, the Provider side just sees it as a local method execution along with a whitelisted IBC token denom. -## Reward Distribution +## Rewards Distribution -The reward distribution is done by the `external-staking` contract on the provider chain, and not part of the +The rewards distribution is done by the `external-staking` contract on the Provider chain, and not part of the IBC protocol. However, it does touch on timing issues and I want to clarify how some of that is handled. -First, rewards are send from consumer -> provider (by one of the methods described below) on infrequent epochs. +First, rewards are send from Consumer -> Provider (by one of the methods described below) on infrequent epochs. Whoever is recorded as staked to that validator at the end of the epoch will receive a share of the rewards. We don't track when during the epoch they staked, so the one who staked 1 minute after epoch N and the other who staked 1 minute before epoch N+1 will both receive the same share of rewards. @@ -37,7 +40,7 @@ about [serializability](./Serializability.md) in this case, we wouldn't allow an packets (from all users) and finalized. This is unrealistic, so we can relax this a bit to "read committed", which is close enough as there are no invariants to enforce here, and we already accepted an approximation above. -This means, that we don't update a user's reward shares for any in-flight IBC transaction until we have received a success ack. +This means, that we don't update a user's reward shares for any in-flight IBC transaction until we have received a success ACK. This is such an edge case on which side of the epoch they fall, and both decisions are equally valid from an economic perspective. However, in order to favor the approach of database consistency, we will use the "read committed" approach, and only count rewards after a successful ACK. @@ -47,7 +50,7 @@ after a successful ACK. As mentioned above, the general protocol design supports multiple bridges, and the external staking contract need not be aware of the bridge used. However, it is essential that the converter contract select an adequate process -that allows it to (a) send the reward tokens to the provider chain, +that allows it to (a) send the reward tokens to the Provider chain, (b) execute the `external-staking` contract to distribute the tokens, and (c) handle any IBC errors (timeout, etc) that may occur. @@ -58,10 +61,10 @@ Below we discuss some proposed implementations: **Not Recommended** In this case, there must be an established ics20-v1 channel between both chains, -and a ICA channel between the converter contract and the provider chain. +and a ICA channel between the converter contract and the Provider chain. -The converter contract will send the reward tokens to the provider chain via ICS20, -and when the ack is received, it will send an ICA message to execute `distribute_rewards` +The converter contract will send the reward tokens to the Provider chain via ICS20, +and when the ACK is received, it will send an ICA message to execute `distribute_rewards` on the `external-staking` contract, along with the previously transferred funds. **Problem**: There are currently no contract callbacks on ICS20 nor ICA. This means we cannot reliably @@ -77,7 +80,7 @@ be stranded, as there is no callback to inform them. Osmosis developed a nice addition to the ICS20 standard, which allows the sender to specify a memo field. This is used to specify a contract execution to be performed with the transferred tokens. It explicitly doesn't allow associating those tokens -with a particular address which is fine for this use case, as `distribute_rewards` +with a particular address; which is fine for this use case, as `distribute_rewards` is permissionless and accepts any payment in the proper denom. This is a nice solution, as it allows the converter to perform all actions in a single @@ -91,16 +94,16 @@ solution when one exists. Hopefully this will spur deployment of such methods. ### ICS20 with Custom IBC Middleware -With the consumer chain including the mesh-security-sdk, we have the opportunity to add a custom IBC-middleware -to the IBC-stack for ICS-20. This middleware can be used to do callbacks to contracts on packet ack/timeout. +With the Consumer chain including the mesh-security-sdk, we have the opportunity to add a custom IBC-middleware +to the IBC-stack for ICS-20. This middleware can be used to do callbacks to contracts on packet ACK/timeout. The process would be: -- Contract sends ICS-20 message and register for a callback on the IBC packet ID (via custom message) -- All metadata is stored on chain only and not relayed -- When the packet is ACKed/timeout, the contract receives the callback from the middleware (after the ICS-20 module) with the ack/timeout data -- When there is confidence that the ICS-20 operation succeeded, the contract can trigger the reward distribution work with the callback +- Contract sends ICS-20 message and register for a callback on the IBC packet ID (via custom message). +- All metadata is stored on chain only and not relayed. +- When the packet is ACKed/timeout, the contract receives the callback from the middleware (after the ICS-20 module) with the ACK/timeout data. +- When there is confidence that the ICS-20 operation succeeded, the contract can trigger the reward distribution work with the callback. -Note: the callback execution must not fail to not interfere with the ack/timeout process +Note: the callback execution must not fail to not interfere with the ACK/timeout process. The benefit of this solution is that it is not depending on other technology. IBC-middleware and callback registration would be provided and maintained by the mesh-security-sdk project. @@ -114,31 +117,32 @@ the Provider chain has enabled the memo field extension. In the end, ICS20 is just two trusted contracts passing some numbers back and forth with a bit of accounting. No tokens ever move. Just a promise to release the token on the source chain. So we can do this without ICS20 -if we just want the reward tokens on the consumer chain. +if we just want the reward tokens on the Consumer chain. Imagine the OSMO-JUNO scenario. The Provider on OSMO may want to get their reward tokens in IBC-JUNO on Osmosis if they wish -to sell it immediately. However, if they want to stake it (or compound on Juno), the will have to first withdraw on Osmosis, +to sell it immediately. However, if they want to stake it (or compound on Juno), they will have to first withdraw on Osmosis, then IBC Transfer it back to JUNO, then (mesh-)stake it there. -If we assume the typical use case is not selling the reward token on the provider chain, but rather re-investing it (or using it) -on the Consumer chain, there is another approach that works without ICS20. The consumer chain doesn't send any tokens, but -rather it holds the reward tokens and sends a packet over the control channel to the provider chain to execute -`distribute_rewards` with this amount. Later, when a user wants to withdraw their rewards on the provider chain, -it will send a packet to the consumer chain over the control channel to release the tokens to any address on the consumer side. +If we assume that the typical use case is not selling the reward token on the Provider chain, but rather re-investing it (or using it) +on the Consumer chain, there is another approach that works without ICS20. The Consumer chain doesn't send any tokens, but +rather it holds the reward tokens, and sends a packet over the control channel to the Provider chain to execute +`distribute_rewards` with this amount. Later, when a user wants to withdraw their rewards on the Provider chain, +it will send a packet to the Consumer chain over the control channel to release the tokens to any address on the Consumer side. This approach is basically like an embedded ICS20 inside the mesh protocol, but it doesn't issue any tokens or provide fungibility -on the provider chain. This reduces the complexity as we only have one channel and packet and don't need to orchestrate multiple +on the Provider chain. This reduces the complexity, as we only have one channel and packet and don't need to orchestrate multiple packets over multiple channels. -The biggest question is whether the functionality change is acceptable (or even desirable). Note that this will make restaking the -reward tokens easier, while making selling them on the provider chain harder. This may be a good thing (in the eyes of the -consumer chain at least), as it encourages longer-term holders. +The biggest question is whether the functionality change is acceptable (or even desirable). Note that this will make re-staking the +reward tokens easier, while making selling them on the Provider chain harder. This may be a good thing (in the eyes of the +Consumer chain at least), as it encourages longer-term holders. ## Selected Implementation -To reduce external dependencies and to encourage restaking of the consumer staking rewards, +To reduce external dependencies and to encourage re-staking of the Consumer staking rewards, we are selecting the last option for MVP, adding one more message to the control channel. -This is part of the `mesh-security 1.0.0` IBC implementation. If there is a change in the reward distribution mechanism, it would likely be breaking and would bump the IBC protocol to v2.0.0. +This is part of the `mesh-security 1.0.0` IBC implementation. If there is a change in the reward distribution mechanism, +it would likely be breaking and would bump the IBC protocol to v2.0.0. This involves two new packet variants, one on each side. The Consumer side has a new variant `ConsumerPacket::DistributeRewards { validator, rewards }`, which specifies the reward `Coin` @@ -147,9 +151,9 @@ that should be distributed among all stakers on the given validators. It is up t must guarantee to hold `rewards` tokens in reserve to be released by future IBC packets on this channel. -The provider side has a new variant `ProviderPacket::TransferRewards {}` which specifies the -among and the recipient address on the consumer chain. If the converter contract hold those -tokens in reserve (always in the case of a correct provider chain), it must release -them to the given address. The consumer chain must be designed to handle rollbacks here -on any error (especially due to invalid recipient address that cannot be validated on the -provider side). +The Provider side has a new variant `ProviderPacket::TransferRewards {}` which specifies the +amount and recipient address on the Consumer chain. If the converter contract hold those +tokens in reserve (which should always be the case in a correct Provider chain), it must release +them to the given address. The Consumer chain must be designed to handle rollbacks here +on any error (especially due to invalid recipient address, that cannot be validated on the +Provider side). From 1f5f19026a8b8aaf8fd5cffe834af588f04ce0ed Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 13:21:16 +0200 Subject: [PATCH 39/43] Point to ControlChannel doc for channel establishment section --- docs/ibc/Overview.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/docs/ibc/Overview.md b/docs/ibc/Overview.md index adacf7a9..438c1b17 100644 --- a/docs/ibc/Overview.md +++ b/docs/ibc/Overview.md @@ -36,13 +36,7 @@ the connection once it is all configured (for security). Establish a channel (allow list): -1. Admin deploys _Converter_ and _Virtual Staking_ on the Consumer side, both referencing each other. -2. Admin deploys _External Staker_ on the Provider side, registering `(connection, port)` - from _Converter_ contract from (1). -3. Admin updates _Converter_ (1) to allow the `(connection, port)` from (2). -4. Anyone establishes a new channel between _Converter_ and _External Staker_. - Both contracts ensure the other side matches the stored `(connection, port)` and refuse the channel otherwise. -5. Neither contract accept any more channels. They only accept one channel and only what matches their config. +See [Control Channel](./ControlChannel.md#deployment) for more information on how the IBC connection is established. Now that we have a channel and know which contract is talking to whom, we need to authorize them: From 52a572ae4589323c233c4365dc7b1b5ded3dd5b4 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 13:40:17 +0200 Subject: [PATCH 40/43] Fix / Use local links --- docs/README.md | 10 +++++----- docs/provider/Vault.md | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/README.md b/docs/README.md index 57686a41..d3ab1468 100644 --- a/docs/README.md +++ b/docs/README.md @@ -65,7 +65,7 @@ Addressing some common points people raise up, which are hidden in the docs. There are many questions if this isn't "fractional reserve banking" or such. This does use the same collateral to back multiple claims (staking), but -the [final invariant in the vault](https://github.com/osmosis-labs/mesh-security/blob/main/contracts/provider/vault/README.md#invariants) +the [final invariant in the vault](./provider/Vault.md#invariants) ensures there is sufficient collateral to cover the maximum loss (e.g. if all local and cross-staked validators double-sign). If the double slash penalty is 5%, you can safely cross stake 20x. @@ -81,13 +81,13 @@ to take over more than one-third, or even two-thirds or the power of a smaller c (e.g. $STARS) it is cross-staking on. Clearly the consumer chain wants to put some limits. The first limit is the -[discount applied during the conversion](https://github.com/osmosis-labs/mesh-security/blob/main/docs/consumer/Converter.md#price-normalization). +[discount applied during the conversion](./consumer/Converter.md#price-normalization). This doesn't just provide margin for price fluctuations but also means that on average a remote token has less voting power (and rewards) per USD-value than a local token, favoring local stakers. In addition, the Virtual Staking module enforces a -[max cap per provider](https://github.com/osmosis-labs/mesh-security/blob/main/docs/consumer/VirtualStaking.md#module). +[max cap per provider](./consumer/VirtualStaking.md#interface). This limits how many virtual tokens a given provider can stake. For computational reasons, it is defined as a number of tokens, not a percentage of stake, but looking at an average number of local tokens staked, the chain governance on @@ -118,9 +118,9 @@ For price increases, we have the max cap described under "Power Limits" which pl on the provider chain's influence in all circumstances, so this isn't a large problem. For a rapid price decrease, we must consider the time frame. -It is a [requirement of an oracle](https://github.com/osmosis-labs/mesh-security/blob/main/docs/consumer/Converter.md#price-feeds) +It is a [requirement of an oracle](./consumer/Converter.md#price-feeds) to provide timely feeds, say once a day or week, so we should focus on the relative price movement -in such a period. [The discount](https://github.com/osmosis-labs/mesh-security/blob/main/docs/consumer/Converter.md#price-normalization) +in such a period. [The discount](./consumer/Converter.md#price-normalization) provides such a buffer. If there is a discount of 40% and the provider tokens drop 30% relative to the consumer tokens in one oracle epoch, then it is still over-collateralized relative to voting power. If, however, it falls 60%, then it would only have 2/3 of the collateral locked on the provider diff --git a/docs/provider/Vault.md b/docs/provider/Vault.md index 75ddfc6d..8eb4a453 100644 --- a/docs/provider/Vault.md +++ b/docs/provider/Vault.md @@ -4,9 +4,9 @@ The entry point of Mesh Security is the **Vault**. This is where a potential staker can provide collateral in the form of native tokens, with which he or she wants to stake on multiple chains. -Connected to the _Vault_ contract, is exactly one [Local Staking contract](./LocalStaking.md) +Connected to the _Vault_ contract, is exactly one [Local Staking](./LocalStaking.md) contract which can delegate the actual token to the native staking module. It also can connect to an -arbitrary number of [External Staking contracts](./ExternalStaking.md) which can make use +arbitrary number of [External Staking](./ExternalStaking.md) contracts which can make use of said collateral as "virtual stake" to use in an external staking system (one that doesn't directly use the vault token as collateral). From 590de3e522faec7ce5d01f73532c24b4c6016055 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 13:46:43 +0200 Subject: [PATCH 41/43] Fix: s/Receiver/Provider/ --- docs/provider/DAOs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/provider/DAOs.md b/docs/provider/DAOs.md index ff639694..a8acc278 100644 --- a/docs/provider/DAOs.md +++ b/docs/provider/DAOs.md @@ -84,9 +84,9 @@ flowchart LR F -- $JUNO --> G[Native Staking]; ``` -Note this would require a different implementation for vault (to handle cw20), +Note this would require a different implementation for vault (to handle CW20), and likely a different "local staker" interface (you don't select validators, but rather unbonding time). -The "External Staker JUNO" would be similar to the normal [Receiver model](../consumer/Receiver.md), and +The "External Staker JUNO" would be similar to the normal [Provider model](../provider/Provider.md), and we will need a full implementation of the [Consumer side](../consumer/Consumer.md) implemented on the same chain. From 618f70f98c7cb30f8eff113ad1cfd0c28b7bf9f8 Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 13:51:22 +0200 Subject: [PATCH 42/43] Fix typos --- docs/consumer/VirtualStaking.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/consumer/VirtualStaking.md b/docs/consumer/VirtualStaking.md index f7e5fd45..91f4952d 100644 --- a/docs/consumer/VirtualStaking.md +++ b/docs/consumer/VirtualStaking.md @@ -21,7 +21,7 @@ they split a limited resource. The `Converter` should be able to call into the `Virtual Staking` contract with the following: ```rust -pub enum VirtualStakeExecMsg { +pub enum VirtualStakeMsg { /// This mints "virtual stake" if possible and bonds to this validator. Bond { amount: Coin, @@ -38,7 +38,7 @@ pub enum VirtualStakeExecMsg { The `Converter` should be able to query the following info from the contract: ```rust -pub enum VirtualStakeQueryMsg { +pub enum VirtualStakeQuery { #[returns(BondStatusResponse)] BondStatus { contract: String, @@ -52,7 +52,7 @@ pub struct BondStatusResponse { ``` Finally, the virtual staking contract should make the following call into the `Converter`, -which is send along with a number of `info.funds` in the native staking token: +which is sent along with a number of `info.funds` in the native staking token: ```rust pub enum ConverterExecMsg { From 9aef3bb6e2040191c8bbf8f910e6d70273831afb Mon Sep 17 00:00:00 2001 From: Mauro Lacy Date: Mon, 10 Jul 2023 14:01:02 +0200 Subject: [PATCH 43/43] Fix links / details --- docs/consumer/Consumer.md | 10 +++++----- docs/consumer/Converter.md | 2 +- docs/ibc/Overview.md | 4 ++-- docs/ibc/Staking.md | 6 +++--- 4 files changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/consumer/Consumer.md b/docs/consumer/Consumer.md index b022eb3e..5db9d3c4 100644 --- a/docs/consumer/Consumer.md +++ b/docs/consumer/Consumer.md @@ -1,11 +1,11 @@ # Consumer -The consumer side of the system receives "virtual tokens" from +The Consumer side of the system receives "virtual tokens" from some trusted providers and uses those to update the local staking weights. It then provides rewards to those providers in exchange for the security promise it receives. -This is the other half of [the provider flow](../provider/Provider.md). +This is the other half of the [Provider](../provider/Provider.md) flow. ```mermaid flowchart LR @@ -60,19 +60,19 @@ When the IBC connection is established between chains, a channel can be setup be and the [External Staking](../provider/ExternalStaking.md) contract on the Provider. The External Staking contract must be already instantiated with the proper IBC channel information (i.e. proper connection id and port id information in the `AuthorizedEndpoint` struct, set as part of their `InstantiateMsg`). -See the [Provider](../provider/Provider.md) Setup for more information. +See the [Provider Setup](../provider/Provider.md#setup) for more information. Also, see [IBC Deployment](../ibc/ControlChannel.md#deployment) for more information on how the IBC connection is established. ## Converting Foreign Stake -Not all providers are treated equally. (And this is a good thing) +Not all providers are treated equally (and this is a good thing). Each Converter accepts messages from exactly one provider and is the point where we establish trust. The Converter is responsible for converting the staking messages into local units. It does two transformations. This first is convert the token based on a price oracle. The second step is to apply a discount, -which captures both the volatility of the remote asset, as well as +which captures both the volatility of the remote asset, and a general preference for local/native staking. This is described more in depth under [Converter](./Converter.md#staking-flow). diff --git a/docs/consumer/Converter.md b/docs/consumer/Converter.md index e129af13..5c46e946 100644 --- a/docs/consumer/Converter.md +++ b/docs/consumer/Converter.md @@ -2,7 +2,7 @@ The Stake Converter is on the consumer side and is connected to an External Staker on the Provider side. This handles the normalization of the external tokens and _converts_ them into "Virtual Stake". -There is a 1:1 connection between a Converter and a [Virtual Staking Contract](./VirtualStaking.md) +There is a 1:1 connection between a Converter and a [Virtual Staking](./VirtualStaking.md) contract which handles the actual issuance. The converter is connected to the Provider chain via IBC and handles the various packets coming from it. diff --git a/docs/ibc/Overview.md b/docs/ibc/Overview.md index 438c1b17..6f186477 100644 --- a/docs/ibc/Overview.md +++ b/docs/ibc/Overview.md @@ -23,7 +23,7 @@ flowchart RL ## Deployment -As mentioned in [the consumer section](../consumer/Consumer.md), +As mentioned in the [Consumer](../consumer/Consumer.md) section, we need to establish a trust relationship between the Provider side and the Consumer side. This is done in multiple stages. I refer to "Admin" here in most of the setup, which can be any address that can make calls on two chains. @@ -47,7 +47,7 @@ to authorize them: can verify the contract code and configuration. 3. Consumer chain governance votes to authorize this _Virtual Staking_ contract to have special privileges on the staking system (see [Virtual Staking](../consumer/VirtualStaking.md)). -4. Authorization on the Provider chain [is not required](https://github.com/CosmWasm/mesh-security/blob/begin-architecture/docs/provider/Vault.md#design-decisions), +4. Authorization on the Provider chain [is not required](../provider/Vault.md#design-decisions), but the default cross-stake frontend application should add the _External Staker_ to the recommended list. Once this has been completed, everything is set up and the token holders on the Provider side diff --git a/docs/ibc/Staking.md b/docs/ibc/Staking.md index e98a5f69..5308cda9 100644 --- a/docs/ibc/Staking.md +++ b/docs/ibc/Staking.md @@ -13,7 +13,7 @@ We also want to guarantee that the Provider chain always maintains sufficient st in the vault to cover all virtual staking actions currently outstanding on the Provider chain. As [mentioned before](./ControlChannel.md#channel-ordering), we wish to use an unordered channel, -and therefore must bring a degree of understanding of [Serializability](./Serializability.md) +and therefore must bring a degree of understanding of [serializability](./Serializability.md) to this protocol. ## Delegation Syncing @@ -84,7 +84,7 @@ Possible keys with conflicts are: ### Identifying Potential Commutability The general design should be to write all changes only on a successful ACK, but hold any locks needed to ensure those -writes will not fail in any condition. Using the approach of [Value Ranges](./Serializability.md#value-ranges), let us analyze +writes will not fail in any condition. Using the approach of [Value Ranges](./Serializability.md#value-range), let us analyze what needs to be minimally enforced here. For staking, we update: @@ -259,4 +259,4 @@ do_stake(); ### Error correction -**TODO** Ideas about using values in success ACKs to double check the state matches expectations and flag possible errors +**TODO**: Ideas about using values in success ACKs to double check the state matches expectations and flag possible errors.