| CHIP Number | 0035 |
|---|---|
| Title | DataLayer Delegation Capabilities |
| Description | Allow datastore owners to add external puzzles with limited capabilities (writers, admins, oracles). |
| Authors | Michael Taylor, Yak |
| Editor | Dan Perry |
| Comments-URI | CHIPs repo, PR #125 |
| Status | Final |
| Category | Informational |
| Sub-Category | Chialisp Puzzle |
| Created | 2024-07-31 |
| Requires | CHIP-0005 (NFT1 Standard) |
| Replaces | - |
| Superseded-By | - |
The current DataLayer standard only allows the owner of a datastore to spend it. This proposal will add an optional layer enabling the owner to add trusted puzzles with limited capabilities over the datastore. Under the current specification, these puzzles can be classified as:
- Writers: Puzzles that are only allowed to update the datastore’s metadata (e.g., change the value of the store’s root hash).
- Admins: Puzzles that can update the store’s metadata and modify the authorized puzzles list (e.g., remove access of a writer or another admin).
- Oracles: Puzzles that allow the DataLayer store to be used as an oracle. If the functionality is enabled, anyone can spend the store in oracle mode as long as they pay a set amount of XCH to a designated address.
The owner puzzle is in full control of the datastore, meaning it can update the authorized puzzle list or turn off the functionality by removing the delegation layer at any time.
This CHIP also describes a new memo structure to be used by datastores, as well as three new (and optional) metadata fields: label, description, bytes (size).
The DataLayer technology has already enabled significant applications to run on the Chia blockchain, such as the Climate Action Data Trust (CADT) registry. By enhancing the standard to enable owners to delegate store access, we aim to broaden its applicability.
First, writers allow DataLayer technology to be easily integrated into CI/CD pipelines. A provider could then offer an experience similar to that of Netlify or Vercel, where developers simply push a site’s code to a repository, and the code is automatically compiled and uploaded to DataLayer. This would happen without the need for the owner puzzle to be used.
Second, admins allow for a much simpler interaction with DataLayer. Users can continue owning their stores while services listed as admins make managing the store’s permission simple. The owner no longer needs to worry about signing transactions for every change, and the delegated admin handles blockchain-related details such as transaction fees and confirmations in the background.
Third, we believe oracle capabilities will open up a range of new DataLayer applications. By allowing anyone to use the store as an oracle, data publishers can provide essential data while monetizing their store. Such applications include pay-for-inclusion offers that reference a datastore owned by neither party, or the ability to create offers with advanced conditions (e.g., offering 5 XCH for any Chia Friends NFT only if its rarity score is above a threshold in a trusted store).
Fourth, this proposal also opens up the opportunity for better store custody. By setting the owner to a cold wallet (or a DAO) and adding a hot wallet/trusted manager as an admin, users and organizations can allow for simpler day-to-day datastore operation while having the option to recover the store in case the admin is compromised. In this case, the store owner essentially becomes a backup with full control (and final say) over the store.
This proposal includes a new way to hint stores. The provided reference code can recognize the old memo format and follow current stores on-chain. It also supports upgrading these stores (i.e., adding the new metadata fields and/or creating a delegation layer).
At its core, DataLayer enables key-value data to be put into ‘stores’, which can be represented, tracked, and updated on-chain. To achieve this, a set of coins is followed on-chain to determine the store’s current root hash, as well as available mirrors. Currently, the puzzle of the coin that stores the root hash (the ‘datastore’) can be divided into three layers:
-
Singleton Top Layer: Coins on the Chia blockchain can only be spent once. The singleton top layer allows a series of sequential coins to have a unique ID by ensuring that:
- Every coin’s parent either a) had a singleton layer with the same id or b) is the launcher (a singleton’s id is just the launcher coin’s name)
- No more than one output coin has an odd amount
- The odd-coin output’s puzzle has been wrapped in a singleton layer (with the same ID)
Melting is an exception to the last two rules. It happens when a CREATE_COIN output condition from the inner puzzle has a special amount of -113. This stops the singleton top layer from re-creating itself, allowing the odd amount it holds (usually 1 mojo) to be used for other purposes.
-
NFT State Layer: This layer is also used in the NFT1 standard. It allows the coin to hold some metadata, which can only be modified by running a metadata updater puzzle. To trigger the update, the inner puzzle must emit a special condition with an opcode of -24, followed by the metadata updater puzzle [reveal] and its solution. The updater puzzle always returns 3 values: the new metadata to be stored by the coin, a new updater puzzle hash, and a list of conditions to be added to the layer’s output conditions list. DataLayer stores use a puzzle of ‘11’, meaning that the three output values are taken from the ‘solution’ argument of the special condition outputted by the inner puzzle. In other words, the inner puzzle fully controls the value for new metadata, new updater puzzle hash, as well as additional output conditions to be added by the NFT state layer.
It’s important to note that the NFT state layer employs a wrapping logic similar to that of the singleton top layer - i.e., it finds a single odd output CREATE_COIN condition and wraps it with a state layer puzzle.
-
Owner Inner Puzzle: The innermost puzzle is controlled by the store’s owner. Usually, this will be a ‘pay-to delegated puzzle or hidden puzzle’ puzzle, also known as the ‘standard’ puzzle. The puzzle allows two spend paths:
- The owner of a curried-in public key can sign the hash of a delegated puzzle, which is run with a solution provided within the same spend. Essentially, the owner of the corresponding secret (private) key has complete control over the output of this puzzle.
- A ‘hidden’ spend path can be triggered, where a puzzle hidden within the synthetic key can be revealed and evaluated. By default, current wallets set this puzzle to ‘(=)’, which raises an exception when run - effectively disabling this spend path even if the original public key is known.
The structure above is also referred to as NFT0 - it follows the structure of the NFT1 standard, but it does not have an ownership layer, which is mainly responsible for enforcing NFT trade royalties (and other transfer rules, if a custom transfer program is employed), as well as allowing DIDs to own NFTs by ‘stamping’ them. As previously mentioned, the metadata updater is ‘11’, not the puzzle usually used for NFTs - which restricts metadata actions (e.g., adding items to the image URLs list is allowed, but the image hash cannot be changed).
The core proposed change, which could be summarized as ‘allowing more than one puzzle to spend the same datastore’ does not affect a store’s need for uniqueness (provided by the singleton layer) or its ability to store metadata (provided by the NFT state layer). Therefore, the new chialisp code would only change the ‘owner inner puzzle’ layer of a datastore. Moreover, it was important for the feature to be optional, meaning that stores can opt not to delegate access to any puzzles. In that case, no layer or additional logic will be added to stores, thus saving on transaction costs. The owner of a store can choose to ‘upgrade’ the store at any time (add delegated puzzles - including the one that adds oracle functionality). The owner may also choose to downgrade the store, which requires removing all delegated puzzles and reverting to a ‘vanilla’ store. The latter case means that the owner can ‘exit’ or remove the delegation layer at any time, which is also important for future upgradability.
This CHIP also proposes a new way to hint the contents of stores when (re)creating them. This change was informed by the observation that stores currently hint information that can be inferred by parsing and evaluating the spend. More information about the new memo format can be found under ‘Specifications’.
Lastly, this CHIP also adds the option of allowing stores to have optional metadata fields such as label/description, which could be displayed on sites such as blockchain explorers. We’ve evaluated the following options:
- Adding the values as items in the key-value data. They would then be included in the root hash, which is present in the store’s metadata field. We’ve eliminated this option because we’d like explorers to be able to have this information without querying the off-chain data of the store.
- Adding the values in the kv_list of the singleton’s launcher. This would essentially mean that the values are announced when the store is created. We’ve eliminated this option because we’d like the values to be mutable (updateable).
- Announcing the values with a special
CREATE_COIN/REMARKcondition. We’ve eliminated this option because it adds the extra constraint that an observer would need to parse a store’s history to determine the latest field values. It would also be a ‘non-standard’ way of keeping values, especially given that stores have a metadata field. - Including the values as metadata fields. We’ve chosen this option for its simplicity and because it helps define a format where the metadata of a store contains more than one value (i.e., the root hash). For more information, please see the ‘Specifications’ section.
Under this CHIP, datastores will have the same two outermost layers - i.e., the singleton top layer and the NFT state layer. A standard store’s metadata updater puzzle will continue being ‘11’. The 3rd layer (i.e., the NFT state layer’s inner puzzle) will be decided as follows:
- If the store has no delegated puzzles, the owner’s puzzle (e.g., a ‘standard’ puzzle with the owner’s public synthetic key curried in) will be used.
- If the store has one or more delegated puzzles, the delegation layer defined below will be used.
The delegation layer is a new puzzle introduced in this CHIP. On a high level, it allows two spend paths: owner and delegated puzzle. Delegated puzzle hashes are stored in a Merkle tree, while the owner’s puzzle hash is directly stored as a curried-in value. Both options require revealing the puzzle, as well as a solution for that puzzle. Puzzle reveals are hashed and compared against the relevant values to ensure they have permission to run.
For the first spend path, the owner puzzle’s outputs are not parsed or modified in any way. The owner is expected to recreate the store and properly create memos when the delegate puzzles change.
For the second spend path, the output goes through a special morph_conditions function. This function prevents delegated puzzles from directly creating odd amount CREATE_COIN conditions by adding one such condition to the output. Since both upper layers (NFT state layer & singleton top layer) raise if more than one odd amount CREATE_COIN condition is created, this technique ensures that delegated puzzles cannot ‘take over’ the store (i.e., change its owner, exit the delegation layer or melt the store). Delegated puzzles can, however, output a special condition in the form of (list -13 [NEW_ROOT] [...memos]) to specify a new root corresponding to a tree that holds allowed delegated puzzle hashes. If no such condition is found, the current Merkle root will be used to recreate the coin. If more than one such condition is found, the last condition in the list will be used. The function also filters new metadata conditions, ensuring that delegated puzzles cannot ‘brick’ the store by changing the metadata updater.
The delegation layer puts two restrictions on delegated puzzles:
- A delegated puzzle may not output
CREATE_COINconditions: this constraint prevents most takeover attacks. Delegated puzzles may still control the list of allowed delegated puzzles via the special -13 condition. - A delegated puzzle may only update metadata, but not the metadata updater puzzle hash. This ensures that delegated puzzles cannot change the metadata updater puzzle to something that would affect the other puzzles’ (and the owner’s) ability to update metadata in the future.
An admin delegated puzzle is a delegated puzzle with no further restrictions on the output conditions. In other words, an admin can update the delegated puzzles list and metadata of a store.
A writer is a delegated puzzle further restricted by an outer layer called a filter. The writer filter does not allow ‘-13’ conditions. This means that the delegation layer will recreate itself with the same owner/delegated puzzles configuration. In other words, a writer can only update the metadata of a store.
While the reference implementation uses ‘standard’ inner puzzles, any inner puzzles can be used in conjunction with the filters above. This allows for richer ownership capabilities in the future - from secp256k1 wallet support to allowing vaults or DAOs to own or be delegated to stores.
This CHIP also defines a standard oracle delegated puzzle which allows anyone to spend the store provided they pay a fee in XCH to a designated address. This puzzle is structured as follows:
(list (list CREATE_COIN [oracle_fee_address] [oracle_fee]) (list CREATE_PUZZLE_ANNOUNCEMENT ‘$’))
The oracle puzzle does not have any filter. The driver code is responsible for reserving the oracle fee and for securing the spend by making sure the announcement is created.
At the moment, a datastore’s metadata is a list with a single item, the data’s root hash. This CHIP defines root_hash as a required item - i.e., the metadata of any store will begin with one 32-byte value that will be interpreted as the root hash. Aditionally, stores may store optional metadata fields in the form of (key . value). These optional fields can be used, for example, to give indexers additional information about the store. This CHIP introduces 3 such items defined below
| Key | Name | Description |
|---|---|---|
l |
Label | A label for the store. |
d |
Description | A description of the store. |
b |
Bytes | The size on disk, in bytes, required to keep the store's full data (incl. Merkle tree). |
For example. a store with a root hash of 0x00..00, a label of "Test Store", a description of "An ordinary store with extraordinary delegation capabilities" and a size of 1337 bytes would have the following metadata:
(list 0x00..00 ("l" . "Test Store") ("d" . "An ordinary store with extraordinary delegation capabilities") ("s" . 1337))
Note that these values are provided by the store owner/admins/writers. The blockchain does not enforce, for example, that the provided byte size matches the real size a store would take on disk. Datastores that have been launched before this CHIP are valid but do not provide values for label/description/bytes.
This CHIP highly recommends that a store’s label and description are kept as concise as possible. Because they’re kept in a store’s metadata, their values will be revealed with each spend, increasing overall spend bundle cost.
Memos are generally used to expose data that can be used to construct a relevant portion of the output coin’s puzzle. The hint is usually defined as the first memo (provided it is 32 bytes long), but the terms ‘memos’ and ‘hints’ are used interchangeably in this document.
When talking about datastores, two memo formats need to be specified. First, when the store is launched, the singleton launcher allows ‘memos’ to be provided via the kv_list solution item. In the current reference wallet/node implementation, this list is (list root_hash inner_puzzle_hash), where root_hash can be used to construct the NFT state layer (metadata only contains the root hash of the store) and inner_puzzle_hash is the tree hash of the NFT state layer’s inner puzzle. The singleton layer of the child can simply be determined using the singleton’s launcher ID, which is the name of the launcher coin (i.e., the coin being spent). This CHIP changes the kv_list format to (list metadata inner_puzzle_hash . dg_layer_memos), where dg_layer_memos is defined below. Hinting the full metadata instead of root_hash allows the metadata to have more fields.
Second, memos are an optional argument of CREATE_COIN conditions, which are used when a store ‘re-creates’ itself. In the current reference wallet/node implementation, the memo list is (list launcher_id root_hash inner_puzzle_hash). This CHIP will change it to (list launcher_id inner_puzzle_hash . dg_layer_memos) when the store's delegated puzzles are changed, where dg_layer_memos is defined below. The two values that are removed can be determined by off-chain computation:
root_hashcan be obtained by uncurrying the NFT state layer puzzle to obtain the current metadata and its inner puzzle. By running the inner puzzle with the corresponding solution, nodes/wallets can search for new metadata conditions to get the new coin’s metadata. If no such conditions are found, the metadata is the same as that of the coin being spent.inner_puzzle_hashcan be determined by examining the output of the NFT state layer’s inner puzzle being evaluated with its corresponding solution. Assuming the spend is valid, exactly one odd amount CREATE_COIN is guaranteed to exist and will contain the new coin’s inner puzzle hash (unless the amount is -113 - in which case the store is being melted).
The launcher_id is left as a hint (first memo) even though it can easily be obtained by uncurrying the parent's singleton layer. Clients can get a store's history by querying all coins with a hint that equals launcher_id, which is possible because full nodes index coin hints. This methods yields significant performance improvements when compared to the alterative, requesting coin records one-by-one.
It’s worth noting that software interested in just the root hash of the store can obtain it without being aware of dg_layer_memos. The dg_layer_memos list allows software to also obtain the list of delegated puzzles. This is useful when, for example, a client is trying to spend a store via the oracle path. The dg_layer_memos list is obtained as follows:
-
If the store has no delegated puzzles,
dg_layer_memosis empty (i.e., no memos are added) -
If the store has delegated puzzles, the list of delegated puzzles will be parsed and every puzzle will be transformed into memos as indicated in the table below
Puzzle Type Memos Admin 0x1 [puzzle_hash]Writer 0x2 [inner_puzzle_hash]Oracle 0x3 [oracle_fee_puzzle_hash] [oracle_fee]
For example, if the delegated puzzle list contains:
- an admin puzzle with a hash of
0xaa..aa, - a writer puzzle with an inner hash of
0xbb..bb, and - an oracle puzzle with a payout puzzle hash of
0xcc..ccand an oracle fee of 8 mojos (0x8),
The dg_layer_memos list will be (list 0x1 0xaa..aa 0x2 0xbb..bb 0x3 0xcc..cc 0x8).
The puzzles and reference drivers have been thoroughly tested. Note that tests cover both the code used to generate transactions, as well as the code used to parse transactions. The provided tests include:
- Tests for launching a datastore: Ensure that the driver code correctly generates transactions that launch new datastores, which are then correctly parsed to generate store information. Cases vary whether the datastore will use label/description/bytes and whether it will have an admin/writer/oracle delegated puzzle.
- Tests for admin transitions: Ensure that the driver code correctly generates transactions that let an admin delegated puzzle spend a store. Cases vary the initial datastore’s state (root hash, label/description/bytes & whether it has writer/oracle delegated puzzles) and the final datastore’s state (root hash, label/description/bytes & whether it has writer/oracle delegated puzzles, as well as whether the admin was removed, changed or stayed the same).
- Tests for owner transitions: Ensure that the driver code correctly generates transactions that let a store’s owner spend it. Cases vary the initial datastore’s state (root hash, label/description/bytes & whether it has admin/writer/oracle delegated puzzles) and the final datastore’s state (root hash, label/description/bytes & whether it has admin/writer/oracle delegated puzzles, as well as whether the owner was changed or stayed the same).
- Tests for writer transitions: Ensure that the driver code correctly generates transactions that let a writer delegated puzzle spend a store. Cases vary the initial datastore’s state (root hash, label/description/bytes & whether it has admin/oracle delegated puzzles) and the final datastore’s state (root hash, label/description/bytes).
- Tests for oracle transitions: Ensure that the driver code correctly generates transactions that let a store be spent in oracle mode. Cases vary the initial datastore’s state (root hash, label/description/bytes & whether it has admin/writer delegated puzzles).
- Tests for melting: Ensure that the driver code correctly generates transactions that let an owner to melt a datastore. Cases vary the initial datastore’s state (label/description/bytes & whether it has admin/writer/oracle delegated puzzles).
- Tests for admin empty root transitions: Ensure that the driver code correctly generates transactions that let a store’s owner spend it in the special case when an admin left an empty root (i.e., the admin removed all delegated puzzles so the list is empty - which could potentially trick the driver code into treating the store as a ‘vanilla’ one). Cases vary the initial datastore’s state (label/description/bytes & whether it has writer/oracle delegated puzzles) and the final datastore’s state (label/description/bytes & whether it has admin/writer/oracle delegated puzzles). They also vary the intermediary store’s metadata.
- Tests for old memo format: Ensure that the driver code correctly parses datastore launches and transitions that use the ‘old’ (pre-CHIP) memo format. Cases vary the initial datastore’s metadata and the final datastore’s state (root hash & whether it has a new owner)
- Test for filters: Ensure that the admin and writer filters correctly parse and (dis)allow special conditions. Tests vary the filter used and the condition being tested (
NEW_MERKLE_ROOT, new metadata) - Tests for the Merkle tree implementation: Ensure that the Rust implementation for Merkle trees returns the same Merkle root and proofs as the one in the reference wallet/node.
- Other tests: Ensure that the hardcoded puzzle hashes match the hardcoded puzzles and that curry_tree_hash functions are implemented correctly. Ensure that emitting a
CREATE_COINcondition from delegated puzzles (either to take over the store or to melt it) causes the datastore puzzle to raise.
The reference implementation can be found in the DataLayer-Storage/DataLayer-Driver repository, which powers the datalayer-driver NPM package. We plan to include the drivers in the chia-wallet-sdk toolkit.
The Chialisp code has been covered by tests (see ‘Test Cases’) and has been reviewed by people other than the developer. No professional audit firms have reviewed the code. Aside from possible vulnerabilities in the chialisp code, we see the following possible risks:
- Unsafe spends created by custom wallet implementations. This CHIP offers a reference implementation.
- Unsafe custom filters, which would potentially allow delegated puzzles to get admin access. This CHIP offers one filter (writer) for what we consider to be the most needed delegated capability, as well as a safe way to construct oracle delegated puzzles. It’s important to note that, if the underlying datalayer standard is modified (e.g., by changing the metadata updater puzzle or allowing one of the layers to have a new ‘magic condition’), the filters (and delegation layer) will need to be reviewed and potentially updated.
- Inner puzzle risk: The security of the delegated or owner puzzles must also be taken into account. This CHIP does not put any restrictions on the inner puzzles that may be used. Possibilities such as key loss or compromise, as well as chialisp vulnerabilities should be thoroughly considered as they affect the security of the whole store.
The reference implementation is released under the MIT License. chia-wallet-sdk released under the Apache 2.0 License. Rights to this CHIP waived through CC0.