Boost support for FVM deals

[jacobheun]

Summary

FIP Discussion 604 proposes introducing the ability for storage deal proposal generation to occur on the FVM. In order for SP's to run these deals through existing flows there will be required updates to Lotus (currently in progress) as well as Boost.

Proposal

• Boost SP's will have the ability to configure Boost to accept inbound FVM deals AcceptContractDeals: true.
  • Optionally: providing FVM support exists, SP's would be able to limit to specific topics.
• When AcceptContractDeals is true, Boost will subscribe to DealProposalCreated events via the Lotus RPC api
• When valid events are detected (TODO - define validity), Boost will perform an eth_call via the Lotus RPC api to getDealProposal
  • If the proposal includes an http hosted car file, Boost will automatically initiate the retrieve and progress the deal
  • If the proposal includes an unknown transport type, normal offline deal flows would be required by the SP
• Once data is downloaded, Boost will trigger a Propose Storage Deal (same flow as today) with client as per the params of the deal proposal
• The Storage Market Actor then calls the client address and authorizes the deal. (Lotus Web3 Client Contract contains source code on how deals proposed on the FVM would actually be authorized by client contracts, instead of authorized through client signature verification as it works today.)
• Once the deal has been authorized (i.e. PSD message is successful on-chain), Boost will pass the data to Lotus for sealing (existing flows apply)

Scenarios

DataDAO with HTTP hosted CARs for allowlisted SP's

1. DataDAO deploys a limited contract per Contract as a Deal Client Standard FIPs#604
2. DataDAO owner/author/actor calls makeDealProposal for their datadao01.car file hosted at https://my.data.dao/datadao01.car triggering a new deal event
3. Lotus sees the event which is passed off to the subscribed Boost instance
4. Boost sees the event and fetches the deal proposal from Lotus via eth_call
5. Boost performs an http fetch of https://my.data.dao/datadao01.car
6. Once downloaded, Boost starts the deal flow, causing authorization with the client to trigger per the above proposal via a PSD message on-chain.
7. On acceptance, the deal is passed to sealing.

DataDAO with custom data transfer (IPFS/Portable Drives/etc) for allowlisted SP's

1. DataDAO deploys a limited contract per Contract as a Deal Client Standard FIPs#604
2. DataDAO owner/author/actor calls makeDealProposal with metadata clarifying how the SP can manually retrieve the data (if not already negotiated as part of a partnership)
3. Lotus sees the event which is passed off to the subscribed Boost instance
4. Boost sees the event and fetches the deal proposal from Lotus via eth_call
5. Boost, not recognizing the transport, marks the deal for Awaiting Offline Transfer
6. The SP fetches (manually or scripted) the offline data and triggers the next phase of the deal with Boost
7. Boost continues the deal flow, causing authorization with the client to trigger per the above proposal via a PSD message on-chain.
8. On acceptance, the deal is passed to sealing.

Outstanding questions

• How is "racing" / front-running of SP's handled? If 2 SP's submit proposals and only 1 is accepted, we need to clarify this flow.
• How are SP's filtering events for just the contract owners they accept? How are they managing that list?
• The flow above assumes that a location reference is part of the Deal Proposal struct and we get it via the eth_call read-only. Is this where the location reference will be stored?

Dependencies

• Contract as a Deal Client Standard FIPs#604 will need to include a location/transport reference for http retrieval to work. Ideally there should be some metadata to clarify how offline deals can be performed, but this may be negotiated offline by SP's and contract author partnerships.
• Functional eth_ namespace on Lotus (namely eth_call and eth_subscribe) so that Boost can receive events and do read-only JSON RPC calls to fetch VM state as described above.
• We will need to setup a devnet environment to properly test the end to end flow here, and will need support for the FVM and lotus functionality to be in place (even if in branches).

References

• FIP Discussion 604

[jennijuju]

AcceptFVMDeals

quick bike shedding, I feel like AcceptContractDeals is a better name - in the end of the day fvm is just a VM that executes the messages, it doesn't distribute deals or per se.

[jennijuju]

Boost will subscribe to DealProposalCreated events via the Lotus RPC API

As in ALL contracts that emit this event? Will SP get to chose which ones they'd like to subscribe (deal with?)

Boost sees the event and fetches the deal proposal from Lotus via eth_call

So this assumes we stores whole deal proposal in the contract and SP will poll. Curious why we didn't consider to have the event has enough deal proposal info and boost can just get them from event, without pulling from the contract again? (curious about the constraints, gas usage and contract statescalability)

[jennijuju]

Boost will trigger a Propose Storage Deal with client as per the params of the deal proposal

this is just sending PSD correct?

[jennijuju]

(Also note, this flow should work for data contracts that do but are not limited to DataDAO, simple deal client, data bounty, DataDao, perpetual storage, storage automation and so on.

[jennijuju]

How is "racing" / front-running of SP's handled? If 2 SP's submit proposals and only 1 is accepted, we need to clarify this flow.

Good q and i would argue different data platform will have different mechanism - we did some research on this problem here https://www.notion.so/pl-strflt/FilFil-PeSto-and-Filecoin-Archive-47ba9313222c4d86ac0539147b781a6c?p=9b63d39ac97240ce843b24d6b0ec994c&pm=s

[jennijuju]

The flow above assumes that a location reference is part of the Deal Proposal struct and we get it via the eth_call read-only. Is this where the location reference will be stored?

AFAIK the location ref can also be a part of the event - which likely means you don't need eth_call to get it either? Event are then stored as a part of the message receipts.

It seems to me that we should prioritize on deciding if we should take the event only approach OR a event+getter approach.

[LaurenSpiegel]

At the risk of further bike shedding, should we default to listening for these events? So, the configuration would actually be something like IgnoreContractEvents and if the SP wants to ignore they set this to true.

[jacobheun]

quick bike shedding, I feel like AcceptContractDeals is a better name

Fine by me, not strongly held to the names. AcceptFVMDeals changed to AcceptContractDeals.

As in ALL contracts that emit this event? Will SP get to chose which ones they'd like to subscribe (deal with?)

This is listed as an outstanding question. It's still not clear what that stream will look like, how SP's can subscribe to certain topics, etc. Ideally, I imagine SP's are going to want to heavily filter this to relevant topics, but there needs to be some product work done to figure out what those end to end flows look like.

So this assumes we stores whole deal proposal in the contract and SP will poll. Curious why we didn't consider to have the event has enough deal proposal info and boost can just get them from event, without pulling from the contract again?

@nonsense may have thoughts on this? I'm not currently clear on the benefits/drawbacks of having the events hold all information, versus them being lighter and requiring a fetch aside from a lighter resource load to hold an event cache, but this is likely pretty small.

this is just sending PSD correct?

Yeah standard PSD flows that exist today. Clarified in the description.

(Also note, this flow should work for data contracts that do but are not limited to DataDAO, simple deal client, data bounty, DataDao, perpetual storage, storage automation and so on.

100%. Current scenarios listed are just ones that I've heard numerous times, so using them as base use cases.

Good q and i would argue different data platform will have different mechanism

The main question we have here is that response is consistent. The error messaging can vary based on the mechanism, but being clear about what a denial (racing or otherwise) would be helpful, especially considering SP's will have had to download the data before getting to this step. If there's a high fail rate here, there is little incentive for them to burn resources fetching the data.

It seems to me that we should prioritize on deciding if we should take the event only approach OR a event+getter approach.

Agreed, we need to clarify the flow here. I'd love to get the interface calls between Lotus and Boost clarified, as this should be helpful in sussing out any potential missing bits.

At the risk of further bike shedding, should we default to listening for these events? So, the configuration would actually be something like IgnoreContractEvents and if the SP wants to ignore they set this to true.

@LaurenSpiegel for that to happen I think we really need to answer the event filtering problem here. If it's default, that means SP's would get ALL events. If contract deals are restricted to certain SP's, negotiated out of band, they could be getting events they can't make deals for. I'd really like to see clarity on what the end event stream looks like for SP's before defaulting to true out of the gate. We're also going to need to sort out what event retention looks like for SP's as well, which could quickly balloon and become unmanageable if there is a flood of these potential deals they need to sift through.

[brendalee]

@jacobheun i assume listening to all events by default is more a noisy thing that happens, rather than a resource intensive thing?

[nonsense]

@nonsense may have thoughts on this? I'm not currently clear on the benefits/drawbacks of having the events hold all information, versus them being lighter and requiring a fetch aside from a lighter resource load to hold an event cache, but this is likely pretty small.

This is a rather small change on Boost side, depending how it is decided to be implemented. It is a standards question, and something to be discussed at filecoin-project/FIPs#604

At first thought, I'd probably opt for an event that contains all information rather than keep the deal proposal data in state, but this is not what the discussion is currently proposing. Not sure if events have any limitation with respect to the size of the data that they could contain.

[jennijuju]

Good q and i would argue different data platform will have different mechanism

The main question we have here is that response is consistent.

Seems like we are getting at something like below is needed for client contracts:

• in policyOk (method name to be bikesheded later lolol) - we shall emit an event like PSDFailed with some predefined error codes for main scenarios like: enough copy made
  @ZenGround0 do you have any thoughts here?

At first thought, I'd probably opt for an event that contains all information rather than keep the deal proposal data in state, but this is not what the discussion is currently proposing. Not sure if events have any limitation with respect to the size of the data that they could contain.

This is the approach Im currently inclined to as well. Have the same concern on events limitation and im hoping to have some updates next week with the prototype shrenuj is working on.

[aashidham]

The hope with the current interface is to enable a wide variety of data dao / smart contract use cases, which is why we have an event emitted with an id that has to be called by Boost.

When an SP then calls getDealProposal , the client contract can then determine whether or not it makes sense to actually send over the deal proposal. This denial can be based on the number of SPs that might have already replied, the funds left in the contract, whether or not a vote passed and if another deal is expiring. All kinds of parameters can be coded to make getDealProposal return nothing depending on the current state of the contract.

This is why we want multiple SPs to respond if they are a fit for the emitted event. Only a subset may be picked with a non-null return.

---

On another note, the question about what we should contain in the event is a key one. See the key / open question in the proposal:

Right now, the DealProposalCreate event only contains an id. Is there more worth adding to the event signature? Keep in mind each additional topic needs to be 32 bytes at most. Having more descriptive events may allows SPs to filter events so that they can more selectively respond. Some ideas:
(a) MinerID: to let a participating SP filter by its own ID. This is valuable for clients to emit events that are specifically for a constrained set of SPs.
(b) string: a human readable string (of at most 32 characters) that describes the storage deal. Can be optionally filled by clients. Relevant strings can then be filtered by SPs. May lead to confusion / splintered topics since this is a freeform field. May need further standardization if included.
(c) pieceSize: the size of the deal. SPs can then filter by the size of deals they want to store.
(d) verifiedDeal: whether or not the deal is verified. SPs may not want to entertain unverified deals.

Would love the input of the Boost team on this question based on known Boost behavior. How do Boost SPs evaluate deals currently?

I am not currently a fan of including everything in the event, since that means a lot more PSDs will be submitted than the client can afford or meant to submit. The hope is that the emit (push) and the getter (pull) system allows for the PSD payload to be given only to the SPs that the client has approved to receive the payload.

Ideally SPs can filter the deal proposal based on the event parameters (see open question) and the client can filter out SPs based on their selective response to getDealProposal.

[nonsense]

I am not currently a fan of including everything in the event, since that means a lot more PSDs will be submitted than the client can afford or meant to submit. The hope is that the emit (push) and the getter (pull) system allows for the PSD payload to be given only to the SPs that the client has approved to receive the payload.

Ideally SPs can filter the deal proposal based on the event parameters (see open question) and the client can filter out SPs based on their selective response to getDealProposal.

Intuitively submitting an EVENT with a parameter specifying the SP that is approved to receive the payload seems suboptimal. One of the benefits of submitting deal proposals via EVENTS on chain is that it increases visibility, and more SPs can potentially execute these deals... so limiting the EVENT to just one SP seems off to me. I'd rather this heuristic happen elsewhere, so that there are better controls over what set of SPs is allowed to claim a deal proposal, rather than limiting directly in the EVENT.

Not only limiting in the EVENT means that the EVENT size will be larger as we need some parameter, but it also means that we would be spamming 3500 SPs and I have no idea how many other nodes with 1-to-1 messaging on chain...

[jennijuju]

I'm afraid I have to disagree here, i think a better approach here is for SP opt-in to listen to the contract they want to make a deal with.
1-1 messaging from the client side limits the use cases like replica worker, repair worker and deal renewals (or anything that works with miner pool)
This also adds on miner discovery to developers' plate.
(for clients who do prefer to specify the provider, they can add an additional topic - but i dont think that should be the generic frc we recommend)

[nonsense]

Seems like we are on the same page, and not disagreeing.

[aashidham]

The event log is inherently gossipy. Look at the Transfer event for an ERC20. This is an event that gets emitted across the chain for all clients to see even though it only affects the sender's and receiver's balances. This is the case for most events.

Also note -- you can specify a set of SPs but the client might be more lenient. If an enterprising SP responded to a deal proposal that it was not included in, the client might approve that request regardless.

However you are right that there might be better fields to include in the event parameters than a miner ID. Others are also listed as options.

[LaurenSpiegel]

Wouldn't we want deal size and fil being offered by client to store the data so we create a market here? Currently SP's are mostly relying on verified deal block rewards but I would think the fvm deals would be an opportunity to try to get payments for deals flowing.

[brendalee]

I think it would be neat to have some price parameter (how much the client contract would be paying for the deal)

[aashidham]

Incorporated this into the FRC proposal here: filecoin-project/FIPs#604 (comment)

Please have a look and comment.

[nonsense]

The event log is inherently gossipy. Look at the Transfer event for an ERC20. This is an event that gets emitted across the chain for all clients to see even though it only affects the sender's and receiver's balances. This is the case for most events.

I agree on this one. One caveat here is that usually events on Ethereum are used to show that "something happened" already. Here we are using events as a communication protocol.

Also note -- you can specify a set of SPs but the client might be more lenient. If an enterprising SP responded to a deal proposal that it was not included in, the client might approve that request regardless.

I just fail to see what's the value of having an event on chain if you are specifying a concrete SP -- you might as well just submit your proposal over libp2p if that's the case.

However you are right that there might be better fields to include in the event parameters than a miner ID. Others are also listed as options.

I think the value of deal proposal events on chain should be that they are most permissionless with a certain criteria / minimal requirements for SPs how could process these proposals.

---

As for emitting events similar to Transfer ERC20, in Filecoin that would be an event upon a successful PSD message.

[raulk]

The event log is inherently gossipy. Look at the Transfer event for an ERC20. This is an event that gets emitted across the chain for all clients to see even though it only affects the sender's and receiver's balances. This is the case for most events.

I agree on this one. One caveat here is that usually events on Ethereum are used to show that "something happened" already. Here we are using events as a communication protocol.

True, although you can think of this event as representing that an internal state transition happened in the DealProposer contract such that it wants to propose a deal, i.e. it's a publicly visible transition in some internal state machine.

Also note -- you can specify a set of SPs but the client might be more lenient. If an enterprising SP responded to a deal proposal that it was not included in, the client might approve that request regardless.

I just fail to see what's the value of having an event on chain if you are specifying a concrete SP -- you might as well just submit your proposal over libp2p if that's the case.

I think this is not too different from specifying a concrete recipient in a Transfer event? You never know who may be interested in such event -- a bookkeeping app, observability tools, a competing storage provider...

However you are right that there might be better fields to include in the event parameters than a miner ID. Others are also listed as options.

I think the value of deal proposal events on chain should be that they are most permissionless with a certain criteria / minimal requirements for SPs how could process these proposals.

I consider this one specific instance of a storage market -- where any SP can come and take a deal. More often than not, I see clients specifying a particular policy they want the SP to satisfy (e.g. a reputation score, a geography, a latency metric, etc.) Of course, on-chain contracts and oracles for these things don't exist yet given the ecosystem is nascent, but I can imagine deal proposal contracts running some logic to determine which of the "registered" miners satisfies the client policies and then targeting specific SPs that hae been selected through that match-making algorithm.

[aashidham]

Open questions thread:

Q1: How do we share the location of the data with SPs? (link)

• Proposal 1: Using the location ref in the DealProposal struct (link)
• Proposal 2: Using a string link in the DealProposalCreate event (link)
• Current proposal: Place the location of the data in the DealProposal struct as a string. A string seems more flexible and can be used with IPFS and other approaches. In this case, Boost not recognizing the transport == Boost not seeing https or http at the beginning of the string. The hope is to make the standard flexible and not necessarily coupled to libp2p or Boost directly.

Q2: How are SP’s filtering events for just the contract owners they accept? (link)

• Depends on the event signature we agree on. If we just use (id) in the event signature, SPs cannot filter like this. Some proposed event signatures here.
• Current proposal: Not clear. We should come to a consensus on the event signatures. Seems like the best approach for the prototype is just to have (id)?

Q3: What about if 2 SP’s submit proposals and only 1 is accepted?

• Current proposal: Soft acceptance + timeouts (ideas 3 + 4 described here) are likely the long term solution. However short term, we will likely see SP resources get wasted. The good news is that all participating SPs at launch that will be PL internal SPs, so we don’t need to immediately worry about SP resource waste. However, we do have ideas to mitigate this in the future.

[nonsense]

Q1: The current implementation is done with 2 additional fields to the DealProposal struct - Version and Params which are CBOR encoded. See: https://github.com/filecoin-project/boost/blob/main/storagemarket/types/contract_deal_proposal_types.go#L25-L33

So similar to your proposal 1, but not exactly.

Q2: No need for event filtering on the Ethereum Event signature at this point, but we have implemented an AllowList -- see https://github.com/filecoin-project/boost/blob/main/storagemarket/contract_deal_monitor.go#L99-L101

Q3: I don't have an opinion on this just yet.