Design for get_state_update

[tdelabro]

Following #1580 and the remove of the whole DA,settlement things, we are not storing state diffs of each block on disk anymore.
This was our way to answer the get_state_updates rpc route.
Which means we don't return anything anymore.

My intuition is that we can use re_exectute_transaction logic to reexecute the whole block and get the state diffs our of it.

On the other hand, it means that the SNOS input generation will be done in memory only, not storing anything, and just sending the SNOS input to the server in charge of it.
In case of failure of this server, it could be a good idea if we exposed a get_snos_input(block: BlockId) route, that would allow the entity responsible of proving to do some pooling rather than just be triggered by pushes from Madara

Wdyt?

[antiyro]

Based our personal experience with Deoxys:

We ended up storing key/value's, state diffs and the bonsai trie in 3 separate columns, this triplicate our key storage but it is the best way to keep harmony between storage size and execution latency

We're still exploring ways to exploit the bonsai lib at least for getStorageAt calls but the trie log currently occupies an inconceivable space

[tdelabro]

And reexecuting them is not that much. It’s just one block execution. Nobody expect hight frequency on this specific RPC. So I don’t really see reason to store those

[apoorvsadana]

Ya I agree with the re execution approach. The sequencer shouldn't bloat state with the state diffs, we can take a hit on get_stateUpdate performance. Full nodes should be the ones optimising for read call performance imo anyways, so as long as full nodes can sync quickly with Madara, they can store the state updates if they like and return it to users.

I am also in line with get_snos_input method. In fact, the current orchestrator design is pull based i.e. the orchestrator pulls state updates/snsos inputs from the sequencer. We don't need Madara to send anything. So this would work perfectly. Also, let me know if you've any views on this design.

[tdelabro]

Everybody agreed. I created #1595 and #1594 to define the work