Document high level Lassie architecture

[kylehuntsman]

We want there to be documentation around the larger pieces of the application to help give context to how the application is generally structured to outside contributors. We understand that docs that are too in depth or are focused on the more technical complexity of certain parts of the application may become a maintenance burden over time as those items are updated. We would like to stick to higher level explainations of the larger aspects of the application and how they play a role in the system.

Here are some of the topics I feel we should document off the top of my head in no particular order. Feel free to suggest additional topics to discuss and how deeply we should document them.

• We have three top level user interfaces: CLI fetch, HTTP API via daemon, and programmatic library use
• How candidates are found
• We think about the retrieval process in phases
• There are implementations of retrievers that do the actual work of retrieval based on the protocol being used
• We emit events from each step of the retrieval process

[hannahhoward]

Some additional potential topics:

• Augmenting on the first item, I'd really like to offer baseline startup guides to each of the three use cases:
  • why and how would I use the CLI?
  • why and how would I use the HTTP api?
  • why and how would I use Lassie as a library?
  • these may make sense to live outside this repo in some official docs website (IPFS or Filecoin)
• How we deal with storage of data and build streaming CAR files (but don't hold a blockstore)
• A brief bit about how we work with boxo/bitswap