Last updated: July 2nd, 2022
Noosphere is a massively-multiplayer knowledge graph. The technical pillars that Noosphere builds upon are:
Above this substructure, Noosphere gives users:
- Entry to a zero-trust, decentralized network of self-sovereign nodes
- Human-readable names for peers and their public content
- Local-first authoring and offline-available content with conflict-free synchronization
- A complete, space-efficient revision history for any content
- Coherence and compatibility with the hypertext web
You can think of it like a world-wide Wiki.
Subconscious user Bob authors a note about cats, formatted as Subtext:
| Bob's notebook · /cat-thoughts |
|---|
| I love cats. I love every kind of cat. |
Notes in Bob's Subconscious notebook have a corresponding slug (e.g., cat-thoughts), that can be used as anchors for linking across notes. Later on when Bob authors another note on a related topic, they include a link to their first note:
| Bob's notebook · /animal-notes |
|---|
| I have strongly felt /cat-thoughts Dogs are just okay |
By following the link, Bob or another reader can navigate from one note to another note within Bob's local notebook.
One day Bob meets Alice and they get to talking about their shared interest in animals. Bob discovers that Alice also uses Subconscious. They exchange contact details, and soon Alice is able view an index of Bob's public notes:
| Bob's notebook · Links |
|---|
| /animal-notes |
| /cat-thoughts |
| /... |
After reading through Bob's cat-thoughts, Alice decides to reference it for later review. Alice opens up a local note about animals and links to Bob's note from there:
| Alice's notebook · /awesome-animal-links |
|---|
| Here are some cool /zebra-facts I love that /skateboarding-dogs exist Bob sure has some @bob/cat–thoughts |
By following the link @bob/cat-thoughts, Alice can navigate from a note in her local notebook to a note in Bob's notebook.
Alice and Bob do not need to give custody of their personal data or their identities to a third party for this link to work. Nor do they need to rely on a blockchain-based ledger.
Let's break down how a link like @bob/cat–thoughts works. Please note that what follows contains some simplification and shorthand; outbound links to detailed references have been included for the curious reader.
When Alice and Bob exchanged contact details in the story above, the exchanged data included public keys (encoded as DIDs that represent their respective notebooks. Bob's public key was then recorded against a pet name in Alice's notebook:
| Alice's notebook · Names |
|---|
mallory => did:key:z6MktafZTREjJkvV5mfJxcLpNBoVPwDLhTuMg9ng7dY4zMAL |
bob => did:key:z6MkffDZCkCTWreg8868fG1FGFogcJj5X6PY93pPcWDn9bob |
As Alice updated her notebook to include Bob's name, a snapshot of her entire notebook at its latest state was recorded and condensed down to a short, unique ID: a content ID, or CID. Such a snapshot and corresponding CID is recorded for every update to every user's notebook (including Bob's).
Similarly, when Bob updated their animal-notes to include a link to cat-thoughts, a new snapshot of the note was recorded and a CID was computed for it. When Alice viewed the index of links in Bob's notebook, what she actually viewed was a mapping of note slugs to their CIDs:
| Bob's notebook · Links |
|---|
| /animal-notes => Cid(bafy2bza...3xcyge7s) |
| /cat-thoughts => Cid(bafy2bza...thqn3hrm) |
| /... |
Noosphere data is formatted in terms of IPLD and encoded in a low-overhead binary format called DAG-CBOR. Even though a full snapshot is recorded with every revision, new storage space is only allocated for the delta between any two revisions. This strategy is similar to how Git efficiently stores the delta between any two sequential commits.
A data structure that we call a Memo is used to pair open-ended header fields with a retained historical record of revisions to notebooks and their contents:
The properties of immutable data structures allow content to be edited offline for an indefinite period of time and safely copied to replicas without the risk of conflicts at the convenience of the client.
Every user who publishes to the network does so via a gateway server. The gateway represents the boundary edge of user sovereignty, and also gives the user a reliably available foothold in the Noosphere network. The owner of a notebook is also the owner of the gateway, and third-parties neither have or need direct access to it (even in managed hosting scenarios).
The owner of a notebook enables the gateway to publish the notebook to the network using a UCAN-based authorization scheme. UCANs establish a cryptographic chain of proof that allows anyone to verify in place that the gateway was authorized to perform actions such as signing content on the user's behalf (and without asking the user to share cryptographic keys with the gateway).
When the user updates their notebook, they replicate the revision deltas to the gateway over HTTP (as network availability allows), and also tell the gateway which CID represents the latest version of the notebook. The revision deltas are syndicated to the public network via IPFS. Then, the gateway publishes the latest revision CID to a DHT (see: Noosphere Name System) mapping it to the notebook's public key and pairing it with the UCAN proof chain.
After the gateway publishes an update to the DHT, it becomes possible for anyone who knows the public key of a notebook to discover the latest published revision of that notebook as a CID.
When that CID is known, it becomes possible for the entire notebook (or discrete parts of it) to be downloaded from IPFS and replicated to the local client of the reader.
Once the notebook is downloaded, any slug can be resolved to content using the slug-to-CID mapping recorded in the notebook's link index.
Here is Alice's notebook, visualized as simplified data structures; the content referenced by @bob/cat-thoughts is able to be resolved with a combination of metadata in Alice's notebook, and details published by Bob to the DHT:
