feat: protocol rework + API overhaul (#164)

* feat: deserialize

* test: add performance tests to next protocol

* feat: wip

* feat: model serialization

* feat: patch start

* feat: wip

* feat: model work

* feat: hmm

* feat: work is never over

* feat: wip model migration

* feat: wip

* feat: use new model

* feat: compiles and tests pass

* feat: wip

* feat: wip

* feat: wip

* feat: wip

* feat: wip

* feat: observer fun

* feat: finish pack migration

* test: fix tests

* fix: protocol

* feat: remove existing networking example

* feat: effect names

* feat: work

* feat: work on nested properties

* feat: only set model when changed in observe effect

* docs: update readme

* test: fix tests

* fix: triggers

* feat: wip

* docs: proposed api

* feat: api improvements

* feat: remove createComponentType

* feat: make api more functional

* feat: update docs with new api

* docs: net api tweaks

* feat: more api tweaks

* docs: readme example

* feat: experimenting with observer api

* feat: observer.set

* feat: mut array method encoding

* feat: track* array methods and gc fixes

* perf: observer

* perf: update benchmarks

* test: fix tests

* docs: net docs

* feat: more docs

* feat(net): ambiently resolve component model

* docs: events

* docs: world ex

* feat: wip

* feat: dynamic field

* feat: useChanged experimentation

* docs: message producer scratch

* feat: selector initialization work

* feat: query.select

* feat: improve query signature

* feat: message producer progress

* docs: eff->use

* test: wip fixing tests

* feat: track package

* feat: smaller messages

* feat: start decode

* feat: message rewrite

* feat: producer

* feat: onPatch

* test: message_op tests

* test: encode tests

* feat: patch

* feat: remove component type

* feat: model -> core

* feat: observe stuff

* feat: set and push

* feat: remove ModelNodeKind

* feat: pop, shift, unshift, splice

* feat: remove->deleteCount

* docs: docs

* feat: hey it works

* feat: re-add component type size

* chore: begin migrating sanity check example

* feat: spin example

* feat: wip

* feat: eureka

* feat: test

* feat: re-add

* feat: exclude entities added/removed same frame from monitor

* test: fix tests

* feat: camelize

* docs: spin readme

* docs: more message producer docs

.gitignore

@@ -117,3 +117,4 @@ dist
 
 .DS_Store
 zola
+docs-src/public

.yarnrc

@@ -0,0 +1,5 @@
+# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
+# yarn lockfile v1
+
+
+yarn-path ".yarn/releases/yarn-1.22.10.cjs"

README.md

@@ -23,17 +23,20 @@ Visit https://javelin.games
 
 ## Packages
 
-| Package                                        | Description                                          |
-| ---------------------------------------------- | ---------------------------------------------------- |
-| [@javelin/ecs](./packages/ecs)                 | Build games using the ECS pattern                    |
-| [@javelin/net](./packages/net)                 | Synchronize `@javelin/ecs` instances                 |
-| [@javelin/hrtime-loop](./packages/hrtime-loop) | Create a smooth, high-resolution game loop in NodeJS |
+| Package                                        | Description                                        |
+| ---------------------------------------------- | -------------------------------------------------- |
+| [@javelin/model](./packages/model)             | Core types, utilities, and data model              |
+| [@javelin/ecs](./packages/ecs)                 | Build games using the ECS pattern                  |
+| [@javelin/net](./packages/net)                 | Synchronize ECS worlds                             |
+| [@javelin/pack](./packages/pack)               | Convert objects to and from binary arrays          |
+| [@javelin/track](./packages/track)             | Record component mutations                         |
+| [@javelin/hrtime-loop](./packages/hrtime-loop) | Create smooth, high-precision game loops in NodeJS |
 
 ## Examples
 
-| Example                             | Description                                               |
-| ----------------------------------- | --------------------------------------------------------- |
-| [networking](./examples/networking) | Entity/component synchronization over WebRTC datachannels |
+| Example                 | Description                                               |
+| ----------------------- | --------------------------------------------------------- |
+| [spin](./examples/spin) | Entity/component synchronization over WebRTC datachannels |
 
 ## Scripts
 

docs-src/config.toml

@@ -11,12 +11,12 @@ compile_sass = true
 build_search_index = true
 
 highlight_code = true
-highlight_theme = "snow-light"
+# highlight_theme = "snow-light"
 
 
 [extra]
 # Put all your custom variables here
 theme = "book"
 book_numbered_chapters = true
-library_version = "0.21.0"
+library_version = "1.0.0"
 repo_url = "https://github.com/3mcd/javelin"

docs-src/content/ecs/_index.md

@@ -5,7 +5,7 @@ sort_by = "weight"
 insert_anchor_links = "right"
 +++
 
-This section aims to serve as a quick primer on Entity Component Systems (ECS) and how to think in ECS. The goal is not to belittle other methods of building games or make ECS seem like a panacea, because ECS does come with its own challenges. Godot has a great article about [why Godot does not use ECS](https://godotengine.org/article/why-isnt-godot-ecs-based-game-engine) that I recommend you read if you are trying to determine whether or not you should use Javelin.
+This section aims to serve as a quick primer on Entity Component Systems (ECS) and how to think in ECS. The goal is not to belittle other methods of building games or make ECS seem like a panacea, because ECS does come with its own challenges. Juan Linietsky wrote a great article about [why Godot does not use ECS](https://godotengine.org/article/why-isnt-godot-ecs-based-game-engine) that I recommend you read if you're trying to determine whether or not you should use Javelin.
 
 <aside>
   <p>
@@ -15,18 +15,18 @@ This section aims to serve as a quick primer on Entity Component Systems (ECS) a
 
 ## Building a Game
 
-A best practice in OOP game development is to favor composition over inheritance when designing game data and behavior. Take the following example, where a `Player` class accepts `Body` and `Input` objects to enhance players with physics properties and input control:
+A best practice in OOP is to favor composition over inheritance. Take the following example, where a `Player` class accepts `RigidBody` and `Input` objects to enhance players with physics properties and input control:
 
 ```ts
-class Body {
+class RigidBody {
   readonly velocity = { x: 0, y: 0 }
 }
 
 class Player {
-  private body: Body
+  private body: RigidBody
   private input: Input
 
-  constructor(body: Body, input: Input) {
+  constructor(body: RigidBody, input: Input) {
     this.body = body
     this.input = input
   }
@@ -42,7 +42,7 @@ class Player {
   }
 }
 
-const player = new Player(new Body(), new Input())
+const player = new Player(new RigidBody(), new Input())
 
 setInterval(() => {
   player.update()
@@ -51,11 +51,11 @@ setInterval(() => {
 
 When the player presses the spacebar on their keyboard, `player.jump()` is called, and the physics body jumps! Easy enough.
 
-What if a player wants to spectate our game instead of controlling a character? In that scenario, it would be unnecessary for `Player` to care about `Body`, and we'd need to write code that makes `Body` an optional dependency of `Player`, e.g.
+What if a player wants to spectate our game instead of controlling a character? In that scenario, it would be unnecessary for `Player` to care about `RigidBody`, and we'd need to write code that makes `RigidBody` an optional dependency of `Player`, e.g.
 
 ```ts
 class Player {
-  private body?: Body
+  private body?: RigidBody
   ...
   jump() {
     this.body?.velocity[1] += 1
@@ -67,7 +67,7 @@ If there are many states/dependencies a player can have (e.g. spectating, drivin
 
 ## What's an ECS?
 
-Data and behavior are separate concerns in an ECS. There are three main parts to an ECS: **components** – game data, **entities** – game objects (like a tree, chest, or spawn position), and **systems** – game behavior. As we'll see, this architecture enables runtime composition of behavior that would be tricky to implement in the example above.
+Data and behavior are separate concerns in an ECS. There are three main parts to an ECS: **components** – game data, **entities** – game objects (like a tree, chest, or spawner), and **systems** – game behavior. As we'll see, this architecture enables runtime composition of behavior that would be tricky to implement in the example above.
 
 ### Components
 
@@ -93,12 +93,12 @@ for entity of (player, input, body)
     body[entity].y += 1
 ```
 
-This example shows a system which iterates all components that have a `Player`, `Body`, and `Input` (e.g. a gamepad) component. Each player's input component is checked to determine if the jump key is pressed. If so, we locate the entity's body component and add to it's y-velocity.
+This example shows a system which iterates all components that have a `Player`, `RigidBody`, and `Input` (e.g. a gamepad) component. Each player's input component is checked to determine if the jump key is pressed. If so, we locate the entity's body component and add to it's y-velocity.
 
-Spectators can now be represented with a `(Player, Input)` entity. Even though they aren't controlling a physics body yet, the `Input` component might allow them to move the game camera around. If the player chooses to enter the fray, we can insert a `Body` component into their entity, allowing them to control an actor in the scene.
+Spectators can now be represented with a `(Player, Input)` entity. Even though they aren't controlling a physics body yet, the `Input` component might allow them to move the game camera around. If the player chooses to enter the fray, we can insert a `RigidBody` component into their entity, allowing them to control an actor in the scene.
 
 ```ts
-add(entity, Body)
+add(entity, RigidBody)
 ```
 
 This pattern can be applied to many types of games. For example, an FPS game might consist of systems that handle physics, user input and movement, projectile collisions, and player inventory.

docs-src/content/ecs/change-detection.md

@@ -3,30 +3,107 @@ title = "Change Detection"
 weight = 9
 +++
 
-Javelin implements a very basic change detection algorithm using [Proxy](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy) that can observe deeply nested changes made to components.
+Change detection is often useful, but very difficult to do performantly. Javelin does not currently implement a change detection algorithm that automatically "watches" component mutations. Proxies and setters are slow, and the only way to fit in potentially tens (or hundreds) of thousands of tracked changes per 16ms tick is to manually write changes to a cache.
 
-Change detection is very useful, but difficult to do performantly; therefore, **components are not observed by default** to achieve good baseline performance.
+Fortunately for us, a change cache can be represented as a component! You can attach a component to an entity that stores changes using the built-in `ChangeSet` component type:
 
-## Techniques
+```ts
+import { ChangeSet } from "@javelin/track"
+world.attach(entity, ChangeSet)
+```
+
+## Tracking Changes
+
+Retreive an entity's change set just like any other component type: using queries.
+
+```ts
+const qryTrackedBodies = createQuery(Position, Velocity, ChangeSet)
+const sysTrack = () => {
+  qryTrackedBodies((e, [p, v, changes]) => {
+    // ...
+  })
+}
+```
+
+`@javelin/track` exports functions that correspond to various object mutations, like `set` for property assignment:
 
-The `world.getObserved` method returns a copy of a component that will notify the world when its data changes. It's important to remember to use this method when you want to use one of the change detection techniques outlined below. Bugs can arise in your game when you expect a component to be observed but you forgot to manipulate an observed copy.
+```ts
+import { track } from "@javelin/track"
+qryTrackedBodies((e, [p, v, changes]) => {
+  set(p, changes, "x", p.x + v.x)
+  set(p, changes, "y", p.y + v.y)
+})
+```
 
-### Observing
+These functions both perform the specified operation and record the change to the `ChangeSet` component. They also handle paths to deeply nested properties.
+
+```ts
+const sword = 55
+set(inventory, changes, "bags.0.1", sword)
+```
 
-If you want to know exactly what changes were made to a component during the current tick, use `world.getComponentMutations`. This method returns a flattened array of changes made to a component. Take the following example:
+`set` overwrites the previous changes made to the same key. This means they only hold onto the most recent changes made to a component.
 
 ```ts
-for (const [entity, position, input] of queries.vehicles) {
-  const observedPosition = world.getObserved(position)
+const bow = 56
+set(inventory, changes, "bags.0.1", sword)
+set(inventory, changes, "bags.0.1", bow)
+```
 
-  observedPosition.x = 2
-  observedPosition.y = 3
-  observedPosition.extra.asleep = true
+In the above example, the entity's `ChangeSet` component would look like:
 
-  world.getComponentMutations(position) // -> ["x", 2, "y", 3, "extra.asleep", true]
+```ts
+{
+  object: {
+    "bags.0.1": {
+      value: 56,
+      record: {
+        field: 2,
+        traverse: ["0", "1"],
+      },
+    },
+  },
+  array: [],
 }
 ```
 
-## Networking
+### Resetting Changes
+
+```ts
+import { reset } from "@javelin/track"
+reset(changes)
+```
+
+### Array Mutations
+
+A `ChangeSet` can also track common array mutations, like push and pop:
+
+```ts
+import { push, pop } from "@javelin/track"
+push(inventory, changes, "bags.0", sword)
+pop(inventory, changes, "bags.0")
+```
+
+```ts
+{
+  array: [
+    {
+      method: 0,
+      value: 55,
+      record: {
+        path: "bags.0",
+        traverse: ["0"],
+      },
+    },
+    {
+      method: 1,
+      record: {
+        path: "bags.0",
+        traverse: ["0"],
+      },
+    },
+  ]
+}
+```
 
-`@javelin/net` uses this change detection algorithm to optimize packet size by only including the component data that changed during the previous tick in network messages. This means that changes made to unobserved components will not be sent to clients.
+Other functions include `splice`, `shift`, and `unshift` for arrays, `add` and `delete` for sets and maps.