Enable use of application-specific extension types

[davalapar]

Overview

• Transforms class instances into lower-level data structures upon encode, then back to class instances upon decode
• On encoding, class instances are identified / recognized by using for-loop + instanceof
• Internally automates the assigning of application-specific extension types (0 to 127)
• Two variants:
  • class instances - when a class instance is passed directly to our encoder function
  • nested class instances - when a class instance is nested within an object or array passed to our encoder function
• Uses an internal boolean, to check if custom extension types are used
• Tests

const { extend } = MessagePack.initialize(2 ** 24);

class Vector {
  constructor(x, y) {
    this.x = x;
    this.y = y;
  }
}

// transforms class instance to msgpack-compatible data
const encodeVector = (vector) => {
  return [vector.x, vector.y];
};

// transforms data into new class instance
const decodeVector = (data) => {
  return new Vector(data[0], data[1]);
};

extend(Vector, encodeVector, decodeVector);

// `extend` parameters:
// - class (we use instanceof for recognition) : `Object`
// - encodeTransformFunction : `Function`
// - decodeTransformFunction : `Function`

Algo

• For each classes
  • if value instanceof class
    • apply encodeTransformFunction
      • return encoded

---

Another use case:

• A User class instance in client-side can be automatically encoded only using its id property
• In the server-side its received id property can be automatically transformed back into a User class instance
• User -> id -> encode -> transport -> decode -> id -> User

---

References

• Support for custom encoders darrachequesne/notepack#18
• https://github.com/kawanet/msgpack-lite#custom-extension-types-codecs
• https://github.com/msgpack/msgpack/blob/master/spec.md#extension-types

---

Breakdown

• Uses fixext and ext format

fixext 1	11010100	0xd4
fixext 2	11010101	0xd5
fixext 4	11010110	0xd6
fixext 8	11010111	0xd7
fixext 16	11011000	0xd8

fixext 1 stores an integer and a byte array whose length is 1 byte
+--------+--------+--------+
|  0xd4  |  type  |  data  |
+--------+--------+--------+

fixext 2 stores an integer and a byte array whose length is 2 bytes
+--------+--------+--------+--------+
|  0xd5  |  type  |       data      |
+--------+--------+--------+--------+

fixext 4 stores an integer and a byte array whose length is 4 bytes
+--------+--------+--------+--------+--------+--------+
|  0xd6  |  type  |                data               |
+--------+--------+--------+--------+--------+--------+

fixext 8 stores an integer and a byte array whose length is 8 bytes
+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
|  0xd7  |  type  |                                  data                                 |
+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+

fixext 16 stores an integer and a byte array whose length is 16 bytes
+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
|  0xd8  |  type  |                                  data                                  
+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
+--------+--------+--------+--------+--------+--------+--------+--------+
                              data (cont.)                              |
+--------+--------+--------+--------+--------+--------+--------+--------+

ext 8 stores an integer and a byte array whose length is upto (2^8)-1 bytes:
+--------+--------+--------+========+
|  0xc7  |XXXXXXXX|  type  |  data  |
+--------+--------+--------+========+

ext 16 stores an integer and a byte array whose length is upto (2^16)-1 bytes:
+--------+--------+--------+--------+========+
|  0xc8  |YYYYYYYY|YYYYYYYY|  type  |  data  |
+--------+--------+--------+--------+========+

ext 32 stores an integer and a byte array whose length is upto (2^32)-1 bytes:
+--------+--------+--------+--------+--------+--------+========+
|  0xc9  |ZZZZZZZZ|ZZZZZZZZ|ZZZZZZZZ|ZZZZZZZZ|  type  |  data  |
+--------+--------+--------+--------+--------+--------+========+

where
* XXXXXXXX is a 8-bit unsigned integer which represents N
* YYYYYYYY_YYYYYYYY is a 16-bit big-endian unsigned integer which represents N
* ZZZZZZZZ_ZZZZZZZZ_ZZZZZZZZ_ZZZZZZZZ is a big-endian 32-bit unsigned integer which represents N
* N is a length of data
* type is a signed 8-bit signed integer
* type < 0 is reserved for future extension including 2-byte type information

• https://github.com/msgpack/msgpack/blob/master/spec.md#ext-format-family

[davalapar]

BigInt support also related

kawanet/msgpack-lite#95

[davalapar]

https://github.com/kawanet/msgpack-lite#extension-types