The Inter Module Reference System
Authored by DoHyung Kim
The objectives of this document are to present:
- The concepts in and the requirements for the inter-module reference system
- The design of the reference system for an implementation
In Mold, we need to have a notion of object references to link different modules and the Foundry host together. A reference in a module acts as a proxy to an actual object in another module. So modules and the host are weaved together by creating objects in some of modules and the host, taking the references to them, and passing the references to other modules and the host.
An object here is defined as a data that can be acted upon only via a well-defined operations, which are actually functions with access to its associated data. We usually call those functions “methods” to distinguish them from free standing functions. So we can call methods of an object in a module from another module, once we have a reference to the object.
Since modules in a Foundry app are isolated from one another and sandboxed, the method must be run in the module where the object belongs and return a result to the caller in another module. In this regard, the reference system is similar to IPC (Inter-Process Communication) mechanisms, though the actual implementation of references in Mold may or may not leverage an IPC mechanism.
As per RFC 2119:
- Modules written in different languages MUST be able to exchange references and invoke on them
- References MUST NOT be forgeable to access objects not passed explicitly to the caller
- SHOULD NOT preclude optimizations in method invocations enabled by various sandbox/runtime technologies
- SHOULD support a wide range of sandbox/runtime technologies
A reference must be inforgeable, which means a module must not be able to forge references to objects in other modules unless the references are passed to it explicitly. But it doesn’t necessarily mean a reference must be physically opaque and uninspectable (e.g. with a help from OS or application virtual machine like WASM runtime). A malicious module may try to inspect and modify raw bytes in a reference, but the resulting reference must not just result in another reference not shared explicitly to the malicious module. So it is completely ok for a forged reference to refer to another object if and only if a reference to it was already shared to the module forging a reference and still valid.
There may be multiple types of references. A type here is a specific implementation of references rather than just an abstract taxonomy, where different types of references may expose completely different interfaces to modules and the host. So a module or the host must have prior knowledge on how to manipulate references of a specific type. As a result, a module may or may not support a specific type of references, and the host only supports the base references.
Each module and the host has a limited set of reference types it supports. However all modules and the host are required to support the base references to ensure modules can talk with each other and the host. Then why do we need multiple types of references? Depending on the types of the sandboxes for module instances, there might be more efficient ways for mutual communication. For example:
- If two modules are run in separate sandboxed processes and the only IPC mechanism available is Unix domain socket, then we have no way but to serialize and deserialize messages exchanged between them.
- If two modules are run in a JVM isolated with each other via usual Java class loader mechanism, then it’s possible to pass Java object references between them without incurring the overhead of serializing and deserializing messages. But such references are only usable among modules running in the same JVM.
- When WASM will eventually support reference types and interface types, its native references will be passed among WASM modules and a module will directly call into another module without any intermediary, and it’s much more efficient than IPCs. In that case, only WASM modules may make use of such references. The WASM references will be just an incomprehensible piece of information to the modules in non-WASM sandboxes.
A pair of modules need to support at least one type of references in common to communicate with each other. The base reference type is the one that must be supported by every module and the host for the minimum interoperability among them. It is designed to be easily implementable across a variety of sandbox/runtime technologies.
The Foundry host is written in Rust. All the components directly interacting with the host must provide a safe Rust interface regardless of their main implementation languages. Such components are all considered trusted, meaning they must obey invariants set to them voluntarily.
The trusted components include:
- The Foundry host
- Sandbox providers and sandbox instances
- Reference managers
Throughout the design, CBOR (Concise Binary Object Representation) is used as the encoding of the messages exchanged among the components in the system. CBOR is chosen because it’s:
- simple
- reasonably compact
- supported in a wide range of programming languages
- easy to dynamically generated and manipulated without upfront code generation (from the host side)
The formats of all the messages exchanged shall be described separately in the form of CDDL and mapping of interface descriptions to CDDL.
A channel is a link between a pair of modules or of a module and the host, and specific to single reference type. Depending on the implementation of references, a channel may or may not encapsulate an actual medium through which messages flow. However, two sides of a channel shall be created with a common context, which is a pair of crossbeam channels in the case of the base references. Reference Scoping In addition to being an intermediary between two parties, a channel must act as a mechanism for providing a scope to which a set of references are confined. Each side of a channel must be careful not to expose objects shared in another channel. It’s an important requirement for preventing malicious or vulnerable modules from gaining capabilities not explicitly allowed for them. Also if references are transferred between two ends of a channel, the references must be bound to the channel. There must be no means for references exchanged via a channel to escape from the implicit scope the channel sets up.
When a channel is created, each side of the channel may be given a CBOR (Concise Binary Object Representation) message and/or a list of strings from the host. The message describes how to obtain references from the receiving end, and the list of strings represents what slots the references it receives to bind to. As a part of the channel initialization, each end must send the list of the so obtained references to the other end, if it is given a CBOR message, and should also bind references received from the other end according to the slot names, if it is given a list of strings. That way, the two sides bootstrap references to begin with.
Each type of references has its own reference manager. Its main roles are:
- Creation of a channel between two different sandboxes
- Providing services required by sandboxes in implementing references
When linking two modules:
- The host first determines the common types of references they support in common.
- Each module declares its relative preference for the reference types it supports, so the host chooses the most preferred reference type if there are multiple common types supported.
- The host calls out the corresponding reference manager, passes in the bootstrap messages, and has it establish a channel between the two modules. Note that the host is also the reference manager for the base reference type.
For some reference types, the corresponding reference manager may have things to do in transferring a reference to the other side. The subject requesting such a service will usually be a glue code inside a sandbox, and it may do so in band (that is, through the same means used for making method invocations) or out of band (through a special means provided to talk to the reference manager). As an example, if we wanted to perform imports of functions lazily on demand between two WASM modules, then the imports should be explicitly requested to the corresponding reference manager upon the first transfer of a reference to a new object, since those linkage usually requires the WASM host specific means.
Every module and the host must support the base reference type. Therefore it is necessary to clearly define how a module can implement support for the base reference type. Different from the other type of references, the host acts as the reference manager for the base references.
Every sandbox must be able to create a half of a channel from a pair of crossbeam Sender and Receiver for exchanging encoded CBOR messages. The two sides of a channel exchange messages via the crossbeam channels, and the sandboxes behind a channel may choose whatever mechanism they like to actually relay messages to/from them. For instance, an OS process-based sandbox may use file descriptors to send and receive messages. A V8 isolate based sandbox may use ArrayBuffer to take out and pass in messages. But they all can easily relay those message exchanges to the crossbeam channels. Threading Regardless of the multi thread support in each side of a channel, an invocation of a method on a reference may block depending on the nature of the method. And the receiving end of a method invocation may choose whatever policy in the threading model. It may serve all the invocations in a dedicated thread or from a pool of threads.