Design and implement reflection

[RiscadoA]

Checklist

• Add Type class and reflection macros and functions #480
• Add ConstructibleTrait #481
• Add FieldsTrait #482
• Add ArrayTrait #483
• Add DictionaryTrait #484
• Define reflection for primitive types #538
• Define reflection for glm types #539
• Define reflection for std::vector<T> #540
• Define reflection for std::map<K, V> #541
• Define reflection for std::unordered_map<K, V> #542

Terminology

• By type-erased, I mean data whose type we do not know at compile-time. For example, receiving a void* to an object.

Motivation

Previous system

CUBOS. is currently lacking a way to easily inspect and modify the properties of a type. We have Packages, but there are some fundamental problems:

• You must have an instance of the type to know its fields: this makes impossible knowing the element type of an empty array!
• You cannot annotate type's fields with extra information.
• It only works with types which define both serialize and deserialize functions.
• It is slow! The instance must be first serialized to a package, and after modification, deserialized back.
• No support for types defined at runtime (scripting, for example).

Packages were added on top of the serialization system as a sort of make shift reflection: they take advantage of the serialization functions to get information about the types. But isn't this a bit awkward?

What do we need

It would make more sense to think in the opposite way: we provide runtime type reflection, which lets us know the properties of registered types. Then, serialization becomes a second thought - we could just implement it using the reflection data. No need to define serialize() and deserialize() - just expose the type information and its good to go.

So, here is a list of particular cases we must be able to handle:

1. push to an empty type-erased array
2. change the type of the value held by a type-erased variant.
3. inspect and modify a component's properties through the UI.
4. when spawning blueprints, we must somehow convert all the named Entity handles in the components into the spawned Entity handles.
5. when handling variants, it should be possible to switch the variant type given the runtime type, even if the type is unknown at compile time.
6. allow hooks to override the serialization behaviour of specific types. For example, we should be able to set a hook to (de)serialize entities as strings instead of an index and a generation (which are their actual fields).
7. annotate type information with extra user information - would be useful for UI stuff like specifiying a range for an integer value.
8. allow type-erased data to be compared to each other, essential for Implement save for SceneBridge #352.

Other random points.

• We should avoid static registration as it has caused problems before and its very easy to cause UB, instead, types should be registered manually (not that big of a problem, we are already registering components manually anyway).
• We should avoid a global registry of types - we will be registering types per Cubos application anyway, and will be easier to remove/add script types later.
• Bonus points if code generation becomes unnecessary!

Design

This would be a new module under the core library, within the new namespace core::reflection.

Type

Reflected types are described by instances of the Type class, which exposes the methods:

• name() const - user defined name for the type.
• has(const Type&) - checks if the type has the trait with the given type.
• get(const Type&) - gets a void pointer to the trait with the given type (aborts if !has).
• has<T>() const - non-type erased version of above.
• get<T>() const - non-type erased version of above.

It also provides the following methods, used to create, modify and destroy them.

• static create(name) - creates a type with the given name and returns a pointer to it.
• with(const Type&, const void*) - adds the given trait to the type. Must be move constructable or copy constructable.
• with<T>(T) - adds the trait T to the type, and returns a pointer to the type, for chaining.
• destroy() - frees the type.

Types are identified in the code through simple constant references to them.

Traits

Extra type information is stored in traits, which can be of any type. This makes the type system really extensible and flexible.

Initially, I was leaning towards a more OOP solution, where types could either be object types, array types, dictionary types, etc. But this has a problem: why are we forcing types to be of a single kind? An std::vector can also be seen as an object with three fields: a pointer, a size and a capacity. Why are we forcing a single view?

One other problem is that the base type class tends to become unnecessarily complicated: we'd have to add a virtual function for every comparison operator, getters for the size and alignment, etc, when probably most types won't even expose that information.

So the solution is - you can add the 'array trait' to your type, and also the 'object trait' or the 'equality trait' - the user of the reflected data chooses what to do with the information you give them.

Reflection function

Type information can be retrieved for any type which implements reflection using the global function reflect<T>().

Assigning reflection types to real types

Reflection is implemented for a type by either:

• adding the method static const cubos::core::reflect::Type& reflect() to the type, which should always return the same type.
• specializing struct Reflect<T> { static const cubos::core::reflect::Type& reflect(); } for the type.

The first approach is what the user will usually see. The second approach is used for external types for which we cannot add methods to: ints, std::vector<T>, etc. We can add macros to make the defining and declaring a method easily:

// fruit.hpp
struct Fruit
{
  CUBOS_REFLECT;
  std::string name;
};

// fruit.cpp
CUBOS_REFLECT_IMPL(Fruit)
{
    return Type::create("Fruit")
        .with(FieldsTrait::create().field("name", &Fruit::name));
}

Defining reflect for an external type then would look like:

// string.hpp
CUBOS_REFLECT_EXTERNAL_DECL(std::string);

// string.cpp
CUBOS_REFLECT_EXTERNAL_IMPL(std::string)
{
    return Type::create("std::string")
        .with(CopyConstructableTrait::create<std::string>())
        .with(MoveConstructableTrait::create<std::string>())
        .with(DestructableTrait::create<std::string>());
}

Custom traits

The dev can always define their own traits, which allows adding user info to the types themselves.

[RiscadoA]

🚀