The Projectionist: Projector management and versioning system

If you are building an EventSourced/CQRS system in PHP, you need a solid system to handle building your projections. Enter the "projectionist".

This is a library that makes it easy to consume events and manage the lifecycle of projectors in PHP, keeping track of where each projector is in the event stream. It is currently a WIP so I wouldn't advise using it right now, but feel free to have a look around (and read this readme, lots of details here).

It's based on a lot of trial and error from building these kinds of systems, so my hope is that it will allow others to leapfrog us and gain from our mistakes.

How it works

The projectonist is given a collection of projectors. It can either boot or play these projectors.

Booting projectors prepares new projectors for launch. If they're run_from_launch, they will just get set to the latest event, and no events will be played. Other projectors will be played up to the most recent event. This should be run as part of your release script, ensuring that all new projectors and played and up to date before you make the app live.

Playing projectors takes all the active projectors that aren't run_once, and plays them to the latest event. This is intended to run in the background when the system is live, constantly listening for new events and attempting to apply them.

How to use this library

This how you create a projectionist.

// Define the config for the projectionist system
$config = new Projectionist\App\ConfigFactory\InMemory(); 

// Create the projectionist 
$projectionist = new Projectionist($config); 

// Load all your projectors into an array of Projector references
$projectors = [new RunFromLaunch, new RunFromStart];

// Boot the projectors
$projectionist->boot($projectors);

// Play the projectors
$projectionist->play($projectors);

That's it. The tricky part is in the details though. To use this library, you must write integrations for your system. We don't know what how you've implemented your system, so instead we've made it easy to integrate with this system, no matter what your implementation.

To do this, you have to implement the config for the projectonist.

Config is an interface that outputs three adapters and one strategy, these also need to be implemented.

• Adapters:
  • EventLog - Check if there are events, get the latest event, or get a stream of events after an event ID.
  • EventStream - Get events one at a time, until there are none left
  • EventWrapper - Wraps your event, you just need to implement how you get the id, so the projectionist can keep track of the projectors position.

The adapters act as integration points to your application, allowing your system and the projectionist to work together, no matter what the implementation.

You can also choose to override the default event handler by defining your own EventHandler Strategy.

• Strategy:
  • EventHandler - Play an event into a projector. Simple to write, gives you full flexibility.

It allows you define how your projectors work. By default it uses a class name based handler, which handle projectors with this handler style.

<?php 

use Domain\Selling\Events\CartCheckedOut;

class Projector 
{
    public function whenCartCheckedOut(CartCheckedOut $event)
    {
        // Call method on projecion of servie, whatever you need to do
    }    
}

The above is handled by this strategy.

<?php namespace Projectionist\Strategy\EventHandler;

use Projectionist\Strategy\EventHandler;

class ClassName implements EventHandler
{
    public function handle($event, $projector)
    {
        $method = $this->handlerFunctionName($this->className($event));

        if (method_exists($projector, $method)) {
            $projector->$method($event);
        }
    }

    private function className($event)
    {
        $namespaces = explode('\\', get_class($event));
        return last($namespaces);
    }

    private function handlerFunctionName(string $type): string
    {
        return "when".$type;
    }
}

However, you can write your own version however you want. This means you're not stuck with the handler system we've implemented.

Using a different storage engine for projector positions

By default this system uses redis to keep track of projector positions. If you'd like to use another implementation, you'll need to write your own. This isn't too hard though, just create an adapter that implements the interface and passes the integration tests. You probably needs more detail, but the easiest way to figure out how to do this is look at the Redis implementation and the integration test for it.

Modes

Projectors tend to have three distinct modes, which control how each behaves when booted, or played.

1. run_from_start: Start at the first event and play as normal
2. run_from_launch: Start at most recent event, only play forward from there
3. run_once: Start at the first event, only run once

These modes allow for various usecases. Most projectors will be run_from_start, which is the default (ie. you don't need to define a mode), while run_from_launch is useful when you only want the projector triggered by new events, not existing ones. run_once is useful for the opposite reason, only run through existing events, ignore new ones.

These can be configured by add a MODE const to your projector, and setting the appropriate mode.

<?php

class Projector
{
    const MODE = ProjectorMode::RUN_FROM_LAUNCH;
}

Versioning and the power of seamless deploys

Projectors can be versioned. This means that while it is the same projector, it has changed in some way that requires all the events to be played though again.

Versioning is an important part of any projector system, and so we've made it as easy as possible to handle.

When booting, if the projector version has not been booted or played before, it will be considered new. This is important, as during a boot process, you'll want the new version to be booted up, while leaving the old version running on the live codebase. This allows you to boot a new version of a projector, without causing issue with the existing live version. If there's an issue during the boot, the process will fail, and the existing projectors will keep playing. If everything goes well, the existing codebase is told to stop playing projectors, and the new codebase takes over, allowing a seamless transition.

Here show you set the version of a projector.

<?php

class Projector
{
    const VERSION = 2;
}

Be default each projector is assumed to be version 1. When you need to bump the version, simple define the const and bump the number, the projectionist will take care of the rest.

Broken projectors

Sometimes projectors break, say you have a bug in your code, or an API call, or an SQL query fails, etc... The projectionist catches any uncaught throwables, and marks the projector as broken, before wrapping the exception and throwing it again. This way your error handler can still process the error.

Broken projectors are ignored by the play method, if they're broken they should be ignored, while working projectors should carry on as normal.

Fixing broken projectors is easy. The boot method will attempt to play broken or stalled projectors from where they left off, so if your you release a fix for a projector, boot will play it to the latest event from it's last valid position.

If a projector fails during boot, all other projectors are marked as "stalled", allowing the next call to boot to attempt to play them forward from their last valid position.

TODOs

My list of todos for this project

• Need to a way to handle broken releases, so you can switch back to the previous codebase and ensure the projectors are all up to speed
  • Do the same thing as standard migration systems, replay last batch
• Simplify 'fetchCollection' logic embedded in tests, its's getting hard to decipher and may not be required anymore
• Restructure test folders to make more sense
• Write a better tutorial for the adapters
• Use Ports and Adapters in tests