Apply event state before persistence

[aidan-casey]

Description
Automatically apply event to aggregate upon recording to ensure state is accurately represented.

Example
In Greg Young's example of SimpleCQRS, you will note that the state is applied instantly to the aggregate the event is recorded on and the change tracked for persistence. It seems like this would make sense to replicate to simplify instances like the following:

final class Organization
{
    use WithEvents;
    use WithAggregateVersioning;

    #[Identifier]
    private OrganizationId $id;

    private string $name;

    public static function create(OrganizationId $organizationId, string $name): self
    {
        //
    }

    // This will fail because it is not defined upon persistence!
    public function getId(): OrganizationId
    {
        return $this->id;
    }

    public function getVersion(): int
    {
        return $this->version;
    }

    #[EventSourcingHandler]
    private function applyOrganizationWasCreated(OrganizationWasCreated $event): void
    {
        $this->id = new OrganizationId($event->organizationId);
        $this->name = $event->name;
    }
}

The only solution to this specific instance would be requiring the ID in the construct of the aggregate so it can be utilized during persistence.

[unixslayer]

@aidan-casey can you open an PR with a test case?

[dgafka]

@aidan-casey if I understand correctly we trigger EventSourcingHandler method right away, when recordThat is used.
That would be possible with a bit more modified internal recorder.

However this approach is only in there because, for long time that was primary way of writing event sourcing aggregates.
It's not advised way of writing ES Aggregates as it makes Aggregates being state holders for list of events.
The advised way if to return set of events, called pure aggregate approach.

From my perspective it does not fit into pure aggregate approach. Can you explain, how do you see it?

[jlabedo]

@dgafka , why the pure aggregate approach should be advised instead of the internal recorder approach ?
The pure approach make the domain object unusable without the framework. We can see the problem with @aidan-casey use case and also with issue #204 where it is not possible to return an event sourced aggregate from another one.

Having internal recorder state in event sourced aggregate is not different from having it in state aggregate and in case of event sourcing, events ARE the state so it makes sense to have them stored in the aggregate memory. Wdyt ?

[unixslayer]

@jlabedo Event Sourcing Aggregate should not create another Event Sourcing Aggregate. If ES Aggregates shares the same stream, there is no need for that but when they have their own streams you have two options really

1. define factory method as Event Handler in aggregate being created - convenient as long those aggregates won't "talk" to each other after that
2. reuse events between streams It is not possible to use the same events between two aggregates. #149

The pure approach make the domain object unusable without the framework

I don't understand this one. Can you elaborate?

[unixslayer]

@aidan-casey

Automatically apply event to aggregate upon recording to ensure state is accurately represented

This would require Ecotone to provide an abstraction of AggregateRoot or, like @dgafka mentioned, modification to the internal recorder. However, that may raise some performance issues if, e.g. Reflection would be used. Check for example annotation finder.

However, your example will fail only in the following situations:

1. Organization::getId() gets called in the factory method. But why would you do that?
2. you are not using Messaging but work with Aggregates in the vanilla approach, e.g. in tests which I assume due there is no #[CommandHandler] defined

public function test_something(): void
{
    $organization = Organization::create(new OrganizationId(1), 'name');
    
    assertEquals(new OrganizationId(1), $organization->getId());
}

When using Messaging, Aggregate will be recreated from its events before being called.

[jlabedo]

@jlabedo Event Sourcing Aggregate should not create another Event Sourcing Aggregate. If ES Aggregates shares the same stream, there is no need for that but when they have their own streams you have two options really

1. define factory method as Event Handler in aggregate being created - convenient as long those aggregates won't "talk" to each other after that
2. reuse events between streams It is not possible to use the same events between two aggregates. #149

I think it is not a good idea to make aggregate share the same stream as it defeats the very first goal of aggregates to protect their internal state with their internal invariants. I would be glad to discuss that matter with your precise use case !
I don't understand why an "Event Sourcing Aggregate should not create another Event Sourcing Aggregate." ? What is different from State based aggregate ?

The pure approach make the domain object unusable without the framework

I don't understand this one. Can you elaborate?

I mean that in case of pure event sourced aggregate, you cannot do :

$organization = Organization::create(OrganizationId::from(1), "Acme");
// $organization is an array of events and not an Organization as someone would expect

In the case of internal recorder es aggregate :

$organization = Organization::create(OrganizationId::from(1), "Acme");
// $organization is an Organization
// but the next line will throw an error as EventSourcing handlers are not executed without framework support
return $organization->getId();

I think that the choice of event sourced vs state aggregate should remain a technical detail internal to the aggregate. Wdyt @unixslayer ?

This would require Ecotone to provide an abstraction of AggregateRoot or, like @dgafka mentioned, modification to the internal recorder. However, that may raise some performance issues if, e.g. Reflection would be used. Check for example annotation finder.

Totally agree with that. So the execution of event sourced handlers should remain a domain matter as Ecotone will never require the user to extend an abstract class :) but Ecotone could provide support for helping executing the handlers (like internal recorder state and traits).

[unixslayer]

The difference between State-based and Event Sourcing Aggregates is the way they are persisted and how they apply state changes.

State-based apply state change directly on itself and the result of operation made on that aggregate is an aggregate with the already applied state change.

For Event sourcing aggregate things are playing a bit differently. The current state is always rebuilt from a stream of events and the result of any operation being called is the set of events that are changed after its execution.

Whether to use Event Sourcing or State-based aggregate is an implementation detail that should be determined by the knowledge the team has about the business.

I think that the internal recorder as it should be considered a legacy is the source of confusion here. After all both aggregates, whether they use an internal recorder or not are Event Sourcing aggregates and they are expected to return a set of events. The only difference is how to retrieve those events. However, I think that after #204 is resolved, that will change in some way ;) Also when an internal recorder is used in State-based aggregate it behaves differently on persistence.

[jlabedo]

Whether to use Event Sourcing or State-based aggregate is an implementation detail that should be determined by the knowledge the team has about the business.

Honestly, my team is heavily using event sourced aggregate in ecotone because :

• it is so easy thanks to ecotone to create (design) new aggregates (thanks @unixslayer 🥇 ).
• it really forces splitting the view side from the mutating side, making our brains really work on business rules instead of query considerations so our aggregates are much simpler, even with low domain knowledge.

Maybe its wrong but it is just amazing how devs with no particular event sourcing background want to use event sourced aggregates also because of DX matters, not business matters.

Having said that, we are not unit testing aggregates right now on the project using ecotone and ES. But it seems legitimate to allow unit testing without framework support. This makes me think ecotone should handle loading and persisting event sourced aggregates with internal event handlers, and should permit writing this test case:

public function test_something(): void
{
    $organization = Organization::create(new OrganizationId(1), 'name');
    
    assertEquals(new OrganizationId(1), $organization->getId());
}

[dgafka]

it is so easy thanks to ecotone to create (design) new aggregates (thanks @unixslayer 🥇 ).

That's the point. Development should be fun, not exhausting ;)
This comes from pushing abstraction levels higher and higher, so what is left is business code.

Having said that, we are not unit testing aggregates right now on the project using ecotone and ES. But it seems legitimate to allow unit testing without framework support.

Well for Ecotone it does not really matter how you store the Aggregate, e.g. you could provide your own Event Sourcing Repository, at the end it's just an object that provides API in form of Commands and Events Handlers.
And for sending Command and Events, Ecotone provides testing support.

why the pure aggregate approach should be advised instead of the internal recorder approach ?

Internal Recorder is not going anywhere, even so I call it legacy, it's fine to use it :)
However I call it legacy for a reason, as Aggregate itself can only hold enough state to protect invariant and with Internal Recorder we are keeping state which is only needed for rebuilding the aggregate.

If you take a look on functional ES, they have functions that provide new VO every time we apply event, nothing is hold in between. The so called pure aggregate is aiming into more functional direction, which is more "pure" due to lack of the need to hold intermediate state.

This makes me think ecotone should handle loading and persisting event sourced aggregates with internal event handlers, and should permit writing this test case:

It's up to you and your team to decide on which abstraction level you work at. Ecotone provides abstraction for different levels, so you can pick the level you feel okey with.
For example, I met people that don't want to put attributes in their Aggregates and that's fine, they use External Command Handlers. They get more autonomy, however as a cost they need to write and maintain more non-business related code.

The same is with pure ES Aggregates, if you decide to go this path, you lose possibility to do what you've suggested

public function test_something(): void
{
    $organization = Organization::create(new OrganizationId(1), 'name');
    
    assertEquals(new OrganizationId(1), $organization->getId());
}

You've chosen higher level abstraction of doing things, so that's the cost. However you gain more business focused aggregates, as they do not need hold intermediate state. If you however need this kind of implementation, you may use ES with Internal Recorder and above will work just fine.
So abstraction level does matter :)

[unixslayer]

So abstraction level matter :)

But it shouldn't be your concern when you implement business logic so you won't be limited to abstraction.

This discussion starts to touch on a lot of topics :D

In the case of unit testing ES aggregate, you have to ask one really simple question: what do you want to test exactly? And the answer should be to test aggregate behavior under business requirements. And to do that you may want to assert what was the result returned from your ES aggregate.

The example above is not testing anything besides implementation and in my opinion, this isn't any reason at all to change the behavior of the internal recorder. To unit test ES aggregate you should put it in a designated state and check what gets returned. For convenience, you can use Object Mother for that

class OrganizationMother
{
    public static function aggregateInDesiredState(): Organization
    {
        $organization = new Organization();
        // apply whatever is valid for business requirement
        $organization->apply...

        return $organization;
    }
}

public function test_something(): void
{
    $organization = OrganizationMother::aggregateInDesiredState();

    assertEquals(
        [], // expected events
        $organization->doSomething(new DoSomethingCommand())
    );
}

However, it will be way more convenient to use Ecotone Test Support

public function test_something(): void
{
    $ecotone = EcotoneLite::bootstrapFlowTesting(
        classesToResolve: [Organization::class]
    );

    assertEquals(
        [], // expected result events
        $ecotone
            ->withEventsFor(
                new OrganizationId(1),
                Organization::class,
                [], // events already persisted on stream
            )
            ->sendCommand(new DoSomethingCommand())
            ->getRecorderEvents()
    );
}

[jlabedo]

class OrganizationMother
{
    public static function aggregateInDesiredState(): Organization
    {
        $organization = new Organization();
        // apply whatever is valid for business requirement
        $organization->apply...

        return $organization;
    }
}

By letting the apply... methods public in your aggregate, you can let your aggregate become in an inconsistent state, as your business rules are not checked.
With events applied internally, these methods can be private, and you have to use your commands (public interface) to go to your desired state. It seams cleaner to me. Wdyt ?

[unixslayer]

@jlabedo I was reffering to #[EventSourcingHandler] methods which are public.

[jlabedo]

I was referring to those too :) I think they should be internal to the aggregate (private), which is possible with events applied internally

[aidan-casey]

Hey all! Sorry I have been traveling and this blew up so hopefully I can touch on the important points here. 🙂

Let me set the scene for you before I dive into some examples. We are indeed using messaging, but are being careful to take a hexagonal / DDD approach that separates our core domain (aggregates, events, etc.), our application (services, commands, etc.), and our infrastructure.

This is totally possible, it seems, from the Ecotone side of things which awesome! I love the flexibility, but it is a tad clunky.

So for example, this is how I currently have it set up:

OrganizationRepository
Note that this is an interface, so we are okay with it being in the domain and mildly referencing Ecotone since it would still be relatively easy to replace in infrastructure by creating an implementation.

namespace Foo\OrganizationManagement\Core;

use Ecotone\Modelling\Attribute\RelatedAggregate;
use Ecotone\Modelling\Attribute\Repository;

interface OrganizationRepository
{
    #[Repository]
    public function findById(OrganizationId $organizationId): Organization;

    #[Repository]
    #[RelatedAggregate(Organization::class)]
    public function save(OrganizationId $organizationId, int $currentVersion, array $events): void;
}

OrganizationService

<?php

namespace Rego\Hub\Core\OrganizationManagement\Application;

use Ecotone\Modelling\Attribute\CommandHandler;
use Foo\OrganizationManagement\Application\Commands\ChangeNameCommand;
use Foo\OrganizationManagement\Application\Commands\CreateOrganizationCommand;
use Foo\OrganizationManagement\Core\Organization;
use Foo\OrganizationManagement\Core\OrganizationId;
use Foo\OrganizationManagement\Core\OrganizationRepository;

final class OrganizationService
{
    public function __construct(private readonly OrganizationRepository $repository)
    {}

    #[CommandHandler]
    public function create(CreateOrganizationCommand $command): void
    {
        $organization = Organization::create(
            organizationId: new OrganizationId($command->organizationId),
            name: $command->name
        );

        $this->repository->save(
            $organization->getId(),
            $organization->getVersion(),
            $organization->getRecordedEvents()
        );
    }
}

I think that it is important to note that Greg Young, the creator of ES, applies his events in memory first and then persists them. This ensures consistent state before it is persisted and avoids these types of problems, because the state of an ES aggregate should indeed be built by the events.

Okay, so with that context, I'll address some of the conversation. 🙂

From my perspective it does not fit into pure aggregate approach. Can you explain, how do you see it?

I'll be honest, I never liked the pure aggregate approach and it doesn't seem to work well with the level of abstraction we are working with. Open to having my mind changed there, but it feels like a solution that was created primarily for a level of automation within Ecotone.

However, that may raise some performance issues if, e.g. Reflection would be used.

Totally understand the concern here. It's a tough solution to figure out, especially where our goal is less reliance on outside dependencies within our domain! I think reflection performance could be overcome, particularly where you only need to reference $this class, reflection is pretty fast. I've used it with other event sourcing libraries I've worked with and contributed to.

---

I'll end by saying I think some of this comes down to design philosophy. Ecotone is so close to being the only event sourcing package out there that actually allows the level of hexagonal/DDD abstraction we are looking for here with minimal effort. I'd love to see it keep moving in that direction instead of head away from that.

Some things that I've never really understood are elements such as:

• Handling commands from the aggregate itself.
• Returning an array of events from the aggregate method.
• etc.

And I think all of these come down to more automation in the handling of the message, that is nice, but removes more ability to abstract.

[jlabedo]

I love this discussion :) And I think it is very important to give an answer to this one before #204 and also #149

To be clear, I am exclusively using the pure aggregate approach in my projects, but I have to agree with @aidan-casey : I don't think it is that good. I also think it is why we are having trouble in #204 ES case.

I would really love to see the exact use case with CertificateCycle of @unixslayer . I bet a coin that it could be solved with a pattern like the moderate post of Vernon sample, where the Forum aggregate (CertificateCycle in your case I guess) is responsible to moderate the Post aggregate (Certificate). To do that we need to apply event state immediately.

What do you think of this design (working example here) :

// userland
#[EventSourcingAggregate]
class InternalHandlerAggregate
{
    use WithInternalExecutor;

    #[AggregateIdentifier]
    private int $id;

    public static function create(int $id): self
    {
        $self = new self();
        $self->recordThat(new InternalHandlerAggregateWasCreated($id));
        return $self;
    }

    #[EventSourcingHandler]
    private function whenInternalHandlerAggregateWasCreated(InternalHandlerAggregateWasCreated $event): void
    {
        $this->id = $event->id;
    }

    public function getId(): int
    {
        return $this->id;
    }
}

trait WithInternalExecutor {

    #[EventSourcingHandlerMap]
    private static array $eventToHandlerMap = [];
    private array $recordedEvents = [];

    public function recordThat(object $event): void
    {
        if($methodName = static::$eventToHandlerMap[get_class($event)]) {
            $this->{$methodName}($event);
        }

        $this->recordedEvents[] = $event;
    }

    #[EventSourcingExecutor]
    public static function fromEventStream(array $events): static
    {
        $self = new self();
        foreach ($events as $event) {
            if($methodName = static::$eventToHandlerMap[get_class($event)]) {
                $self->{$methodName}($event);
            }
        }

        return $self;
    }

    #[AggregateEvents]
    public function getRecordedEvents(): array
    {
        $recordedEvents = $this->recordedEvents;
        $this->recordedEvents = [];

        return $recordedEvents;
    }
}

#[\Attribute(\Attribute::TARGET_METHOD)]
class EventSourcingExecutor {
}

#[\Attribute(\Attribute::TARGET_ALL)]
class EventSourcingHandlerMap {
}

class InternalHandlerAggregateWasCreated
{
    public function __construct(public int $id)
    {
    }
}

So the responsiblity of applying events is left to userland but Ecotone could give an help with EventSourcingHandler, EventSourcingHandlerMap attributes.

[dgafka]

I think we could improve Internal Recorder in a way you're proposing @jlabedo. I wonder, if we could somehow pass the map to avoid putting effort on the end user, especially that this is something what Ecotone already knows (considering we leave reflection out of the scope).

So ES Aggregate with Recorder would apply events immediately and we could dynamically create them from other aggregates #204 (as we would return an object instead of events).
On other hand Pure ES Aggregate would not have this ability to apply immediately and to be created from other aggregates, yet would keep it internal state clean of all the mentioned methods and properties.

My concern is, if this different behaviour will not be confusing for the end users of Ecotone tho.

[dgafka]

Some things that I've never really understood are elements such as:
Handling commands from the aggregate itself.
Returning an array of events from the aggregate method.
etc.

Event Sourcing deserves new documentation, as it's more like an API doc, than guide atm.

But for State-Stored Aggregates and Command Handlers you may follow CQRS section. It was recently refactored to provide more insights on why (being guide, not API doc).

[jlabedo]

I think we could improve Internal Recorder in a way you're proposing @jlabedo. I wonder, if we could somehow pass the map to avoid putting effort on the end user, especially that this is something what Ecotone already knows (considering we leave reflection out of the scope).

This is the point of EventSourcingHandlerMap attribute: injecting EventSourcingHandlers into the static property . Here is a quick implementation on Ecotone side.

So ES Aggregate with Recorder would apply events immediately and we could dynamically create them from other aggregates #204 (as we would return an object instead of events). On other hand Pure ES Aggregate would not have this ability to apply immediately and to be created from other aggregates, yet would keep it internal state clean of all the mentioned methods and properties.

My concern is, if this different behaviour will not be confusing for the end users of Ecotone tho.

I don't have this concern: It seems less confusing than returning a collection of events and other aggregates at the same time :)

My concern would be that we are still linked to the framework to populate the $eventToHandlerMap (during unit tests for instance).

[dgafka]

I don't have this concern: It seems less confusing than returning a collection of events and other aggregates at the same time :)

That would still need to be covered, as this is exactly the case @unixslayer was mentioning.
But even so, we can improve internal recorder aside from #204, as it serves different use case.

[jlabedo]

Ok then let's let this mature for a few days and if we agree on this I would be happy to work on this feature.

[unixslayer]

Frankly, I'm going to convert this issue into a discussion due it is not a problem of Ecotone but how it is used in that particular example.

The problem you are facing can be solved in your project directly just by implementing your own internal recorder, e.g. as @jlabedo suggested.

But let me elaborate more about a problem I want to address.

The internal recorder works for a very specific purpose and applying the changes you are looking for may not only cause performance issues but also introduce huge BC Break by forcing a specific abstraction of aggregate design wherever it will be used and bring unexpected behavior. Remember that Ecotone's biggest advantage is to separate abstraction and technical stuff from your actual domain modeling.

And that's the thing right here... Don't get me wrong but I notice that with every developer who tries to implement DDD. You are focusing too much on technical stuff rather than modeling the actual domain.

What was done implementing OrganizationService was taking the internals of Ecotone and redoing them again. With some drawbacks also

• you are exposing aggregate internals so OrganizationService can work as expected;
• you've removed events publication because the repository won't do it;
• you've implemented an anemic domain model. Implemented everything in *Service makes aggregates only as data storages;

IMO there is no problem, at least not any that should be solved by adding more abstraction.

Besides implementing your own internal recorder (what Ecotone really cares about is only #[AggregateEvents] attribute however, in your setup, it is not used at all) you may use values from the command. I assume the problem is only after a new instance gets created because loading already persisted aggregate will trigger apply methods.

final class OrganizationService
{
    public function __construct(private readonly OrganizationRepository $repository)
    {}

    #[CommandHandler]
    public function create(CreateOrganizationCommand $command): void
    {
    	$organizationId = new OrganizationId($command->organizationId); // command can already have instance of OrganizationId
        $organization = Organization::create(
            organizationId: $organizationId,
            name: $command->name
        );

        $this->repository->save(
            $organizationId,
            0, // for newly created instances initial version is always 0
            $organization->getRecordedEvents()
        );
    }
}

However, I strongly recommend to rethink your design without taking any technical stuff into account.

Maybe this will help. Certificate examples that I mention in other issues are taken from my current project. For the first 9 months, we were implementing the domain without any kind of infrastructure. even without a database.

I've taken a part of that domain, simplified it, and made it as an example here. The description is actually a recruitment assignment we are sending to the candidates.

[dgafka]

I'm going to convert this issue into a discussion due it is not a problem of Ecotone but how it is used in that particular example.

Converting to Github Discussion is really cool feature. All comments were preserved :)

Remember that Ecotone's biggest advantage is to separate abstraction and technical stuff from your actual domain modeling.

Besides that, Ecotone provides different levels of abstraction and does not force to do things in particular way. Thefore people can easily switch from their existing tooling by matching specific level of abstraction they currently using. Of course they can jump to another level of abstraction later (e.g. external command handlers -> aggregate command handlers).

However, I strongly recommend to rethink your design without taking any technical stuff into account.

Yep, let's suggest alternatives, yet not force people to do things in particular way. Developers know the full context of the system they are working on, they know what is the skillset of developers, what level of abstraction are acceptable and what will be easiest solutions that will help with daily development. So the best decision they can make, is actually their own decision.
And to let them make the correct decision, to some extend we need to support different ways of working.

IMO there is no problem, at least not any that should be solved by adding more abstraction.

Considering how people are building ES Aggregates, this is actually the most common case. What @aidan-casey is suggesting is pretty standard and to help people switching to Ecotone, we should support that too.

introduce huge BC Break by forcing a specific abstraction of aggregate design

I agree that it should not create any B/C break, especially that Internal Recorder is also used for State-Stored Aggregates.
We can do that by introducing separate trait e.g. withAppliedEvents, which will be designed only for this purpose.
This way we keep withEvents untouched, yet we deliver extended version, which applies to ES Aggregates.

[jlabedo]

Regarding the BC break.
For internal recording case, the aggregate needs to provide a method for rehydrating an aggregate from a stream of events. This method would be marked with an attribute like EventSourcingExecutor.
If the framework detects this attribute, it will apply events internally. If not, it will apply events from the framework, as it is done today.
This way there is no BC break.

[unixslayer]

the aggregate needs to provide a method for rehydrating an aggregate from a stream of events

[#EventSourgingHandler] is for that.

What kind of problems are we trying to solve here? For me, this is clearly a problem of local aggregate design which has not that much to do with Ecotone internals.

Two things must be taken into account for the solution:

1. it cannot force directly how aggregates should be designed.
2. it cannot increase the complexity of Ecotone internals itself.

It is Ecotone that provides API to which local abstraction should adapt, not the other way around.

I can agree that a new, modified version of Internal Recorder may be implemented which could be used wherever necessary. It may bring more to local abstraction at the same time using the same API.

For reference, there was a similar situation with prooph components

[dgafka]

So my understanding of what is needed right now, is way to apply event immediately. I don't see option of modifying existing trait without affecting B/C or performance impact. That's why having a separate trait feels like right direction.

If we agree on having a separate trait, then it's a matter of deciding on how this new recordThat should know about methods where events are applied and should we still use EventSourgingHandler or let developer to do all the mapping?