Distilling the CQRS/ES capability

CQRS/event sourcing are patterns, and to be more specific they are Application Design patterns, and their implementation spans across 3 main Architectural areas (*).

• Structural i.e. Reliability/Availability/Scalability/Robustness and Continuity
• Operational i.e. Maintainability/Configurability/Portability/Upgradeability and Extensibility
• Cross-Cutting i.e. Security/Privacy/Supportability and Usability

To help pinpoint the exact implementation areas, let us look at two terms commonly used in Enterprise Architecture - Capabilities and Capability Maps.

Capabilities define what an organization needs to do to achieve the outcomes defined as part of a strategy successfully. A Capability Map is a map that visualizes those required capabilities in a future state to help achieve that outcome.

Essentially, the CQRS/ES Capability Map lays down all the required capabilities that need to be in place to help achieve the outcome of the rollout of an Enterprise-Grade CQRS/ES Platform.

These are classified into two main areas:

• The first area covers those capabilities required to help implement a logical infrastructure, i.e., an API to model and support CQRS/ES operations, preferably using DDD concepts.
• The second area covers those required to help implement a physical infrastructure to support CQRS/ES operations - Event Persistence/Retrieval and Message Routing (for Commands/Queries and Events).

A visual representation of the Capability Map is depicted below, followed by a high-level overview of the capabilities.

Logical Infrastructure

The Logical Infrastructure requires the availability of an API that provides the implementation of the various concerns necessary for client applications to perform CQRS/ES operations.

Modeling

The API needs to provide Modeling capabilities for the main CQRS/ES State/Structural and Operational Stereotypes.

• Aggregates - Structural/State Stereotype
• Message Handlers - Operational Stereotype
• Event Sourcing Handlers - Operational Stereotype

While not necessary, it is preferable to use DDD concepts to model the Structural/State stereotypes.

Event Generation

The API needs to provide capabilities to generate Events whenever a particular Command Message has been processed.

Event Publishing

The API needs to provide capabilities to publish generated Events.

Event Versioning

The API needs to provide capabilities to process various versions of Events as they evolve during the lifecycle of an application.

Event Handling

The API needs to provide capabilities to receive and process Event Messages.

Command Handling

The API needs to provide capabilities to receive and process Command Messages.

Query Handling

The API needs to provide capabilities to receive and process Query Messages.

Event Replay

The API needs to provide capabilities to replay events to allow a projection to be (re)built from the beginning of the stream or any arbitrary point in the stream. In addition, the order in which events are streamed should be consistent across replays.

Exception Handling

The API needs to provide adequate exception handling capabilities for any Messaging operation.

Physical Infrastructure

The Physical Infrastructure requires the capabilities to store Events (i.e., an Event Store) as well as the ability to route Messages (i.e., a Message Bus)

Append Events

The Event Store needs to store the events in an append-only fashion as they occur within the Application. In terms of append capabilities, the Event Store needs to provide for the following.

• Consistency - As we append events into the Event Store, it needs to validate and ensure that the sequence number for the events are stored incrementally with no duplication. The sequencing is critical since the correct construction of an Aggregate state requires an ordered read of events.
• Atomicity - An operation on an Aggregate could result in multiple events. The Event Store has to ensure that all these events are written together or none at all.
• Durability - Committed events in the event store need to be protected against data loss.
• Snapshots - Capability to create and store snapshots for a set of Events of an Aggregate to optimize the state construction phase.

Read Events

The Event Store needs to provide the capability to read the stored events. In terms of reading capabilities, the Event Store needs to provide the following.

• All for an Aggregate - Ability to read all events that have been stored for a specific Aggregate, including snapshots, if any.
• Since a point in time - Ability to read all events stored since a point in time to help build our query/read models.
• Ad-hoc Queries - Capability to do ad-hoc queries against the Event Store.
• Isolation - Ensuring stored events are only available to read once the transaction
  is committed to the store.
• Optimization - Optimized to read more recent Events.

Route Messages

The Event Store is responsible for routing all Message Types (Commands/Queries and Events) to their respective handlers. Essentially this capability is similar to that of a Message Bus with support for the required broker capabilities - sync/async modes and guaranteed deliveries.

Scalability

The Event Store needs to provide constant and predictable performance for Appending, Reading, and Routing events as the Event Store grows to potentially billions of events.

Availability & Reliability

The Event Store needs to be highly available and operate in a clustered mode with capabilities for load balancing, automatic/fast failover, and recovery.

To conclude, rolling out CQRS/ES requires implementing a set of concerns - logical, i.e., an API, and physical, i.e., an Event Store and a Message Bus.