Elevate Your Messages
Event Driven Architectures (EDA) are a powerful solution pattern. They enable a high level of decoupling between systems that might otherwise be tied together at the hip by types, schemas, and API surface descriptions. EDA makes it simpler to implement application logic and easier to implement error handling by promoting idempotency and providing automated retry dynamics alongside “dead letter” queues. Event driven solutions are easier to evolve topologically, especially when it comes to moving from a one to one to a one to many communication style. EDAs are easier to scale and in many ways make it generally simpler and easier to build for high throughput with every major public cloud supporting auto-scaling to maintain a predictable average or maximum event processing time no matter what the volume.
Despite the name, most EDAs are dealing in messages, not events. The biggest difference between messages and events is persistence. A message is ephemeral. It exists only for the purpose of notification or moving a small packet of data. An event is something of more lasting significance - an occurrence with meaning and implication of its own.
The Problems with Treating Events Like Messages
Using messages to carry events leads to a number of problems. Data loss is first among them. By treating events like messages, your system is by design throwing away data that has importance and value. Loss of events makes systems harder to debug as a critical record of “what happened” is explicitly lost. Numerous event driven applications would benefit from an ability to replay events to create or update current state. This is not possible with temporary messages. Your messages have a schema whether they know it or not. Handling schema changes (versions) is more difficult for ephemeral messages than it is for persistent events where version is explicitly stored and does not have to be implied from the time of creation or handling.
Searching messages is hard. They are not designed to be queried. Unless you construct a custom store with indexes to cover all common query patterns, effective search will be impossible or very slow.
Message schemas have a way of spinning out of control. Outside of the normal attention and governance given to persisted data formats, it can be quick and convenient to add “just one more little field” to a message structure to achieve some short term “temporary” goal. That “temporary” fix has a way of lasting for years and those fixes compound to result in over-complicated and hard-to-understand message formats full of poorly-named and badly-documented historical baggage. It is very unusual to treat messages and their implicit schemas as something that needs to be reasoned about and communicated outside of technical circles and an opportunity to establish and leverage ubiquitous language is lost, causing collaboration and communication to fall short of where it could be.
With messages, it is common that little thought is given to maintaining ordering. In fact, many common optimization paths make absolute ordering very hard or impossible. A lack of ordering guarantees makes it much more difficult to reason about and maintain complex systems. If ordered and persisted, many message streams could be used to feed audit processes. Lack of order also means that, even if they were persisted, message streams could not be used to power replay processes.
I am not trying to say that ephemeral, un-ordered messages are always the wrong choice. There are plenty of applications where this style of communication works well. That said, experience has shown that there are many more applications that would benefit from elevating messages to events to capture more data and take fuller advantage of what is possible with EDA.
How Kurrent Elevates Your Messages
Kurrent stores your events forever. OK. That was maybe a bit strong. Kurrent wants to store your events forever, but it will delete them if you ask nicely. That said, why would you ever want to delete your events? More so than ever, data is gold. Keep as much gold as you can.
The simple act of storing something elevates it from intermediary to first class citizen within a technical ecosystem. And for good reasons. Storage creates opportunities for value and that value is amplified by attention, design, structure, and careful evolution. By persisting your events in Kurrent, you set the stage of using events as a contract within your system and an opportunity for efficient discussion about system function, design, implementation, and evolution.
Kurrent brings order to your messaging chaos. And I mean that literally. Kurrent guarantees a strict order across all events. Knowing with certainty what happened, when it happened, and what happened before and after it is a massive accelerant on debugging and a foundational enabler of audit. Kurrent enables organizing events into fine-grained streams, usually to track the evolution of an entity or workflow. A comprehensive, ordered view of the entire lifetime of an object or process is a rich basis for effective application development.
Having a persistent ordered store is also an enabler of projection, which may be Kurrent’s most impressive super power. Projection is the act of taking a collection of events and processing them into another format. For example, one might take the contents of Kurrent and project them into an elastic search store optimized for free text query. Kurrent provides the infrastructure for creating projections and keeping them up to date on an eventually consistent basis. Because Kurrent doesn’t throw anything away, you can create projections starting from the beginning of time at any time meaning you don’t have to anticipate early on every possible read model you might want to provide for your event data.
Unlock the Next Tier of Value
By choosing to use EDA, you’ve potentially unlocked one tier of value to build highly scalable and decoupled applications. By choosing to use Kurrent, you will be able to double down on your EDA investment and unblock an entirely new tier of value.
Xolvio is a Kurrent professional services partner. We work with Kurrent customers every day to elevate their messages into events and get their development program to the next level.
