Anti-Patterns
← Part 9
Observability for Streams
Every architecture pattern comes with a set of failure modes that are specific to that pattern. Monoliths fail with tight coupling and deployment bottlenecks. Microservices fail with network complexity and service proliferation. Event-driven architecture fails with a distinctive set of problems: the system looks decoupled but is deeply coupled in non-obvious ways, debugging requires correlating events across ten services, and no single engineer can explain the full flow of a business process.
Most of these problems are self-inflicted through anti-patterns that are easy to fall into and hard to escape. This post names them clearly so you can avoid them.
Distributed Monolith via Events
The distributed monolith is the most common EDA anti-pattern. It happens when teams distribute their monolith but preserve all of its coupling—just mediated through events instead of function calls.
The signature: every service must process events from every other service in a specific order, and the system breaks when any service is slow or unavailable. The broker is a courier for a tightly coupled workflow, not a decoupling mechanism.
Signs you have a distributed monolith:
- A consumer always processes event X, then waits for event Y before doing anything useful
- Deploying service A requires coordinating deployment of services B, C, and D
- You have a private internal topic that only two specific services use
- The "consumer" is really an RPC proxy that calls the producer back synchronously
The fix: design for partial failure. Each service should be able to react to events independently without requiring all other services to be current or available. If your inventory service cannot do anything useful with an OrderCreated event until pricing has run, that's a workflow that should be orchestrated—not choreographed via events.
Event-as-RPC
Event-as-RPC disguises a synchronous request-response interaction as event exchange. A producer publishes a GetUserRequest event and then blocks waiting for a GetUserResponse event on a reply topic.
// Anti-pattern: treating events as RPC
{
"type": "GetInventoryRequest",
"requestId": "req-123",
"replyTo": "order-service.replies",
"payload": { "sku": "WIDGET-42" }
}
// The "response event"
{
"type": "GetInventoryResponse",
"requestId": "req-123",
"payload": { "quantity": 150 }
}This is HTTP over a message broker. You have added:
- A broker hop on each direction (2x latency)
- A timeout management problem (the broker never times out for you)
- A debugging problem (correlate request and response across topics)
- A reliability problem (if the responder is down, your queue fills with unprocessed requests)
And you've gained nothing that HTTP wouldn't give you more simply.
The fix: use HTTP or gRPC for synchronous data retrieval. Events are for broadcasting facts, not fetching data.
Schema Sprawl
Schema sprawl happens when event schemas grow organically without governance. Six months into a project, you have:
OrderCreated,OrderPlaced, andNewOrderEventall doing the same thing from different teams- The same concept represented differently in different events:
customer_idvscustomerIdvsuserId - Events carrying 80% of their payload as optional fields that only some consumers use
- No schema registry, so consumers have bespoke parsing logic for each event type
# The sprawl manifests as defensive consumer code
def handle_order_event(event: dict):
# Which field name did this producer use?
order_id = (
event.get("orderId")
or event.get("order_id")
or event.get("id")
or event.get("OrderId") # Someone used PascalCase
)
# Which customer field?
customer_id = (
event.get("customerId")
or event.get("customer_id")
or event.get("userId") # Someone conflated user and customer
)
if not order_id or not customer_id:
# Silently skip malformed events
returnThe fix: enforce a schema registry from day one (see Post 4), establish naming conventions before the second team publishes events, and review event schemas in architecture review the same way you review API contracts.
Temporal Coupling Through Sequence Dependencies
A subtler version of the distributed monolith: services that technically react to events independently, but only produce correct output when events arrive in a specific order.
# Anti-pattern: consumer breaks if events arrive out of order
class InventoryConsumer:
def handle_order_placed(self, event):
# Assumes the product exists in our database
# But what if ProductCreated hasn't arrived yet?
product = db.get_product(event["data"]["sku"])
if not product:
raise ProductNotFound(event["data"]["sku"]) # Will fail randomlyOut-of-order delivery is not an edge case—it is normal behavior in distributed systems, especially across partitions. Design consumers to handle events regardless of the order of arrival. Common approaches:
- Store and check: persist the event, process it when all dependencies are available
- Idempotent upsert: if the product doesn't exist, create a placeholder and process when it arrives
- Eventual consistency timeout: if dependencies are not available within a time window, route to DLQ for manual review
Chatty Events
Chatty events are events that are too fine-grained and emitted too frequently, causing the broker and all consumers to process noise.
Chatty events saturate consumers with low-value messages and obscure the meaningful business events. The fix: emit events at business-meaningful boundaries, not at every state change. Aggregate micro-events on the producer side and emit coarser domain events.
Event Sourcing Without Read Model Separation
Teams that adopt event sourcing sometimes query the event store directly for reads, treating it like a database.
# Anti-pattern: reading from event store for API responses
def get_order(order_id: str) -> dict:
events = event_store.get_events(aggregate_id=order_id)
order = Order()
for event in events:
order.apply(event)
return order.to_dict()This works for a handful of orders. At scale, replaying 200 events to answer a single HTTP request is unsustainable. Read models (projections) exist precisely to avoid this: events write to the event store, projections derive queryable state.
Summary: The Anti-Pattern Map
Key Takeaways
- The distributed monolith via events preserves all coupling of a monolith while adding all the operational complexity of distributed systems—the worst of both worlds.
- Event-as-RPC is HTTP disguised as event exchange; use HTTP or gRPC directly for synchronous data retrieval.
- Schema sprawl is a governance failure that shows up as defensive consumer code and silent data loss—enforce a registry early.
- Design consumers to handle out-of-order events; assuming temporal ordering is assuming a guarantee the broker does not provide.
- Chatty events (UI-level or field-level changes) overwhelm consumers with noise; emit events at business action boundaries.
- Event sourcing requires read model separation—never query the event store directly to serve API responses at scale.
← Part 9
Observability for Streams