338 named an honest gap: the pipeline's queue can't survive a crash and can't cross instances. This lesson closes it, with the exact tools Part IX already built for the job.
338 ended with a specific, named problem: OrderProcessingService's in-memory Channel<T> loses every queued work item on a crash or restart, and can't be seen by any OrderFlow instance other than the one that queued it. You already have every tool needed to fix that — 286 taught why services should communicate through events instead of direct calls, 288 taught Kafka's partitioned, replicated log as the durable transport for those events, 294 taught the Outbox pattern for guaranteeing an event actually gets published when the database write that triggered it commits, and 290 taught idempotency for when a message inevitably gets delivered more than once.
None of those four lessons get re-taught here. What follows is how they combine to give OrderFlow a durable, multi-instance-safe answer to the exact gap 338 named — and a fifth event, OrderPlaced, published through order-events, consumed independently by InventoryService, ShippingService, and NotificationService.
OrderFlow publishes one event, OrderPlaced, to a Kafka topic called order-events, partitioned by OrderId (288's partition-key discipline). Three independent consumer groups read the same topic, each at its own pace, each doing its own job:
| Consumer group | Reacts by |
|---|---|
| inventory | InventoryService reserves stock for the order's items |
| shipping | ShippingService schedules a shipment once stock is confirmed |
| notification | NotificationService emails the customer their confirmation |
288's fan-out model applies exactly as taught: these three consumer groups don't compete for the event — each one gets its own full, independent copy of every OrderPlaced event published, at whatever pace it can keep up.
Two separate problems, both already named, both needing an answer. First, 338's durability gap: a Kafka topic, unlike an in-memory channel, survives a process restart entirely — the event is sitting in the broker, not in any one instance's memory. Second, a subtler risk 338's single sequential pipeline didn't have to face: writing the Order row and publishing the event are two separate operations, and a crash between them would mean a real, saved order that no consumer ever hears about. 294's Outbox pattern is the exact, already-taught answer — write the event as a row in the same local transaction as the Order insert, and let a separate publisher pick it up and actually send it to Kafka afterward, guaranteeing eventual publication without needing a distributed transaction across the database and the broker.
order-events topic, partitioned by OrderId (288)inventory group shipping group notification group// ═══ In OrderService.PlaceOrderAsync (334/336) — one atomic local transaction ═══
public async Task<Guid> PlaceOrderAsync(Guid customerId, List<OrderItem> items, CancellationToken ct)
{
var order = new Order(customerId, items);
await orders.AddAsync(order, ct);
// The event, in the SAME transaction as the Order insert — 294's core guarantee
await outbox.EnqueueAsync(new OutboxMessage
{
Id = Guid.NewGuid(),
EventType = "OrderPlaced",
PayloadJson = JsonSerializer.Serialize(new OrderPlacedEvent(order.Id, order.CustomerId)),
OccurredAtUtc = DateTime.UtcNow
}, ct);
await unitOfWork.SaveChangesAsync(ct); // both rows commit together, or neither does
return order.Id;
}
// ═══ OrderFlow.Infrastructure — a BackgroundService, exactly 165/338's shape ═══
public class OutboxPublisherService(
IServiceScopeFactory scopeFactory, IKafkaProducer producer, ILogger<OutboxPublisherService> logger)
: BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
using var scope = scopeFactory.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<OrderFlowDbContext>();
var unpublished = await db.OutboxMessages
.Where(m => m.PublishedAtUtc == null)
.OrderBy(m => m.OccurredAtUtc)
.Take(50)
.ToListAsync(stoppingToken);
foreach (var message in unpublished)
{
await producer.ProduceAsync("order-events", key: message.Id.ToString(), message.PayloadJson, stoppingToken);
message.PublishedAtUtc = DateTime.UtcNow;
}
await db.SaveChangesAsync(stoppingToken);
await Task.Delay(TimeSpan.FromSeconds(2), stoppingToken);
}
}
}Meaning: If the process crashes between the Order insert and Kafka actually receiving the event, the OutboxMessage row is still sitting there, unpublished — the next poll picks it up and publishes it. No event is ever silently dropped because of a crash at exactly the wrong moment; that's 294's whole guarantee, now protecting OrderFlow's real checkout path.
public class InventoryEventConsumer(OrderFlowDbContext db, IProductRepository products)
{
public async Task HandleAsync(OrderPlacedEvent evt, string messageId, CancellationToken ct)
{
// 290's idempotency check — a UNIQUE constraint on ProcessedMessageId, not just an in-memory check
var alreadyHandled = await db.ProcessedMessages
.AnyAsync(p => p.MessageId == messageId && p.Consumer == "inventory", ct);
if (alreadyHandled) return; // Kafka redelivered this — skip, don't reserve stock twice
foreach (var item in evt.Items)
await products.ReserveStockAsync(item.ProductId, item.Quantity, ct);
db.ProcessedMessages.Add(new ProcessedMessage { MessageId = messageId, Consumer = "inventory" });
await db.SaveChangesAsync(ct); // the reservation AND the dedup record commit together
}
}This is exactly why 290 mattered here specifically: Kafka's at-least-once delivery means order-events can genuinely redeliver the same OrderPlaced event to the inventory consumer group after a rebalance or a retry. Without this check, a redelivered event would reserve the same stock twice for one order — a real, customer-visible bug, not a theoretical edge case.
338's in-memory queue was like calling three people, one after another, to tell them the news — if you got interrupted partway through your calls, whoever you hadn't reached yet simply never finds out. Publishing OrderPlaced to order-events is like posting a bulletin to a board that inventory, shipping, and notification each check independently, at their own pace — the bulletin stays on the board whether or not you're still there, and each reader keeps their own bookmark for what they've already read, so nobody misses it and nobody reads the same notice twice by mistake (that bookmark is exactly 290's idempotency check).
338's OrderProcessingService doesn't disappear — its shape survives entirely, exactly as 338's own "Under the Hood" section predicted. What changes is only what feeds it: instead of reading from an in-memory Channel<T> written directly by the controller, OrderFlow's consumers (InventoryEventConsumer, and equivalents for shipping and notification) now read from Kafka, fed by OutboxPublisherService instead of the controller directly. The controller's job shrinks to exactly one thing: write the Order and the OutboxMessage in one transaction, then return. Every consumer is its own independent BackgroundService, which is also what finally makes 333's earlier open question — could any of these five services become a genuinely separate deployment — a real, available option: InventoryService could move to its own process, or its own container, without ShippingService or NotificationService needing to change at all, because they were never calling each other directly in the first place.
OutboxPublisherService and every event consumer are still BackgroundService subclasses, built exactly per 165 — polling loops, ExecuteAsync, IServiceScopeFactory for scoped dependencies, all unchanged from 338. What's different is that they now read from a Kafka topic instead of an in-process channel — the hosting mechanism this course already taught didn't need to change at all to gain durability.
Kafka being durable means an event, once published, won't be lost. It says nothing about the moment before publication — a crash between the Order insert and the Kafka call would still lose the event entirely without the Outbox pattern. The Outbox table is what guarantees the event gets published at all, exactly once as a durable intent; Kafka is what guarantees it isn't lost after that. Both pieces are load-bearing, for different halves of the same crash window.
Calling producer.ProduceAsync right after SaveChangesAsync in the same method, reasoning "Kafka is durable, so this is fine." If the process crashes between those two calls — after the Order committed, before Kafka received anything — the event is gone forever, with no record it was ever supposed to be sent. Write the event as an OutboxMessage row in the same transaction as the Order, and let OutboxPublisherService handle the actual Kafka call afterward, exactly as 294 specifies.
Having InventoryEventConsumer reserve stock unconditionally on every message received, treating duplicate delivery as an edge case not worth the extra table and query. At-least-once delivery is Kafka's actual, documented guarantee, not a rare failure mode — 290's deduplication check with a genuine unique database constraint is required, not optional, for any consumer whose action isn't already naturally idempotent.
Producing to order-events with no key, or a random one, letting Kafka spread related messages across partitions arbitrarily. Key by OrderId (or the outbox message's own id, tied back to one order), exactly as 288's Mistake 1 warned — this is what keeps any set of events for the same order consistently ordered relative to each other, should OrderFlow ever publish more than one event per order.
You've seen how OrderFlow's async pipeline becomes durable and multi-instance-safe. Let's confirm the reasoning behind each piece.
1. Why does PlaceOrderAsync write an OutboxMessage row instead of calling producer.ProduceAsync directly, right after saving the Order?
Correct: B
Why B is correct: This is exactly Common Mistake 1 and the Why Does It Exist? reasoning — the Outbox pattern closes the crash window between "the order is saved" and "the event is actually sent," which a direct Kafka call right after SaveChangesAsync leaves wide open.
Why A is incorrect: The real issue isn't a technical restriction on calling Kafka from inside a transaction — it's that Kafka publication can't participate in the same local database transaction as the Order insert, so a separate atomic mechanism (the Outbox row) is needed instead.
Why C is incorrect: Nothing in ASP.NET Core itself requires an Outbox table — this is a deliberate architectural choice made for OrderFlow's specific durability needs, not a framework requirement.
Why D is incorrect: The lesson's own "Common Mistakes" section describes exactly the scenario of calling Kafka directly from the same method that handles the request — nothing technically prevents it; it's simply unsafe.
Reinforcement: The Outbox row is what guarantees the event's eventual publication survives a crash at the worst possible moment.
2. Why does InventoryEventConsumer check a ProcessedMessages table before reserving stock, instead of reserving stock unconditionally every time a message is received?
Correct: B
Why B is correct: This is exactly the idempotency reasoning from 290, applied to OrderFlow — Kafka's real guarantee is at-least-once, so redelivery is expected behavior, not a rare edge case, and the check is what prevents a redelivered event from reserving stock a second time.
Why A is incorrect: This inverts Kafka's actual guarantee — it is explicitly at-least-once, not exactly-once, which is precisely why the check is necessary rather than a defensive formality.
Why C is incorrect: Stock counts live on the Products table (336); ProcessedMessages exists purely to track which messages have already been handled.
Why D is incorrect: This is a deliberate design choice made for OrderFlow's specific consumers, not a blanket framework requirement.
Reinforcement: At-least-once delivery makes idempotent consumers a requirement, not an optional safeguard.
3. What happens to OrderProcessingService and the BackgroundService pattern from 338 once OrderFlow adopts Kafka and the Outbox pattern in this lesson?
Correct: B
Why B is correct: Under the Hood and Common Confusion #1 both state this directly — the hosting mechanism (BackgroundService) is unchanged from 165/338; this lesson only replaces the queue underneath it with a durable, shared Kafka topic fed by the Outbox pattern.
Why A is incorrect: Kafka doesn't require or provide its own .NET hosting mechanism — BackgroundService remains exactly how consumers and the publisher are hosted.
Why C is incorrect: Kafka is a transport for events — it has no awareness of OrderFlow's actual business logic (charging, reserving, shipping); real application code, running inside BackgroundService subclasses, still has to do that work.
Why D is incorrect: BackgroundService is built entirely around async/await, as both 165 and 338's examples demonstrate — this claim is simply false.
Reinforcement: This lesson changed OrderFlow's transport and durability guarantees, not the underlying .NET mechanism used to host background work.
OrderFlow's core is complete — layered (334), secured (335), persisted and queried efficiently (336), cached correctly (337), and now processed asynchronously with genuine durability (338-339). The build continues in 340, where OrderFlow gets the logging and observability a real production system can't ship without.
dotnetmadeeasy.com — Learn C# and .NET, the right way.