343 - Containerization · dotnetmadeeasy.com

A tested, monitored OrderFlow is still just source code until it ships as something a cluster can actually run, restart, and know the health of. That something is a container image.

Lesson 298 taught you what a container actually is and how to write a correct, multi-stage .NET Dockerfile. Lesson 299 covered the ASP.NET-Core-specific details of running well inside one. Lesson 302 covered Kubernetes health probes — liveness, readiness, and startup — and precisely what each one does when it fails, built on the health-check endpoints from lesson 268. Each of those lessons taught the mechanics once, generically. This lesson writes OrderFlow's actual Dockerfile and its actual Kubernetes probe configuration, using OrderFlow's real dependencies — a Postgres database (336), an IDistributedCache (337), and a Kafka broker (339) — as the concrete health checks that back its readiness endpoint.

Nothing here reintroduces what a container is or how a probe schedule works. This lesson is entirely about the specific choices OrderFlow's Dockerfile and Kubernetes manifest actually need to make, and why each one is made the way it is.

What Is It?

The Simple Explanation

Containerizing OrderFlow means packaging its published output into a small, multi-stage Docker image (298) that runs as a non-root user, exposes the two health endpoints lesson 268 already gave it, and gets a Kubernetes Deployment manifest whose liveness probe asks "is this process itself stuck" while its readiness probe asks "can this instance actually reach Postgres, the cache, and Kafka right now" — three completely different dependencies, checked by one endpoint, with three completely different failure signatures.

The Technical Definition

OrderFlow's image is built via the standard two-stage pattern from lesson 298 — an SDK-based build stage compiling the solution, and a slim aspnet runtime stage running only the published output — with lesson 299's containerized-ASP.NET-Core specifics applied on top: a non-root user, environment-driven configuration (previewing lesson 345), and graceful shutdown handling for in-flight requests during a rolling deploy. Its Kubernetes manifest configures a livenessProbe against /health/live (a narrow, dependency-free check) and a readinessProbe against /health/ready (which, per lesson 302, includes real IHealthCheck implementations for the database, cache, and Kafka broker), so a temporary outage in any one dependency pulls the pod from traffic without triggering an unnecessary restart of an otherwise-healthy process.

Why Does It Exist?

The Problem — OrderFlow Has Three Dependencies, Not One, and They Fail Independently

Lesson 302's example was a single database health check. OrderFlow has three real external dependencies that can each fail on their own schedule: Postgres (336) can be mid-failover, the distributed cache (337) can be temporarily unreachable while IDistributedCache falls back or errors, and the Kafka broker (339) can be undergoing a partition rebalance. A readiness probe that only checks the database would happily keep routing checkout traffic to a pod that can't actually reach Kafka to publish OrderPlaced — a silently broken instance still marked healthy.

The Solution — One Readiness Endpoint, Three Real Dependency Checks

The fix is exactly lesson 268's IHealthCheck composition model, applied to all three of OrderFlow's real dependencies at once: /health/ready aggregates a Postgres check, a distributed-cache check, and a Kafka-connectivity check, and only reports healthy when all three genuinely respond. /health/live stays deliberately narrow — it answers "is the .NET process itself still responsive," with none of those three dependencies involved at all, exactly the liveness/readiness separation lesson 302's Mistake 1 warned against conflating.

Big Picture — OrderFlow's Image and Probe Shape

Build Stage

mcr.microsoft.com/dotnet/sdk:10.0 — compiles and publishes; never ships

Final Stage

mcr.microsoft.com/dotnet/aspnet:10.0 — only the published output, running as a non-root user

livenessProbe → /health/live

No dependency checks — a failure here means the process itself is genuinely stuck

readinessProbe → /health/ready

Checks Postgres (336), the distributed cache (337), and Kafka (339) — any one failing pulls the pod from traffic

How It Works — OrderFlow's Dockerfile

The .csproj files are copied and restored before the rest of the source, exactly so Docker's build-layer cache can skip the restore step entirely on a code-only change — a real, ordinary win independent of anything specific to OrderFlow. USER $APP_UID is the concrete instance of lesson 299's non-root guidance: the final container never runs as root, so a container-escape vulnerability in the running app doesn't hand an attacker root inside the image.

Simple Example — OrderFlow's Readiness Check Composition

builder.Services.AddHealthChecks()
    .AddNpgSql(builder.Configuration.GetConnectionString("OrderFlowDb")!, name: "postgres")
    .AddCheck<DistributedCacheHealthCheck>("distributed-cache")
    .AddKafka(options =>
    {
        options.BootstrapServers = builder.Configuration["Kafka:BootstrapServers"];
    }, name: "kafka");

app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    // Deliberately empty predicate — no registered check runs here at all.
    Predicate = _ => false
});

app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    // Every registered check (postgres, distributed-cache, kafka) runs here.
    Predicate = _ => true
});

Meaning: /health/live's empty predicate is the point, not an oversight — it guarantees liveness can never fail because of Kafka being briefly unreachable. /health/ready's Predicate = _ => true runs every registered check, so a readiness failure genuinely means "this specific instance can't currently serve a request correctly," which is exactly the question lesson 302's readinessProbe is built to ask.

Real-World Example — OrderFlow's Kubernetes Probe Configuration

containers:
  - name: orderflow-api
    image: registry.example.com/orderflow-api:1.4.2
    ports:
      - containerPort: 8080
    startupProbe:
      httpGet: { path: /health/live, port: 8080 }
      failureThreshold: 30
      periodSeconds: 2       # up to 60s of patient startup checking
    livenessProbe:
      httpGet: { path: /health/live, port: 8080 }
      periodSeconds: 10
      failureThreshold: 3
    readinessProbe:
      httpGet: { path: /health/ready, port: 8080 }
      periodSeconds: 5
      failureThreshold: 2

Walk through what each failure actually does, using lesson 302's exact mapping: if Kafka undergoes a brief partition rebalance, /health/ready starts returning 503, the readinessProbe fails twice, and Kubernetes pulls that one pod out of the Service's traffic rotation — the process keeps running, untouched, and rejoins traffic automatically the moment Kafka answers again. If instead the .NET process itself genuinely deadlocks — a scenario lesson 318 covered directly — /health/live stops responding at all, the livenessProbe fails three times, and Kubernetes kills and restarts the container, because no amount of waiting fixes a truly stuck process. The startupProbe exists because OrderFlow's own startup — EF Core migrations check, cache warm-up, Kafka consumer group join — genuinely takes longer than its steady-state liveness schedule would tolerate; without it, a slow cold start could get killed before the app ever finished starting.

Analogy

A Cook Who's Conscious but Out of Ingredients

Lesson 302 already used a kitchen analogy for liveness versus readiness in general; OrderFlow makes it concrete. A line cook who's fully conscious and moving (liveness: fine) can still be genuinely unable to plate a specific dish because the walk-in fridge (Postgres), the prep station (the cache), or the ticket printer (Kafka) is temporarily down — any one of three completely different problems, each making the cook "not ready" for a different reason, none of which means dragging the cook out of the kitchen. OrderFlow's readiness check is the manager doing a real round of "can you actually get to the fridge, the prep station, and the printer right now" — not just glancing over to confirm the cook is standing up.

Under the Hood — Why Composing Checks, Not Writing One Giant Check, Matters

Each of OrderFlow's three readiness checks — AddNpgSql, the custom DistributedCacheHealthCheck, AddKafka — is registered as its own independent IHealthCheck, and ASP.NET Core's health check middleware runs all of them and aggregates the worst result. This matters beyond tidiness: each check can carry its own name in the aggregated response, which means a monitoring dashboard (341) or an on-call engineer looking at a failing readiness probe doesn't just see "not ready" — they see specifically which dependency is the problem, without needing to open a shell into the pod at all. A single hand-rolled check that pings all three dependencies in one method and returns a single boolean throws that diagnostic detail away — exactly the kind of signal loss lesson 340's whole logging discipline exists to avoid elsewhere in the pipeline.

Common Confusion

1. "More dependency checks on the liveness probe means more safety" — it means more unnecessary restarts

It's tempting to add the Postgres/cache/Kafka checks to both probes "just to be thorough." Lesson 302's own Mistake 1 already named the exact consequence: a temporary database blip would then fail the liveness probe too, and Kubernetes would restart a perfectly healthy OrderFlow process to "fix" an outage a restart can't touch. Dependency checks belong on readiness only.

2. "A failing readiness check means something is broken and needs fixing immediately" — often it means the system is working exactly as designed

A pod dropping out of rotation because Kafka is mid-rebalance, and rejoining automatically once it clears, isn't a bug — it's the readiness probe doing precisely the job lesson 302 described: keeping temporarily-unable-to-serve traffic away from a customer, with zero human intervention required, for something that resolves itself in seconds.

Common Mistakes

Mistake 1 — Shipping the SDK image to production

Building OrderFlow's image from mcr.microsoft.com/dotnet/sdk:10.0 as the only stage — it works, but ships the entire compiler and MSBuild toolchain into a production container that never needs to compile anything.

Always use the multi-stage pattern from lesson 298 — build with the SDK, run from aspnet, exactly as OrderFlow's Dockerfile does above.

Mistake 2 — No startupProbe on a service with real startup work to do

Relying on a tight livenessProbe schedule alone for a service that runs EF Core migration checks and joins a Kafka consumer group on startup — under load, or on a slower node, the pod can get killed before it ever finishes starting.

Size a startupProbe generously for OrderFlow's actual worst-case startup time, exactly as configured above, and keep the liveness probe's own schedule tight for steady-state operation.

Mistake 3 — Running the container as root

Omitting USER $APP_UID from the final stage — the default is root, which means any container-escape vulnerability in OrderFlow or one of its dependencies hands an attacker root privileges inside the container.

Always run production containers as a non-root user, exactly as lesson 299 recommends — it costs nothing functionally and closes off a real, avoidable class of privilege escalation.

When Should I Use It?

Mental Model

Multi-stage Dockerfile (298) = SDK to build, aspnet to run, non-root user (299)
/health/live = "is the process itself stuck?" — no dependency checks, ever
/health/ready = "can this instance reach Postgres, the cache, and Kafka right now?" — three checks, composed
startupProbe = patient checking during real startup work, before the tighter liveness schedule takes over

Remember: liveness failure → restart; readiness failure → pull from traffic, no restart. OrderFlow's three real dependencies only ever affect readiness.

Key Takeaway

OrderFlow's Dockerfile is lesson 298's standard multi-stage pattern, plus lesson 299's non-root-user guidance applied concretely — an SDK build stage that never ships, and a slim aspnet final stage that does.
OrderFlow has three real external dependencies — Postgres, the distributed cache, Kafka — each composed as its own named IHealthCheck on /health/ready, so a failure names its cause instead of hiding behind one opaque boolean.
/health/live deliberately never checks any of those three dependencies — a liveness failure should only ever mean the .NET process itself is genuinely stuck, per lesson 302's exact liveness/readiness distinction.
A startupProbe sized for OrderFlow's real cold-start work (migrations, cache warm-up, Kafka consumer group join) protects it from being killed before it ever finished starting.

Check Your Understanding

You've seen OrderFlow's actual Dockerfile and Kubernetes probe configuration, built on lessons 298, 299, and 302. Let's confirm it clicked.

1. Why does OrderFlow's /health/live endpoint deliberately exclude checks for Postgres, the cache, and Kafka?

A) Because .NET's health check middleware only supports one registered check per endpoint
B) Because a liveness failure triggers a container restart, and a temporary outage in an external dependency can't be fixed by restarting an otherwise-healthy process — those failures belong on readiness instead
C) Because Postgres, the cache, and Kafka health checks are not compatible with ASP.NET Core's health check API
D) Because liveness probes are checked less frequently than readiness probes, so dependency checks would be too slow there

Show answer

Correct: B

Why B is correct: This is exactly lesson 302's liveness/readiness distinction, applied to OrderFlow's three real dependencies — a restart is the wrong tool for a temporary outage in something the process itself doesn't control, so those checks belong on readiness only.

Why A is incorrect: The middleware supports any number of registered checks per endpoint, filtered by a predicate — that's exactly how /health/ready aggregates all three.

Why C is incorrect: All three checks are registered through the same IHealthCheck API — they're fully compatible, just deliberately excluded from the liveness predicate.

Why D is incorrect: Probe frequency isn't the reasoning — the concern is what action a failure triggers (restart vs. pull-from-traffic), not how often the check runs.

Reinforcement: A dependency check belongs on readiness specifically because its failure shouldn't trigger a restart.

2. Kafka undergoes a brief partition rebalance, and OrderFlow's /health/ready starts returning 503 for one pod. What does Kubernetes do?

A) It restarts the container, since any health check failure triggers a restart
B) It removes that pod from the Service's traffic rotation, without restarting it — the process keeps running and rejoins traffic automatically once /health/ready succeeds again
C) It deletes the pod permanently and does not replace it
D) It ignores the failure, since Kafka is an external dependency outside Kubernetes's awareness

Show answer

Correct: B

Why B is correct: This is precisely the readinessProbe behavior from lesson 302, applied to OrderFlow's Kafka dependency — pull from traffic, leave the process running, rejoin automatically once healthy again.

Why A is incorrect: Only a livenessProbe failure triggers a restart — a readinessProbe failure explicitly does not touch the container process at all.

Why C is incorrect: The pod is never deleted for a readiness failure — it's simply excluded from the Service's active endpoint list until it recovers.

Why D is incorrect: The whole point of registering a Kafka health check on /health/ready is to make Kubernetes aware of exactly this kind of external dependency issue.

Reinforcement: A readiness failure is a routing decision, not a lifecycle decision — the process is untouched.

3. Why does OrderFlow register three separate, named health checks (postgres, distributed-cache, kafka) instead of one custom check that pings all three dependencies and returns a single boolean?

A) ASP.NET Core's health check middleware technically forbids combining multiple dependency pings inside one custom check
B) Separate, named checks let a failing readiness probe's response identify specifically which dependency is the problem, without needing to open a shell into the pod
C) Named checks run faster than a single combined check, regardless of what each one actually does
D) Kubernetes requires at least three registered health checks per readiness probe

Show answer

Correct: B

Why B is correct: This is exactly the Under the Hood reasoning — composing independent, named checks preserves diagnostic detail in the aggregated response, so an on-call engineer sees which dependency failed, not just that something did.

Why A is incorrect: Nothing technically forbids a single combined check — it's a design choice this lesson argues against for diagnostic reasons, not a hard restriction.

Why C is incorrect: Performance isn't the reasoning given — the benefit described is diagnostic clarity in the response, not raw speed.

Why D is incorrect: There's no such Kubernetes requirement — the number and shape of registered checks is entirely an application-level decision.

Reinforcement: Composed, named checks preserve exactly the information a real incident response needs — which dependency, not just "unhealthy."

Next up: lesson 344 goes looking inside this same containerized OrderFlow for a real, measured slow spot — applying Part V's profiling and benchmarking toolkit, and Part XI's performance/load testing, to find and fix it properly.