Lesson 218 taught you what a deadlock is and why it happens. It never had to teach you how to find one in a process you can't step through — because on your own machine, you can always just reproduce it under a debugger. In production, you usually can't.
You already know exactly what a deadlock is. Lesson 218 walked the two classic shapes in full — the two-lock ordering deadlock and the notorious sync-over-async deadlock built on a captured SynchronizationContext — in enough depth that re-deriving any of it here would just waste your time. None of that comes back up in this lesson.
What lesson 218 couldn't teach you, because it wasn't the question yet, is this: a production service has gone completely unresponsive. Requests hang and never return. There's no exception, no crash, nothing in the logs pointing anywhere. You strongly suspect a deadlock — but you can't attach a debugger with breakpoints to a live process the way lesson 060 taught, and even if you technically could, the deadlock has already happened by the time you'd get there; there's no line of code to put a breakpoint on. This lesson is entirely about that gap: how you actually confirm a deadlock occurred in a process you can't step through, using a process dump and the SOS debugging extension — and, just as importantly, how you tell a genuine deadlock apart from thread pool starvation (lesson 319, next), which produces the exact same "everything is hanging" symptom for a completely different reason.
Diagnosing a live deadlock means capturing a frozen snapshot of a hung process's entire state — every thread, every stack, every lock currently held — and reading that snapshot afterward to find the specific cycle of threads waiting on each other. You're not watching it happen; you're photographing the aftermath and reconstructing the scene from the photograph.
A process dump is a complete, point-in-time capture of a process's memory — for a .NET process, that includes every managed thread's call stack, every object on the heap, and the CLR's own internal bookkeeping about locks and synchronization objects. dotnet-dump is the diagnostics-suite tool (same family as dotnet-trace and dotnet-counters from lessons 315 and 317) that captures this snapshot from a running process, and analyzes it afterward using SOS — the CLR's own debugging extension, which understands managed threads, stacks, and lock objects well enough to answer questions no generic memory viewer could.
Lesson 315 explained why you generally can't pause a live production process to inspect it — pausing drops real traffic. A genuinely deadlocked process is the one situation where that concern mostly evaporates: if the threads serving requests are already permanently stuck, they aren't making progress anyway, and capturing a dump doesn't cost you anything those threads were still doing. This is exactly why dotnet-dump collect is a safe, standard first move the moment you suspect a genuine hang — you're not disrupting healthy work, you're documenting work that had already stopped.
Lesson 218's interactive debugging works beautifully when you can reproduce the exact interleaving that causes a deadlock, on demand, on your own machine. Production deadlocks are rarely that cooperative — they might depend on a specific, rare timing window under real concurrent load that never shows up in a local repro attempt. By the time an alert fires and an engineer is looking at the system, the deadlock already happened; there's no "step forward one line" available because nothing is stepping forward at all, forever.
Instead of trying to catch the deadlock as it forms, dotnet-dump captures everything at once, after the fact — every thread's exact call stack, and the CLR's own record of which synchronization objects are currently held by which thread. That snapshot can then be analyzed calmly, offline, without the process even needing to still be running — the investigation and the incident are decoupled in time, which is exactly what a live, unrepeatable hang requires.
Before touching any tooling, it's worth settling this distinction clearly, because it's the single most common misdiagnosis in this territory, and lesson 319 depends on you having it straight: both a real deadlock and thread pool starvation present, to a user, as "the request never comes back." They are not the same failure, they don't share a root cause, and they don't share a fix.
The one-sentence version worth memorizing: a deadlock is threads waiting on each other; starvation is threads waiting for a free worker thread that isn't there yet. No thread in a starvation scenario is "held hostage" by another thread's lock — there's no cycle to find, because there's nothing to be circular about.
dotnet-dump collect -p <pid> — writes a full snapshot of the process to a dump file. This is the exact "capture evidence before restarting" advice from lesson 315, applied here.dotnet-dump analyze <dumpfile> — opens an interactive SOS session against the frozen snapshot. The original process can be restarted or even killed at this point; the dump is now a fully self-contained artifact.threads SOS command lists every managed thread in the dump. Switching to one and running clrstack shows exactly where that thread was stopped — which method, and often which specific lock-acquisition call it was blocked inside.syncblk SOS command lists every active sync block in the process — each lock currently held, which thread owns it, and which threads are waiting on it. This is the single most direct way to confirm a deadlock: it shows ownership and waiting relationships explicitly, not just individual stack traces.syncblk shows Thread A waiting on a lock owned by Thread B, and Thread B waiting on a lock owned by Thread A, that's the exact cycle lesson 218 defined — now confirmed from real, captured evidence, not assumed from a hunch.Here's a simplified, illustrative shape of what a syncblk listing communicates once you're looking at the two-lock deadlock from lesson 218 frozen inside a dump — the exact field names and formatting vary by SOS version, but the information it's built to surface is always this:
> syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Object Waiters
2 0000021a0 1 0 0x4a10 (Thread A) 0x02f... Thread B
3 0000021b8 1 0 0x4a24 (Thread B) 0x031... Thread A
> setthread 0x4a10
> clrstack
... TransferFromAccountXToY(...)
... at System.Threading.Monitor.Enter(Object obj) ← blocked here, waiting for _lockB
> setthread 0x4a24
> clrstack
... TransferFromAccountYToX(...)
... at System.Threading.Monitor.Enter(Object obj) ← blocked here, waiting for _lockAMeaning: The syncblk table alone already tells the story — Thread A owns one lock and has Thread B listed as a waiter; Thread B owns the other lock and has Thread A listed as a waiter. Following each thread's clrstack confirms exactly what lesson 218 predicted: each is parked inside Monitor.Enter, permanently, waiting for a lock the other thread will never release. This is the confirmed circular-wait signature — not a guess based on "the app seems stuck," but read directly off frozen, real evidence.
An inventory service's health probe (lesson 302) starts failing intermittently, and support reports that certain order-processing requests simply never complete — no timeout, no error, no recovery, ever, for those specific requests. Other endpoints on the same instance keep working fine, which already rules out a whole-process crash or resource exhaustion. Following this lesson's workflow, an engineer runs dotnet-dump collect against the hung instance before restarting it, then dotnet-dump analyze offline. syncblk reveals exactly two threads holding each other's locks — traced back to a recent change where a new "reserve stock" code path and an existing "release stock" code path acquire the same two per-warehouse locks, but in opposite order, exactly the lesson-218 pattern, shipped without anyone noticing because it only manifests under a specific, rare concurrent timing. The fix is the same one lesson 218 already gave you — consistent lock ordering — but finding it here required this lesson's dump-analysis workflow, not a debugger, because the bug wasn't reproducible on demand.
You weren't there to watch the standoff form — there was no stakeout, no live surveillance running the moment it happened. What you have instead is a single, comprehensive photograph of the frozen scene, taken right after the fact: every person's exact position, what's in each of their hands, and who's facing whom. A process dump is exactly this photograph. syncblk is like a caption on the photo listing who's holding what and who's reaching for it — and once you have that caption, tracing the cycle ("this one is reaching for what that one's holding, who is in turn reaching for what this one's holding") doesn't require having witnessed the standoff form at all. The photograph alone is enough to prove the standoff exists.
dotnet-dump collect relies on the operating system's own native process-snapshot facility (on Linux, the CLR's createdump mechanism; a comparable mechanism on Windows and macOS) to capture the process's full memory image — stacks, heap, and loaded module information — at one instant. Crucially, that raw memory image on its own is just bytes; it has no notion of "a C# thread" or "a lock" by itself. That's exactly what SOS supplies: a CLR-aware debugging extension that knows how to walk that raw memory and reconstruct managed concepts from it — which native OS threads correspond to which managed threads, what each one's managed call stack actually looked like, and which CLR-level sync blocks (the data structure backing every lock statement, per lesson 218's "under the hood" section) were held by whom. dotnet-dump analyze is the harness that loads a dump and hosts SOS's commands interactively, which is why threads, clrstack, and syncblk all work identically whether you're inspecting a dump captured seconds ago or one captured last week.
This is the exact misdiagnosis the Big Picture section above exists to prevent. Hanging requests are also the headline symptom of thread pool starvation (lesson 319) — a completely different root cause with a completely different fix. The only way to actually tell them apart is to look: a dump showing a genuine syncblk cycle confirms a deadlock; a dump showing plenty of threads simply not running anything in particular, alongside dotnet-counters showing a large, growing ThreadPool queue, points to starvation instead.
Near-idle CPU is consistent with a genuine deadlock, but it's equally consistent with starvation (threads simply have no work assigned to run) or even a process legitimately waiting on slow external I/O with nothing pathological going on at all. CPU usage alone narrows the space of possibilities; only syncblk and a confirmed cycle actually proves a deadlock occurred.
The instinctive first move — kill and restart — relieves the symptom immediately but destroys the only evidence that could confirm what actually happened, or rule out a deadlock in favor of a different cause entirely.
Wherever operationally possible, run dotnet-dump collect first. Since the hung threads weren't making progress anyway, this costs you essentially nothing you weren't already losing.
Spending an hour reading every lock block in a large codebase, hunting for a plausible ordering bug, on the assumption that the symptom alone proves it's a deadlock.
Confirm via a dump and syncblk first — it tells you immediately whether a real cycle exists at all, and if it does, exactly which objects and which threads are involved, turning a codebase-wide search into a targeted one.
Reaching for the only tool a local-development background has taught (lesson 060's debugger), which — even where technically reachable — offers no way to "rewind" to the moment a deadlock already happened, and would pause a process that's arguably already not making progress on plenty of its threads.
Reach for dotnet-dump specifically for a suspected hang — it's built for exactly this after-the-fact, non-interactive investigation.
dotnet-dump collect, then read it offline with dotnet-dump analyze.dotnet-dump collect captures a full snapshot of a live (or hung) process's threads, stacks, and sync blocks; dotnet-dump analyze loads that snapshot offline for interactive inspection via SOS — the CLR's own debugging extension.threads lists every managed thread; clrstack shows where a specific one was frozen; syncblk shows exactly who owns each lock and who's waiting on it — the direct, evidence-based way to confirm a real circular wait.You've seen how to confirm a deadlock in a process you can't step through, and — critically — how it differs from thread pool starvation. Let's check it landed.
1. Why is capturing a process dump generally considered safe to do against a process you suspect is genuinely deadlocked, even though lesson 315 warned against pausing a live production process?
Correct: B
Why B is correct: This is the exact nuance the lesson draws out — a genuinely hung process's stuck threads aren't producing useful work regardless of whether you capture a dump, so the usual "don't pause a live process" caution from lesson 315 doesn't really apply the same way here.
Why A is incorrect: Capturing a dump does involve a brief pause to take the snapshot — the point isn't that there's zero pause, it's that the pause costs you nothing extra for threads that were already stuck.
Why C is incorrect: The lesson explicitly frames dotnet-dump as a standard production diagnostic tool for a suspected hang, not something restricted to staging.
Why D is incorrect: A deadlock typically affects specific threads/requests, not the entire process — other endpoints and threads can remain completely healthy, as the Real-World Example illustrates, so there is often still real traffic elsewhere in the same process.
Reinforcement: The safety argument is specific: stuck threads have nothing left to lose from a brief snapshot.
2. In a captured dump, what does the syncblk SOS command specifically show, and why is it the most direct way to confirm a genuine deadlock?
Correct: B
Why B is correct: This is exactly what the lesson describes — syncblk shows ownership and waiting relationships explicitly for every active lock, which is what lets you directly read off a circular wait instead of piecing it together indirectly from separate stack traces alone.
Why A is incorrect: syncblk is specifically about synchronization objects (locks) and their owners/waiters — it isn't a historical log of every method call during the process's lifetime.
Why C is incorrect: CPU usage per thread isn't what syncblk reports — that kind of metric belongs to tools like dotnet-counters or dotnet-trace (lesson 317), not SOS's lock-inspection commands.
Why D is incorrect: Dump analysis is purely observational — it inspects a frozen snapshot and takes no automatic action on the (now potentially no-longer-running, or still-running-unaffected) original process.
Reinforcement: syncblk is the command that turns "I suspect a deadlock" into "confirmed: Thread A owns lock X and waits on lock Y, Thread B owns lock Y and waits on lock X."
3. A production service has hanging requests. A dump is captured, and syncblk shows several threads simply not holding or waiting on any sync block at all — they're just not running anything, while dotnet-counters shows a large, growing ThreadPool queue length. What does this most likely indicate?
Correct: B
Why B is correct: The absence of any real syncblk cycle, combined with a growing ThreadPool queue, is exactly the starvation signature this lesson contrasts against a deadlock — no thread is waiting on another thread's lock; there simply aren't enough available threads to keep up with queued work.
Why A is incorrect: A deadlock requires a confirmed syncblk cycle — threads owning and waiting on each other's locks. Here, syncblk shows no such ownership/waiting relationship at all, ruling out a genuine deadlock.
Why C is incorrect: A sync-over-async deadlock (lesson 218) is still a real deadlock — it would show up in syncblk (or an equivalent captured-context wait) as threads/continuations stuck waiting on each other, which isn't what's described here.
Why D is incorrect: A growing ThreadPool queue reflects a backlog of work waiting to be dequeued, not heap object reachability — that's an unrelated concern from lesson 316, diagnosed with a different tool (dotnet-gcdump) entirely.
Reinforcement: No cycle in syncblk, plus a growing ThreadPool queue, points to starvation — exactly the distinction the next lesson builds on.
4. Why does this lesson deliberately avoid re-explaining what causes a deadlock, or why sync-over-async deadlocks occur?
Correct: B
Why B is correct: This lesson explicitly builds on lesson 218 as a prerequisite, deliberately not re-deriving the two-lock or sync-over-async deadlock shapes — its entire focus is the missing half of the picture: how to confirm a deadlock happened in a process you can't interactively step through.
Why A is incorrect: Nothing in the lesson suggests deadlocks are no longer relevant — quite the opposite, the Real-World Example shows one shipping in a recent production change.
Why C is incorrect: A sync-over-async deadlock is still, ultimately, threads/continuations waiting on each other — it's fully diagnosable via a dump and SOS, just potentially requiring inspection of the captured SynchronizationContext state rather than a plain lock statement.
Why D is incorrect: This conflates deadlocks with database bottlenecks (lesson 320), an entirely different failure mode with a different root cause and different diagnostic tools.
Reinforcement: Lesson 218 owns the "what and why," this lesson owns the "how do you prove it happened" — deliberately non-overlapping.
5. An engineer suspects a production hang is a deadlock, immediately restarts the affected pod to restore service, and only afterward tries to investigate the cause. What did this approach cost them, according to this lesson?
Correct: B
Why B is correct: This is exactly Mistake 1 from the lesson — restarting before capturing a dump relieves the symptom but destroys the frozen thread/lock state that dotnet-dump and syncblk depend on, making it impossible afterward to confirm whether it was truly a deadlock.
Why A is incorrect: The lesson explicitly recommends capturing a dump before restarting wherever operationally possible — restarting first is called out as a common mistake, not a recommended order.
Why C is incorrect: Restarting a process has no inherent connection to causing a memory leak — that's an unrelated claim not supported anywhere in this lesson or lesson 316.
Why D is incorrect: Ordinary application logs generally don't capture thread stacks or sync block ownership at the level of detail a dump does — logging alone doesn't substitute for the evidence a dump would have preserved.
Reinforcement: Whenever it's operationally feasible, capture the dump before you restart — you rarely get a second chance at that exact frozen evidence.
You now know how to confirm — not guess at — a deadlock in a live process using dotnet-dump and SOS, and exactly how it differs from a lookalike failure mode. Next up: that lookalike, in full. Thread Pool Starvation.
dotnetmadeeasy.com — Learn C# and .NET, the right way.