Intermediate showed you the shape. This lesson makes you design one for real — with the validation, the composition decisions, and the honest limits that come with it.
Back in Intermediate, WhereNotNull taught you the recipe for a custom LINQ extension: a static method, this IEnumerable<T>, yield return to stay lazy, and a public/private split so validation runs eagerly instead of silently deferring. That recipe is correct — and also, on its own, a little too easy. WhereNotNull is a one-item-in, zero-or-one-item-out filter. It never has to think about state that spans multiple items, it never has to decide what its own return type should even be, and it never has to reckon with the fact that yield return — the very trick that makes it lazy — only works for one specific kind of LINQ: LINQ-to-Objects.
This lesson builds a genuinely harder custom operator — Batch<T>, which groups a sequence into fixed-size chunks — and along the way covers the real design questions a simple filter never forces you to answer: what return type keeps an operator composable, how to validate arguments correctly when the iterator has to track state across items instead of just one, and why an operator built this way only ever works against IEnumerable<T> — never against IQueryable<T> translated to SQL.
A custom LINQ operator, at this level, is still just a static extension method on IEnumerable<T> — nothing about the mechanical recipe has changed since Intermediate. What's different here is the design work: deciding what shape the operator's output should take, how it should behave with invalid input (a negative batch size, a null source), and how it holds onto state while iterating, since some real operators — unlike a pure filter — need to remember something about the items they've already seen.
Batch<T> takes a flat sequence and a chunk size, and yields fixed-size arrays, each containing up to that many consecutive items — the last chunk shorter if the source doesn't divide evenly:
public static IEnumerable<T[]> Batch<T>(this IEnumerable<T> source, int size).NET 9 actually shipped an official equivalent of this — Enumerable.Chunk — so building Batch here is deliberately a learning exercise, not a gap you need to fill in real code: you already have .Chunk(size) available. The value is in seeing exactly what it takes to build something like it correctly, from first principles, with the state-tracking and validation concerns a plain filter never exposes.
Batching shows up constantly in real systems: sending 10,000 emails through a provider that only accepts 100 recipients per API call, inserting 50,000 rows through a bulk-insert API capped at 500 rows per batch, or paging through a huge in-memory report for a UI that renders 20 rows at a time. Without a dedicated operator, every one of those call sites reinvents the same manual chunking loop:
// The same manual chunking logic, rewritten at every call site
var recipients = GetAllRecipients(); // 10,000 items
var chunk = new List<Recipient>();
foreach (var r in recipients)
{
chunk.Add(r);
if (chunk.Count == 100)
{
SendBatch(chunk);
chunk = new List<Recipient>();
}
}
if (chunk.Count > 0)
SendBatch(chunk); // don't forget the leftover partial batch!// Written once, reused everywhere, and the "leftover partial batch" edge case
// can never be forgotten again — it's handled inside Batch itself
foreach (var chunk in recipients.Batch(100))
SendBatch(chunk);That manual version has a genuinely easy bug to introduce — forgetting the trailing partial chunk after the loop ends. Wrapping the pattern in a well-tested operator means that edge case gets solved exactly once, not re-solved (or re-forgotten) at every call site.
[1, 2, 3, 4, 5, 6, 7][ [1,2,3], [4,5,6], [7] ]
IEnumerable<T[]>, not List<T[]> or some custom BatchResult<T> type. Returning the most general, standard LINQ-shaped type — a plain IEnumerable<T> of something — is what lets the result chain straight into .Select(...), .Where(...), or another foreach, exactly like a built-in operator. A narrower, custom return type would force every caller to unwrap it before using ordinary LINQ on the result.WhereNotNull: a public method checks source for null and size for validity immediately, before any iteration begins, then delegates to a private iterator method for the actual yield return work.WhereNotNull's stateless per-item check, the iterator here must accumulate items into a buffer across multiple loop iterations, and know when that buffer is full (yield it, start a new one) versus when the source runs out mid-buffer (yield whatever's left).Batch never materializes the entire source into memory first — it holds only the current chunk's items in a small buffer, yielding and clearing it as it goes. This is what lets Batch work correctly even against a huge or unbounded source, streaming chunk by chunk.public static class EnumerableBatchExtensions
{
// Public, non-iterator method — validates immediately, on the call, every time
public static IEnumerable<T[]> Batch<T>(this IEnumerable<T> source, int size)
{
ArgumentNullException.ThrowIfNull(source);
ArgumentOutOfRangeException.ThrowIfLessThanOrEqual(size, 0);
return BatchIterator(source, size);
}
// Private iterator — holds a small buffer, one chunk at a time
private static IEnumerable<T[]> BatchIterator<T>(IEnumerable<T> source, int size)
{
var buffer = new List<T>(size);
foreach (var item in source)
{
buffer.Add(item);
if (buffer.Count == size)
{
yield return buffer.ToArray();
buffer.Clear();
}
}
// trailing partial chunk — only yielded if anything is left over
if (buffer.Count > 0)
yield return buffer.ToArray();
}
}
int[] numbers = [1, 2, 3, 4, 5, 6, 7];
foreach (int[] chunk in numbers.Batch(3))
Console.WriteLine($"[{string.Join(", ", chunk)}]");
// [1, 2, 3]
// [4, 5, 6]
// [7]Code → Meaning → Result: ArgumentOutOfRangeException.ThrowIfLessThanOrEqual(size, 0) runs the instant Batch(3) is called — before BatchIterator ever runs, exactly the eager-validation discipline from Intermediate. The buffer inside BatchIterator is the new piece: state that persists across loop iterations, cleared and refilled as chunks complete.
static IEnumerable<int> Naturals()
{
int n = 1;
while (true)
yield return n++;
}
// Batch never tries to consume the whole (infinite) source first —
// it only pulls as many items as one chunk needs, on demand
foreach (int[] chunk in Naturals().Batch(4).Take(2))
Console.WriteLine($"[{string.Join(", ", chunk)}]");
// [1, 2, 3, 4]
// [5, 6, 7, 8]If Batch had eagerly built a List<T[]> instead of using yield return, this would hang forever trying to consume an infinite sequence. Streaming one chunk at a time is what keeps it safe.
An email provider's API rejects any request with more than 100 recipients. A marketing job needs to notify 10,000 subscribers about a new feature. Batch turns this into a two-line loop instead of hand-rolled chunking logic scattered wherever bulk sends happen:
public record Subscriber(int Id, string Email);
public async Task NotifyAllSubscribersAsync(IEnumerable<Subscriber> subscribers, IEmailProvider provider)
{
foreach (Subscriber[] batch in subscribers.Batch(100))
{
await provider.SendBulkAsync(batch.Select(s => s.Email));
// each call to SendBulkAsync gets AT MOST 100 recipients,
// satisfying the provider's hard limit — automatically, for
// 10,000 subscribers or 10 million, without changing this code
}
}Notice batch.Select(...) in that loop body — because Batch returns plain T[] chunks (arrays implement IEnumerable<T>), every standard LINQ operator works on each chunk exactly as it would on any other sequence. That's the composability payoff of choosing IEnumerable<T[]> as the return type back in How It Works.
Picture a conveyor belt carrying items one at a time — that's your source sequence, arriving lazily, one per moment. At the end of the belt stands a packer with an empty box (the buffer). Each item that arrives goes into the current box. The moment the box holds exactly 100 items, the packer seals it, sends it off (yield return), and grabs a fresh empty box. If the belt stops with the box only partially full, the packer still seals and sends that last, smaller box rather than leaving it standing there forever. The packer never needs to see the whole belt's worth of items at once — just enough to fill one box at a time.
IQueryable<T> providers (like EF Core) work by walking an Expression tree — a data structure describing the query — and translating recognized nodes into SQL. A method written with yield return, like BatchIterator here, compiles down to ordinary IL: a hidden class implementing IEnumerable<T>'s MoveNext() pattern. There is no expression tree involved anywhere — just executable code.someList.Batch(100) runs entirely in memory, in your application's own process — there's nothing to translate, because nothing needs to be sent anywhere else to execute. This is exactly why Batch, as built here, works perfectly against a List<T>, an array, or any other in-memory IEnumerable<T>.Batch were called on context.Products (an IQueryable<Product>), C# would silently bind it to the IEnumerable<T> overload — since no matching IQueryable<T> overload exists — which forces EF Core to materialize the entire Products table into memory first, then run Batch against that in-memory list. That's a full, unfiltered table load happening silently, exactly the kind of accidental client-side evaluation the LINQ-to-EF-Core lesson later in this Part warns about explicitly.Expression<TDelegate> instead of a plain delegate, and manually building or rewriting expression tree nodes that a real provider (like EF Core's) knows how to walk — genuinely advanced, provider-specific work, and a completely different skill from writing a yield return iterator. Every custom operator in this lesson is a LINQ-to-Objects tool only — worth knowing clearly, not treating as a limitation to "fix."It compiles — C# will happily resolve context.Products.Batch(100) against the IEnumerable<T> overload, since IQueryable<T> inherits from IEnumerable<T>. But "compiles" and "translates to efficient SQL" are unrelated questions, as the LINQ-to-EF-Core lesson later in this Part covers in depth. Here, specifically: it compiles, and then it silently pulls the entire table into memory before batching runs. No exception, no warning — just a much larger query than you asked for.
Enumerable.Chunk, shipped since .NET 6, does exactly what Batch does here — and it's the one you should actually call in real code. Building Batch from scratch in this lesson is purely to understand, by construction, how an operator like it works: the buffering, the eager validation, the laziness. Once you understand that, prefer the framework's own, already-tested version over a hand-rolled duplicate.
size inside the yield return iterator itself Putting ArgumentOutOfRangeException.ThrowIfLessThanOrEqual(size, 0) directly inside BatchIterator — the same deferred-validation bug the Intermediate lesson covered for WhereNotNull, and one that's just as easy to reintroduce here: the exception wouldn't fire until the first foreach iteration, not at the moment .Batch(-1) is actually called. Keep validation in the public, non-iterator method, exactly as shown throughout this lesson.
yield return buffer; followed by buffer.Clear() on the same List<T> instance — since the caller might not have finished consuming the first chunk before the second one overwrites the same underlying list, this can silently corrupt data the caller thought it already had. Yield a fresh, independent array with buffer.ToArray() (or a new list) each time, so every chunk the caller receives is a stable, independent snapshot.
Only yielding when buffer.Count == size, with no check after the foreach loop ends — silently drops the last, smaller chunk whenever the source's length isn't an exact multiple of the batch size. Always check if (buffer.Count > 0) once the loop over source has finished, exactly as shown in the Simple Example.
List<T>, an array, a stream already materialized) — LINQ-to-Objects territory.Chunk, DistinctBy, GroupBy, and others cover a lot of ground already).IQueryable<T> you want translated to SQL — a plain yield return operator will silently force full materialization instead, as covered in Under the Hood.yield return, that runs entirely inside your process."IEnumerable<T> (or IEnumerable<T[]>, etc.) — the most general shape — to keep it composable with the rest of LINQ.yield return iterator.yield return operator has no expression tree, so it can never translate to SQL against an IQueryable<T>.
Batch<T> — often need to track state across multiple items, not just check one item at a time.IEnumerable<T>) so the result keeps composing with the rest of LINQ.yield return method) — the same discipline from Intermediate, still essential here.Chunk or DistinctBy is a legitimate way to learn how it works — prefer the framework's actual version in production code.yield return-based operator only works for LINQ-to-Objects — it has no expression tree, so it cannot translate to SQL, and calling it on an IQueryable<T> silently forces the entire source into memory first.You've designed a genuinely more sophisticated custom LINQ operator, and seen exactly where it stops working. Let's check the reasoning stuck.
1. Why does Batch<T> return IEnumerable<T[]> instead of a custom type like BatchResult<T>?
Correct: B
Why B is correct: As covered in How It Works, choosing a standard, general shape for the return type is exactly what keeps a custom operator composable — a caller can chain further LINQ onto the result without first unwrapping some bespoke type.
Why A is incorrect: Extension methods can return any type, including custom ones — the constraint here is a design choice about composability, not a language limitation.
Why C is incorrect: There's no meaningful performance difference tied to the return type shape here — the reasoning is entirely about how easily the result composes with other LINQ code.
Why D is incorrect: The return type has a real, practical consequence for whether the operator "feels like" a LINQ operator at the call site — this is a genuine design decision, not a stylistic one.
Reinforcement: Favor the most general, standard shape LINQ already uses, so custom operators snap into the rest of a query the same way built-in ones do.
2. What happens if Batch<T> is called as context.Products.Batch(100), where context.Products is an IQueryable<Product> from EF Core?
Correct: C
Why C is correct: As explained in Under the Hood, since IQueryable<T> inherits from IEnumerable<T>, and no IQueryable<T>-specific overload of Batch exists, C# silently binds the call to the IEnumerable<T> version — which requires the entire source to already be enumerable in memory, forcing a full, unfiltered table load with no warning.
Why A is incorrect: A plain yield return operator has no expression tree for EF Core's provider to inspect — there is nothing for it to translate, so no SQL-level batching happens.
Why B is incorrect: This compiles fine — IQueryable<Product> satisfies the IEnumerable<T> constraint, since it inherits from it. The problem is a silent runtime behavior, not a compile error.
Why D is incorrect: EF Core's translation failure exception applies to expressions inside the IQueryable<T> pipeline itself — this scenario never reaches EF Core's translator at all, because the call already resolved to plain in-memory IEnumerable<T> LINQ before EF Core gets involved.
Reinforcement: A yield return-based custom operator silently forces full materialization when called on an IQueryable<T> — it compiles cleanly and gives no error, which is exactly what makes it a dangerous, easy-to-miss mistake.
3. Why must Batch<T>'s argument validation live in the public method, not inside BatchIterator?
Correct: B
Why B is correct: As reinforced in How It Works and Common Mistakes, an entire yield return method body — including any validation inside it — gets swept into the compiler-generated state machine, so it only executes once iteration actually begins, not at the moment the method is called.
Why A is incorrect: Iterator methods can absolutely contain if statements — the buffering logic inside BatchIterator itself uses one. The issue is strictly about *when* code inside such a method runs, not whether certain statements are allowed.
Why C is incorrect: Private methods can throw exceptions freely — accessibility has nothing to do with whether an exception can be thrown from a method.
Why D is incorrect: ArgumentOutOfRangeException (and any exception type) can be thrown from a method of any accessibility level — public, private, or otherwise.
Reinforcement: Whenever a custom operator needs both eager validation and a yield return iterator, split them into two methods — this is a recurring, essential pattern, not a one-off fix for a single example.
4. The lesson mentions that .NET already ships Enumerable.Chunk, which does essentially what Batch<T> does here. Why build Batch<T> from scratch anyway?
Correct: B
Why B is correct: As stated explicitly in What Is It? and Common Confusion, this exercise exists to show, by construction, exactly how an operator like this works — the buffering, the validation split, the laziness — while being clear that real code should reach for the framework's own tested implementation instead of duplicating it.
Why A is incorrect: Enumerable.Chunk is a current, actively supported part of .NET — nothing about it is deprecated.
Why C is incorrect: No performance claim like this was made, and there's no general reason a hand-rolled reimplementation would outperform a framework-shipped, well-tested equivalent.
Why D is incorrect: Enumerable.Chunk works on IEnumerable<T> — the same in-memory LINQ-to-Objects territory Batch<T> operates in, for the exact same reasons covered in Under the Hood.
Reinforcement: Building your own version of something the framework already provides is a legitimate way to learn — just don't ship the hand-rolled version when a tested, official one already exists.
You've now designed a genuinely non-trivial custom LINQ operator, and can explain exactly why it stops working the moment the source becomes an IQueryable<T> you want translated to SQL. Next up: giving that exact distinction — LINQ-to-Objects versus everything else — its proper, formal name.
dotnetmadeeasy.com — Learn C# and .NET, the right way.