Fan-out-then-finalize: an empty batch reports complete, and finalizer ordering is load-bearing

[rcbevans]

wait_for_batch is documented as the convenience helper for fan-out-then-finalize, and it is the only fan-in idiom in the library. Building on it, I hit two hazards that are both silent, and both live in the gap between "enqueue the items" and "enqueue the job that waits on them".

1. An empty batch reports complete.

If batch_id matches no rows, wait_for_batch returns BatchCompletionStatus(total=0, pending=0, is_complete=True) and logs a WARNING. For the finalize pattern that is indistinguishable from success: the finalizer proceeds to aggregate, report, or advance the pipeline as though every child had succeeded, when in fact no child ever existed. The docstring is upfront that this "may indicate a wrong batch_id", but a WARNING in a worker log isn't a control-flow signal — by the time anyone reads it the finalizer has already committed whatever it does next.

2. Correctness depends on statement ordering that the API can't enforce.

Because enqueue_batch caps at 1000 items, anything larger is several calls sharing an explicit batch_id:

for group in batched(items, 500):
    await ctx.jobs.enqueue_batch(group, batch_id=b)
await ctx.jobs.enqueue(finalizer, FinalizerPayload(batch_id=b))  # must be last

The batch is partially inserted for the duration of that loop, so "the finalizer enqueue is the last statement" is load-bearing. Move it up for readability, enqueue it from a sibling task, or have the fan-out actor crash mid-loop and retry, and hazard 1 fires — the finalizer observes a partial batch, or an empty one, and calls it complete. I've ended up encoding that ordering as a comment plus a regression test, which is the kind of invariant I'd rather the library hold for me, since nothing in the signatures hints at it.

What I'd rather write — the finalizer inserted with the batch, in the same transaction:

await ctx.jobs.enqueue_batch(
    items,
    batch_id=b,
    finalizer=EnqueueItem(actor_ref=my_finalizer, payload=FinalizerPayload(batch_id=b)),
)

and/or an explicit expectation on the wait side, so an under-populated batch is an error rather than a success:

status = await wait_for_batch(db, b, expect_at_least=n_items)   # or on_empty="error"

A third option that would remove hazard 2 by itself: let enqueue_batch accept an iterable larger than the cap and chunk internally within one transaction under one batch_id. Callers already want the "one logical batch" semantics; the cap currently forces them to hand-roll the chunking and inherit the partially-inserted window.

Open questions:

• Is empty-is-complete deliberate for the genuinely-zero-items case (a fan-out that legitimately found nothing to do)? If so, the two cases need to be distinguishable — "nothing to do" and "nothing enqueued yet" want opposite outcomes, and right now both are is_complete=True.
• Is an expected-count better recorded on the batch at creation than passed to every wait_for_batch call? Recording it once seems harder to get wrong, but it needs somewhere to live.
• Is a transactional finalizer feasible given enqueue_batch already runs as one INSERT — is adding one more row in the same transaction as cheap as it looks from outside?

Happy to be told the ordering discipline is simply the intended contract, in which case a note in the wait_for_batch docstring saying so explicitly would have saved me working it out.