Skip to content

Proposal: Dynamic request batching interface for cells #164

Description

@cleaton

Current Durable Objects use a single fetch(request) handler.
Every incoming request runs in its own turn, executes separate SQLite transactions, and waits for its own durability proof before responding.

For write-heavy workloads like event ingestion, counters, and pipeline stages, handling requests one-by-one creates a bottleneck around database transactions and replication syncs. celld can innovate beyond the basic Durable Object model by supporting an optional dynamic batching handler, such as batchFetch(requests).

Why dynamic batching

  1. Higher throughput under load. When requests queue up for an active cell, celld can pass an array of pending requests to the cell in a single turn. The cell processes the batch, writes to SQLite once, and returns an array of responses.
  2. Lower I/O and replication overhead. Instead of running 100 separate SQLite transactions and 100 replication syncs, a batch of 100 requests runs in one atomic transaction and proves durability once.
  3. Flexible latency and throughput trade-offs. Callers keep using standard single fetch(req). Celld aggregates requests dynamically only when concurrency exists, preserving low latency when idle while maximizing throughput under load.
  4. Better fit for pipelines. Developers using cells as stream aggregators or write-ahead buffers get native support for bulk operations without writing custom queueing mechanisms in userland.

If a cell does not define a batch handler, it continues using standard fetch() with zero changes.

edit: I've pushed a POC of this here: 4f3faa4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions