What problem does this solve?
Routerly today handles every request in a single monolithic flow: authenticate → route → forward → respond. As the feature set grows, more and more behaviour needs to intercept that flow — transforming the request before it reaches the provider, observing or modifying the response after it comes back, or short-circuiting the flow entirely (e.g. a cache hit). Without a principled architecture for this, each new feature adds another branch to an already complex handler, making the code harder to reason about and the behaviour harder for users to configure.
Proposed solution
Introduce a first-class request pipeline concept: an ordered chain of processors that can hook into defined points in the request lifecycle — before routing, after model selection, before forwarding, and after the response arrives. Each processor is a discrete, independently configurable unit of behaviour.
The pipeline would be the foundation that a large class of existing and planned features are built on, rather than each being a one-off integration into the core handler.
Alternatives you've considered
Continuing to add features as ad-hoc conditionals in the request handler. This works short-term but does not scale — the handler becomes a kitchen-sink of unrelated concerns, and there is no consistent model for how users enable, order, or configure these behaviours per project.
Who would benefit from this?
Everyone — indirectly. This is an internal architecture change, but it is what makes the rest of the feature roadmap composable and maintainable rather than fragile.
Additional context
Many features already planned or filed as issues are natural pipeline processors: prompt injection (#102), content guardrails (#77), PII scrubbing (#76), semantic caching (#75), session tracking (#94), cost attribution (#95), metrics emission (#93), provider-native prompt caching (#97), and others. Designing the pipeline abstraction first ensures these features share a coherent model rather than each inventing its own integration point.
What problem does this solve?
Routerly today handles every request in a single monolithic flow: authenticate → route → forward → respond. As the feature set grows, more and more behaviour needs to intercept that flow — transforming the request before it reaches the provider, observing or modifying the response after it comes back, or short-circuiting the flow entirely (e.g. a cache hit). Without a principled architecture for this, each new feature adds another branch to an already complex handler, making the code harder to reason about and the behaviour harder for users to configure.
Proposed solution
Introduce a first-class request pipeline concept: an ordered chain of processors that can hook into defined points in the request lifecycle — before routing, after model selection, before forwarding, and after the response arrives. Each processor is a discrete, independently configurable unit of behaviour.
The pipeline would be the foundation that a large class of existing and planned features are built on, rather than each being a one-off integration into the core handler.
Alternatives you've considered
Continuing to add features as ad-hoc conditionals in the request handler. This works short-term but does not scale — the handler becomes a kitchen-sink of unrelated concerns, and there is no consistent model for how users enable, order, or configure these behaviours per project.
Who would benefit from this?
Everyone — indirectly. This is an internal architecture change, but it is what makes the rest of the feature roadmap composable and maintainable rather than fragile.
Additional context
Many features already planned or filed as issues are natural pipeline processors: prompt injection (#102), content guardrails (#77), PII scrubbing (#76), semantic caching (#75), session tracking (#94), cost attribution (#95), metrics emission (#93), provider-native prompt caching (#97), and others. Designing the pipeline abstraction first ensures these features share a coherent model rather than each inventing its own integration point.