Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions tesseract_core/runtime/experimental/vjp_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,26 @@ def set_jax_vjp_cache_size(size: int) -> None:

When ``jax_apply`` and ``jax_vjp`` (from
:mod:`tesseract_core.runtime.jax_recipes`) are called in sequence on the
same inputs (the typical pattern under tesseract-jax's ``custom_vjp``,
including under ``jax.value_and_grad`` and ``jax.jacrev``), caching the
backward function produced by ``jax_apply``'s internal ``jax.vjp`` call
lets the subsequent ``jax_vjp`` skip the redundant forward pass. The win
scales with the cost of the forward pass and goes the wrong way on
trivially-small ones, so benchmark your own workload before enabling.
same inputs (as tesseract-jax does under ``jax.grad`` and
``jax.value_and_grad``), caching the backward function produced by
``jax_apply``'s internal ``jax.vjp`` call lets the subsequent ``jax_vjp``
skip the redundant forward pass. The win scales with the cost of the
forward pass and goes the wrong way on trivially-small ones, so benchmark
your own workload before enabling.

Best fit:
- tesseract-jax workflows that go through ``custom_vjp`` (every
gradient evaluation runs ``apply()`` first via ``fwd``, so the
cache reliably hits on the subsequent ``vjp()`` -- true even
under plain ``jax.grad``).
- tesseract-jax reverse mode: ``jax.grad`` and ``jax.value_and_grad``
call ``apply()`` and then ``vector_jacobian_product()`` on the same
inputs, so the second call hits the cache.
- Manual ``apply()`` followed by multiple ``vjp()`` calls on the
same inputs (e.g. iterative solvers, CG-style inverse problems).

Poor fit:
- Very small models where the forward pass is microseconds: the
cache machinery overhead exceeds the saved work.
- ``jax.jit`` where nothing the compiled function returns depends
on ``apply``'s outputs (e.g. ``jax.grad`` of a loss linear in
them): JAX drops the ``apply()`` call, so nothing fills the cache.

**Supported outputs.** With the cache enabled, ``apply`` may only nest its
outputs in dicts, lists, tuples and NamedTuples, because a cache hit
Expand Down
7 changes: 3 additions & 4 deletions tesseract_core/runtime/jax_recipes.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,10 +311,9 @@ def jax_vjp(
inputs_dict = as_jax_arrays(inputs.model_dump())
cotangent_vector = as_jax_arrays(cotangent_vector)

# Use get (not pop) so the cached residuals can serve multiple sequential
# vjp calls on the same inputs -- for example, when tesseract-jax's
# value_and_grad is followed by jax.jacrev, which decomposes into many
# vjp calls per output basis vector.
# Use get (not pop) so the cached residuals can serve several vjp calls
# on the same inputs, e.g. jax.jacrev through tesseract-jax with
# vmap_method="sequential" on a Tesseract without a jacobian endpoint.
if (
_jax_vjp_cache is not None
and (cached := _jax_vjp_cache.get((apply_jit, _cache_key(inputs_dict))))
Expand Down
Loading