Skip to content

Cached, versioned extractor seam — prefactor for model-based extraction #207

Description

@alexwolson

This was generated by AI during triage.

Parent

#205

What to build

rebuild_agency_bids re-derives every agency_bid row from the held PDFs on each store pass — correct when derivation is regex and costs milliseconds, ruinous when it is a model call per document. This ticket adds a cache keyed on the document's sha256 plus a model+prompt version stamp, so a document that has not changed and whose extraction prompt has not changed is never re-extracted.

The existing regex derivation moves behind this seam unchanged. 800 tests green, zero behaviour change. A second extractor (the model path) becomes a drop-in behind it.

"Skip if rows exist" is the wrong guard — that is what makes stale rows persist. The version stamp is what makes re-extraction trigger on a prompt change, not on whether the table is empty.

Acceptance criteria

  • A cache layer exists, keyed on (sha256, extractor_version), that short-circuits extraction when the key matches
  • Existing regex-based derivation produces identical agency_bid rows through the new seam
  • A changed extractor_version causes re-extraction on the next store pass
  • 800 existing tests pass unchanged

Blocked by

None — can start immediately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions