chore(release): cut 0.15.0 "evals regression gate" (R7) - #206
Merged
Conversation
Close and archive R7 milestone `evals-regression-gate` (7/7 exit criteria) and cut the release. R7 is the first product-feature milestone since R6's substrate work: a tenant runs a named eval set against a candidate model and gets a scored pass/fail verdict against a pinned baseline, so a model swap is proven safe BEFORE it ships — without touching the hot proxy path, auto-promoting on a verdict, or scoring with an LLM judge. Shipped across the milestone (all merged, all gated PASS): - eval-set-store (#201) — /v1/evals/sets + cases; ZDR disposition frozen first - deterministic-scorers (#202) — exact · contains · regex · json-schema, re-run stable - eval-run-executor (#203) — governed set replay: billed traffic, per-tenant breaker - baseline-and-verdict (#204) — pin baseline, exact integer verdict, no_baseline state - evals-console (#205) — verdict-first, read-focused, reuses the /v1 verdict core Version bumped across all eight sites per the RELEASES.md checklist: pyproject + `__version__` fallback, Chart.yaml version (0.2.1->0.2.2) + appVersion, values.yaml gateway+dashboard tags, values-prod.yaml both `-prod` overrides. Dashboard package.json stays independently versioned (0.1.0) — a gateway release does not ship a phantom UI release. Guards green: tests/release_provenance (served /openapi.json == pyproject == fallback, RELEASES newest >= pyproject) and the production digest-pin / multi-arch image guards (14 passed). Waivers: 0.14.0's suite-stability and pgvector-deploy-runbook carry forward unchanged (both RISK-ACCEPTED, owner Tin Dang, expire 2026-09-30); ci-restoration stays DISCHARGED. No new waivers. REMAINING IS HUMAN-RUN (tagging PUBLISHES images via publish-images.yml): 1. git tag -a v0.15.0 on the merge commit, and push 2. verify the publish-images run is green (reads back the manifest for both images; do not deploy a tag whose publish job is not green) 3. staging first, following the pgvector runbook (dump/restore path) OPEN, carried to R8 `soc2-groundwork` and NOT waived: the `required_approving_review_count: 1` gate was satisfied on all five R7 PRs by an operator-directed self-approval via a second account (`pilotspacex-byte`), disclosed on each PR. That evidences a green required check on the merged artifact; it does NOT evidence independent four-eyes review. Every R7 merge still owes a genuine second human with write access before the SOC 2 audit. author: Tin Dang
pilotspacex-byte
approved these changes
Aug 14, 2026
pilotspacex-byte
left a comment
Contributor
There was a problem hiding this comment.
Operator-directed approval via the pilotspacex-byte account to satisfy the required-review gate on a solo-maintainer repo. DISCLOSED and NOT independent four-eyes review: this is the same operator who authored the change, using a second GitHub account. It evidences a green required check on the merged artifact, not a genuine second reviewer. This R7 release (and PRs #201-#205) still owes a real second human with write access before the SOC 2 audit — tracked in R8 soc2-groundwork.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
chore(release): cut 0.15.0 "evals regression gate" (R7)
Closes and archives R7 milestone
evals-regression-gate(7/7 exit criteria) and cuts the release.R7 is the first product-feature milestone since R6's substrate work: a tenant runs a named eval set against a candidate model and gets a scored pass/fail verdict against a pinned baseline, so a model swap is proven safe BEFORE it ships — without touching the hot proxy path, auto-promoting on a verdict, or scoring with an LLM judge.
Shipped across the milestone (all merged, all gated PASS)
/v1/evals/sets+ cases; ZDR disposition frozen first (atomic re-check, slow-double verified)no_baseline/v1verdict core (byte-identical)Release mechanics
__version__fallback,Chart.yamlversion (0.2.1→0.2.2) + appVersion,values.yamlgateway+dashboard tags,values-prod.yamlboth-prodoverrides.package.jsonstays independently versioned (0.1.0) — a gateway release does not ship a phantom UI release.tests/release_provenance(served/openapi.json== pyproject == fallback; RELEASES newest ≥ pyproject) + production digest-pin / multi-arch image guards — 14 passed.Waivers
Two carry forward from 0.14.0, both still in window: suite-stability + pgvector-deploy-runbook (RISK-ACCEPTED, owner Tin Dang, expire 2026-09-30). ci-restoration stays DISCHARGED. No new waivers.
⚠ Human-run after merge (tagging PUBLISHES images via
publish-images.yml)git tag -a v0.15.0on the merge commit, and push⚠ OPEN — carried to R8
soc2-groundwork, NOT waivedThe
required_approving_review_count: 1gate was satisfied on all five R7 PRs by an operator-directed self-approval via a second account (pilotspacex-byte), disclosed on each PR. That evidences a green required check on the merged artifact; it does not evidence independent four-eyes review. Every R7 merge still owes a genuine second human with write access before the SOC 2 audit.