Skip to content

feat(power): ingest and expose power audit provenance / 功率:摄取并公开功耗审计溯源字段(power_invalid_reasons、power_audit) - #939

Open
edwingao28 wants to merge 5 commits into
masterfrom
feat/power-provenance-ingest
Open

feat(power): ingest and expose power audit provenance / 功率:摄取并公开功耗审计溯源字段(power_invalid_reasons、power_audit)#939
edwingao28 wants to merge 5 commits into
masterfrom
feat/power-provenance-ingest

Conversation

@edwingao28

@edwingao28 edwingao28 commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Important

STACKED PR — do not merge out of order. Base is feat/api-power-contract (#938), which is itself stacked on #937. Merge order: #937#938 → this PR. This PR contains only the deltas relative to #938.

Ingests and exposes the power audit provenance fields (power_invalid_reasons, power_audit) that the producer (InferenceX aggregate_power.py, via the row-level power provenance change on feat/power-row-provenance in InferenceX, PR queued) emits alongside the power_valid verdict. #938 reserved the two fields in the OpenAPI schema; this PR turns them live end-to-end while remaining fully tolerant of their absence (legacy artifacts, runs predating the producer change, legacy DB rows).

What changed

Storage — migration 014_power_provenance.sql

  • Two dedicated jsonb columns on benchmark_results (power_invalid_reasons, power_audit), mirroring the migration-006 workers precedent: metrics is a flat Record<string, number>, so structured data gets its own columns.
  • Recreates latest_benchmarks with the migration-012 definition verbatim (recursive append-only-curve body plus both indexes) solely so br.* picks up the new columns — not the obsolete 006 definition.

ETL — benchmark-mapper.ts

  • Both keys join NON_METRIC_KEYS: Number(['5']) === 5, so without the guard a malformed single-element reasons array would be auto-captured as a bogus numeric metric.
  • extractPowerInvalidReasons: keeps snake_case codes (≤ 64 chars), dedupes preserving order, caps at 32; empty result → undefined so the column stores SQL NULL, never [].
  • extractPowerAudit: fixed 8-key shape (window_start_unix, window_end_unix, expected_gpu_count, observed_gpu_count, sample_count, max_sample_gap_s, producer_sha, exporter_image_sha256); finite-number/safe-integer narrowing, malformed numerics omitted (partial audit beats none), shas collapse to null per the contract's string|null, unknown keys dropped, empty husk → undefined.
  • Extraction is unconditional on power_valid — tolerance in both directions.
  • No changes needed in ingest-ci-run.ts or run-overrides.ts: the persistence input is built by spreading ({...row, configId} / applyBenchmarkPointBackfill's {...point}), so the new BenchmarkParams fields flow through.

Ingest — benchmark-ingest.ts

  • Two jsonb unnest lanes mirroring workers; NULL when absent; excluded.* refresh on conflict (fresh artifact is authoritative, same as workers).

Reads — queries/benchmarks.ts (deploy-order safety, the #405/#407 lesson)

  • All four read paths select the columns as to_jsonb(br) -> 'power_invalid_reasons' / to_jsonb(lb) -> ... — never bare column references. Migrations run in the ingest workflows, not at Vercel deploy; a bare reference fails at query plan time until the next ingest applies migration 014, which is exactly how PR feat(power): measured-power multinode support (workers[] + per-stage joules) #405 produced a ~63% error rate. The jsonb key lookup degrades to NULL while the column is missing and is byte-identical once it exists, so merge order and deploy timing are irrelevant. A regression test pins the tolerant form with a negative regex.

API & frontend

  • /api/v1/benchmarks and /history return the fields verbatim when stored; calculator view strips them (payload-trimmed projection excludes measured-power data by design); unofficial-run overlay rows carry both fields via the shared mapper.
  • Chart tooltips (official + unofficial overlay) render a muted bilingual "Measured power withheld / 实测功耗未采信" line only when the point carries reasons; codes are re-validated against the snake_case regex before HTML interpolation (tooltips are raw HTML strings).
  • OpenAPI schema wording moves from "Reserved (forthcoming)" to live; api-route-catalog.ts digests refreshed for the two changed contract sources.
  • power_audit is API-only — no UI surface (follow-up).

Testing

  • bun run --cwd packages/db test:unit — 650 passed (extractor suites, mapper integration incl. the ['5'] guard, ingest lane recording-mock, query-shape assertions with the bare-reference regex guard, supplemental-path provenance sequence)
  • bun run --cwd packages/app test:unit — 4491 passed (transform narrowing, tooltip en/zh + injection filtering + overlay parity, unofficial-run overlay parity, calculator strip, OpenAPI docs invariants, catalog guard)
  • bun run typecheck / bun run lint / bun run fmt — clean
  • Note: packages/mcp server.test.ts fails identically on the base branch in this environment (z.enum undefined — zod resolution, unrelated to this change).

Deploy-order proof: this PR's Vercel preview serves /api/v1/benchmarks against the un-migrated production DB with the two keys null — the live demonstration of the tolerant-read design. The next stage-results/ingest run applies migration 014 and new ingests populate the columns.

Review notes

Two findings from review, both addressed:

  1. Supplemental ingest lane dropped provenanceingest-supplemental.ts participates in the power publication contract (the normalize + scrub from fix(etl): strip measured power metrics at ingest when power_valid=0 | ETL:power_valid=0 时在摄取阶段剥离实测功耗指标 #937) but built its persistence input without powerInvalidReasons/powerAudit, so a supplemental entry carrying the fields would persist NULL columns silently. Fixed in the follow-up commit: the lane now extracts both via the shared narrowers (entry-level fields sibling to metrics, with a metrics-nested fallback since power_valid rides in metrics in that format) and deletes the keys from metrics so the persisted jsonb stays a flat numeric record; the call sequence is pinned next to the fix(etl): strip measured power metrics at ingest when power_valid=0 | ETL:power_valid=0 时在摄取阶段剥离实测功耗指标 #937 supplemental tests.
  2. Vercel-preview live proof not independently probed — the preview deployment is auth-protected (302), so /api/v1/benchmarks could not be hit directly during review. Indirect evidence is strong (green Vercel check, to_jsonb reads pinned by a negative-regex test, Postgres jsonb semantics), but someone with Vercel access should hit the preview once and confirm 200 with power_invalid_reasons/power_audit null before merging the stack.

中文说明

堆叠 PR——请勿乱序合并。 基础分支为 feat/api-power-contract#938),后者又堆叠在 #937 之上。合并顺序:#937#938 → 本 PR

摄取并公开生产端(InferenceX aggregate_power.py,对应 InferenceX 的行级功耗溯源变更 feat/power-row-provenance,PR 排队中)随 power_valid 判定一同产出的功耗审计溯源字段(power_invalid_reasonspower_audit)。#938 已在 OpenAPI 中预留这两个字段;本 PR 将其全链路转正,并对字段缺失(旧产物、早于该生产端变更的运行、历史数据行)保持完全兼容。

  • 存储:迁移 014 在 benchmark_results 上新增两个专用 jsonb 列(沿用迁移 006 workers 的先例),并按迁移 012 的定义原样重建 latest_benchmarks,使 br.* 覆盖新列。
  • ETLmapBenchmarkRow 防御性收窄两个字段(snake_case 原因码去重限长限量;审计对象固定 8 键、剔除非法数值、sha 归一化为 string|null、丢弃未知键;空结果映射为 undefined 以存储 SQL NULL);两个键加入 NON_METRIC_KEYS,避免 Number(['5']) === 5 生成伪数值指标。
  • 摄取:批量插入新增两条 jsonb 通道,缺失即 NULL,冲突时以新产物为准刷新。
  • 读取(部署顺序安全,feat(power): measured-power multinode support (workers[] + per-stage joules) #405/fix(db): tolerate missing workers column in benchmark read queries / 基准测试读取查询兼容缺失的 workers 列 #407 教训):四条读取路径一律使用 to_jsonb(...) -> 'col' 容错形式,列尚未迁移时读取为 NULL,迁移后逐字节等价,合并与部署顺序均无关;回归测试用反向正则钉死该形式。
  • API 与前端:公开 API 原样返回字段;计算器视图剥离;非官方叠加行同样携带;图表提示框仅在存在原因码时渲染双语"实测功耗未采信"行,且在 HTML 插值前再次校验原因码;OpenAPI 文案由预留改为正式,契约摘要已刷新。

测试:db 包 650 项、app 包 4491 项全部通过;typecheck / lint / fmt 干净。(packages/mcpserver.test.ts 在本地环境于基础分支上即失败,与本变更无关。)

评审说明:评审中发现两点,均已处理——补充数据摄取通道(ingest-supplemental.ts)此前未持久化溯源字段,已在后续提交中修复并以单测钉住调用序列;Vercel 预览部署受访问保护(302),评审期间无法直接探测 /api/v1/benchmarks,请有权限的同事在合并前访问预览确认返回 200 且两个新键为 null。


Note

Medium Risk
Touches benchmark ingest, DB migration/matview recreation, and deploy-order-sensitive read paths; behavior is heavily tested and reads are migration-tolerant, but wrong ingest ordering could still affect latest_benchmarks until refresh.

Overview
Adds end-to-end support for producer-emitted power provenance power_invalid_reasons and power_audit, alongside the existing power_valid verdict.

Storage & ingest: Migration 014 adds dedicated JSONB columns on benchmark_results (keeping metrics numeric-only) and recreates latest_benchmarks. mapBenchmarkRow extracts and narrows both fields (snake_case reason codes with caps/dedup; fixed-shape audit object), excludes them from auto-metric capture, and bulkIngest persists them with upsert refresh. Supplemental ingest now extracts the same fields from entry- or metrics-nested payloads.

Reads & API: Benchmark queries select provenance via to_jsonb(...) -> '...' so deploys stay safe before migration 014 runs on ingest. Public benchmark rows return the fields; the calculator trimmed view drops them. OpenAPI docs move from reserved to live semantics.

UI: Chart tooltips (official, overlay, GPU comparison) show a bilingual “Measured power withheld” line when power_invalid_reasons is present, with HTML-safe re-validation of codes. power_audit is API-only (no UI yet). Unofficial-run overlays pass provenance through mapBenchmarkRow like the DB path.

Reviewed by Cursor Bugbot for commit ae58fc8. Bugbot is set up for automated code reviews on this repo. Configure here.

@edwingao28
edwingao28 requested a review from adibarra as a code owner August 28, 2026 02:56
@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
inferencemax-app Ready Ready Preview Sep 4, 2026 11:11pm UTC

Request Review

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 3m 25s —— View job


Review: PLAN-07 power provenance ingest

  • Gather context (diff vs feat/api-power-contract)
  • Review ETL mapper/ingest (extractors, NON_METRIC_KEYS, jsonb lanes)
  • Review read SQL (to_jsonb -> key tolerance)
  • Review API/frontend (view strip, tooltip HTML, overlay parity)
  • Review migration 014 (matview recreate)
  • Check CI status + runtime load
  • Post findings

🟢 LGTM — no blocking issues found

This is a carefully built, well-tested change. Verified the risk areas rather than just reading the diff:

Migration 014 (highest risk). Confirmed the recreated latest_benchmarks body is byte-for-byte identical to the canonical migration-012 definition (both indexes, the recursive append-only-curve body, nulls not distinct PK). Migration 013 does not touch the matview or benchmark_results, so 012 is correctly the source of truth. create materialized view … as (no WITH NO DATA) repopulates on recreate. ✅

Deploy-order tolerance. Both read paths use to_jsonb(br) -> 'power_*' / to_jsonb(lb) -> …. Traced the recursive path: the inner selected_points uses br.* (picks up the columns only post-migration), and the outer to_jsonb(selected_points_row) -> key degrades to NULL before migration and is byte-identical after. This correctly avoids the plan-time failure mode from #405/#407. The negative-regex test pins the tolerant form. ✅

XSS / raw-HTML tooltip. powerWithheldHTML re-validates each code against ^[a-z][a-z0-9_]*$ and a ≤64 length cap before interpolation into the raw-HTML tooltip string — reason codes cannot inject markup even though they're producer-supplied. Mirrors the mapper's extractor regex. ✅

Extractor / metrics-integrity edges. power_invalid_reasons + power_audit are in NON_METRIC_KEYS (guards Number(['5']) === 5), empty results collapse to undefined → SQL NULL (never []/{}), extractPowerAudit narrows field-by-field and collapses shas to null per the string|null contract. Supplemental lane now extracts both (entry-level + metrics-nested fallback) and deletes them from metrics so the flat numeric record stays clean. Calculator view strips both. Overlay parity wired through the shared mapper. ✅

Runtime. Dev server up (200); /inference renders with 0 console errors. The new tooltip line only appears for points carrying reason codes (needs migration 014 applied + populated data, absent in the current read-only DB), so no live rendering of that line is possible yet — consistent with the PR's tolerant-read demonstration.

Chinese copy. The text(en, zh) overview description and the 实测功耗未采信 tooltip string were updated in sync with their English sources; both are accurate, natural technical Chinese with English identifiers preserved. No maintainer escalation needed.

Tests. Coverage spans every layer (extractor suites, ['5'] guard, ingest lane mock, query-shape negative-regex guard, supplemental sequence, transform narrowing, tooltip en/zh + injection + overlay parity, calculator strip, OpenAPI/catalog). ✅

Non-blocking observation (not a finding): to_jsonb(br) serializes the full row twice per returned row on the hot read paths — a deliberate, documented tradeoff for deploy safety; latest-curve result sets are small enough that the cost is negligible, so no action needed.

One reminder carried from the PR body: this is a stacked PR (#937#938 → this) and a human with Vercel access should hit the preview /api/v1/benchmarks once to confirm 200 with the two keys null before merging the stack.

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 2m 15s —— View job


Re-review: power provenance ingest — 🟢 LGTM

No new commits since the previous LGTM review (same 4 commits: 84b3b777, 5eabf9cf, c9131130, a31c60cf). I re-verified the risk areas rather than re-reading the whole diff, and confirm the change is correct and merge-ready.

  • Diff vs feat/api-power-contract
  • ETL mapper/ingest + supplemental lane
  • Read SQL tolerance
  • API/frontend (view strip, tooltip HTML, overlay)
  • Migration 014
  • CI status + runtime load

Migration 014 (highest risk). Byte-for-byte diff of the recreated latest_benchmarks body against the canonical migration-012 definition (both indexes, recursive append-only-curve body, nulls not distinct PK) — identical. Migration 013 does not touch the matview or benchmark_results, so 012 is correctly the source of truth. create materialized view … as (no WITH NO DATA) repopulates on recreate. ✅

Deploy-order tolerance. All four read paths use to_jsonb(br) -> 'power_*' / to_jsonb(lb) -> …; the inner recursive br.* only picks up the columns post-migration, and the outer jsonb lookup degrades to NULL before migration / is byte-identical after — avoiding the plan-time failure mode from #405/#407. Pinned by the negative-regex test. ✅

XSS / raw-HTML tooltip. powerWithheldHTML re-validates each code against ^[a-z][a-z0-9_]*$ + ≤64 length cap before interpolating into the raw-HTML tooltip string — producer-supplied codes cannot inject markup. Mirrors the mapper's extractor regex. ✅

Extractor / metrics integrity. power_invalid_reasons + power_audit are in NON_METRIC_KEYS (blocks Number(['5'])); empty results collapse to undefined → SQL NULL (never []/{}); extractPowerAudit narrows field-by-field, collapses shas to null per the string|null contract. Supplemental lane extracts both (entry-level + metrics-nested fallback) and deletes them from metrics so the persisted jsonb stays a flat numeric record. Calculator view strips both. Overlay parity flows through the shared mapper; power_invalid_reasons reaches InferenceData via ...entry in createChartDataPoint, so official + overlay tooltips both render the withheld line. ✅

Runtime. Dev server 200; /inference renders with 0 console errors. The withheld-power tooltip line only appears for points carrying reason codes (needs migration 014 applied + populated data, absent in the read-only DB), consistent with the tolerant-read design.

Chinese copy. 实测功耗未采信 for "Measured power withheld" is accurate, natural, and register-appropriate for a muted tooltip line; English identifiers preserved. No maintainer escalation needed.

Tests. Coverage spans every layer (extractor suites, ['5'] guard, ingest lane mock, query-shape negative-regex guard, supplemental sequence, transform narrowing, tooltip en/zh + injection + overlay parity, calculator strip, OpenAPI/catalog). ✅

Carried reminder from the PR body: this is a stacked PR (#937#938 → this) — merge in order, and a human with Vercel access should hit the preview /api/v1/benchmarks once to confirm 200 with the two keys null before merging the stack.
· feat/power-provenance-ingest

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 1m 12s —— View job


Re-review: power provenance ingest — 🟢 LGTM

Four new commits since the previous LGTM (701fbbac, f8fb515e, plus base merges 07b9d25d, b4b956db). I diffed a31c60cf..HEAD and confirmed the delta is comment/doc rewording only — no functional or test-logic change.

  • Identify new commits since previous review (a31c60cf)
  • Review the delta
  • Check CI status

What actually changed:

  • metric-keys.ts — the MEASURED_POWER_METRIC_KEY_LIST header comment was condensed and one inline comment above ...POWER_METRIC_KEYS dropped. The list membership (13 keys), METRIC_KEYS derivation, and the contract/diagnostic exclusions are byte-identical.
  • benchmark-mapper.ts — the scrubWithheldPowerMetrics / scrub-ordering doc comments were rewritten more tersely; the power_valid !== 0 fail-closed logic, extractors, and NON_METRIC_KEYS guards are unchanged.
  • benchmark-mapper.test.ts, benchmark-transform.test.ts, metric-keys.test.ts, benchmark-power-validity.test.ts, api-documentation.power.test.ts — only explanatory comments removed and one it() title reworded (tolerates the invalid-verdict companion fields…keeps structured invalid-verdict companions outside the numeric metrics record). All assertions intact.

Everything I verified in the two prior reviews (migration 014 byte-parity with 012, to_jsonb -> key deploy-order tolerance, the powerWithheldHTML XSS re-validation, NON_METRIC_KEYS/Number(['5']) guard, supplemental-lane extraction, calculator strip, overlay parity, 实测功耗未采信 copy) stands unchanged. No new surface to test.

CI. Vercel deploy ✅. The dedicated tests-*/lint workflows aren't attached to this run in the checks list; nothing in this delta could affect their outcome since only comment text changed.

Carried reminder: this is a stacked PR (#937#938 → this) — merge in order, and a human with Vercel access should hit the preview /api/v1/benchmarks once to confirm 200 with power_invalid_reasons/power_audit null before merging the stack.
· feat/power-provenance-ingest

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b4b956d. Configure here.

Comment thread packages/app/src/components/inference/utils/tooltipUtils.ts
@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 55s —— View job


Re-review: power provenance ingest — 🟢 LGTM

One new commit since the previous LGTM: 42f4768f fix: show withheld power in GPU tooltips (plus base merge b4b956db). I focused on the delta and the open Cursor Bugbot finding.

  • Diff new commits vs feat/api-power-contract
  • Investigate Bugbot finding: GPU graph tooltip omits withheld line
  • Check CI status

Bugbot finding — already fixed. Bugbot flagged (against the older commit b4b956db) that generateGPUGraphTooltipContent never called powerWithheldHTML, so the date-comparison GPU graph would omit the withheld-power line even when the point carries power_invalid_reasons. The very next commit 42f4768f closes exactly this gap:

  • tooltipUtils.ts:581 now renders ${powerWithheldHTML(d, locale)} inside generateGPUGraphTooltipContent, so all three tooltip generators (official scatter, unofficial overlay, GPU comparison graph) render the line from the same InferenceData. ✅
  • A regression test (tooltip-utils.test.ts"gives GPU comparison tooltips the same line") pins it, asserting both the English label and a reason code appear. ✅

The re-validation guard is intact: powerWithheldHTML still filters each producer-supplied code through the ^[a-z][a-z0-9_]*$ + ≤64-char cap before interpolating into the raw-HTML string, so the new call site inherits the same XSS protection. No new untested surface introduced.

Everything else unchanged. Migration 014 byte-parity with migration 012, to_jsonb(...) -> 'power_*' deploy-order tolerance (#405/#407 lesson), NON_METRIC_KEYS/Number(['5']) guard, supplemental-lane extraction + metrics strip, calculator view strip, and the 实测功耗未采信 copy all stand from the prior reviews — the delta is one line + one test.

CI. Only the Claude workflows are attached to this synchronize event so far; the dedicated tests-*/lint workflows aren't in the checks list yet, but a one-line tooltip call + test can't affect their outcome. No 🔴 findings.

Carried reminder: this is a stacked PR (#937#938 → this) — merge in order, and a human with Vercel access should hit the preview /api/v1/benchmarks once to confirm 200 with power_invalid_reasons/power_audit null before merging the stack.
· feat/power-provenance-ingest

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 1m 23s —— View job


Re-review: power provenance ingest — 🟢 LGTM

The branch was rebased since the last review into 5 clean commits (1fdbb756, fe3272c3, fd0513c7, 432ef10f, bab2df9e) — new hashes, but I diffed the full net delta against feat/api-power-contract and confirmed it is the same content previously approved, with the Bugbot GPU-tooltip finding folded in.

  • Identify new commits since last review (rebase)
  • Review the delta
  • Check CI status
  • Post findings

Migration 014 (highest risk). Re-ran a byte-for-byte comparison of the recreated latest_benchmarks body against the canonical migration-012 definition (recursive append-only-curve body, both indexes, nulls not distinct PK) — identical (3350 chars, exact match). Migration 013 touches neither the matview nor benchmark_results, so 012 is correctly the source of truth. create materialized view … as (no WITH NO DATA) repopulates on recreate. ✅

Deploy-order tolerance. Both read paths (executeRecursiveBenchmarkQuery inner br.* + outer to_jsonb(br) -> 'power_*', and getLatestBenchmarks to_jsonb(lb) -> …) degrade to NULL before migration 014 and are byte-identical after — avoiding the plan-time failure mode from #405/#407. Pinned by the negative-regex query-shape test. ✅

GPU tooltip (Bugbot finding). generateGPUGraphTooltipContent now renders ${powerWithheldHTML(d, locale)}, so all three generators (official scatter, unofficial overlay, GPU comparison) emit the withheld line from the same InferenceData. The ^[a-z][a-z0-9_]*$ + ≤64-char re-validation runs at every call site, so the new one inherits the same XSS protection. Regression test present. ✅

Extractor / metrics integrity & supplemental lane. power_invalid_reasons + power_audit are in NON_METRIC_KEYS (blocks Number(['5'])); empty results collapse to undefined → SQL NULL; ingest lanes NULL-when-absent and excluded.*-refresh on conflict; supplemental lane extracts both (entry-level + metrics-nested fallback) and deletes them from metrics. Calculator view strips both. Overlay parity via the shared mapper. ✅

CI. Only the in-progress Claude run is attached to this synchronize event so far; the dedicated tests-*/lint workflows aren't in the checks list yet. The delta is a rebase of already-green content plus the GPU-tooltip line + test, so their outcome is unaffected. No 🔴 findings.

Runtime. The withheld-power tooltip line only appears for points carrying reason codes, which requires migration 014 applied + populated data (absent in the read-only DB) — consistent with the tolerant-read design, so live rendering of that specific line isn't reproducible yet, as noted in prior reviews.

Carried reminder from the PR body: this is a stacked PR (#937#938 → this) — merge in order, and a human with Vercel access should hit the preview /api/v1/benchmarks once to confirm 200 with power_invalid_reasons/power_audit null before merging the stack.
· feat/power-provenance-ingest

Base automatically changed from feat/api-power-contract to master September 4, 2026 23:06
edwingao28 and others added 5 commits September 4, 2026 23:07
Turn the reserved power_invalid_reasons / power_audit contract fields live
end-to-end:

- migration 014 adds dedicated jsonb columns on benchmark_results and
  recreates latest_benchmarks with the migration-012 definition verbatim so
  br.* picks up the new columns
- mapBenchmarkRow narrows both fields defensively (snake_case reason codes,
  fixed 8-key audit shape, empty -> undefined so the columns store SQL NULL)
  and both keys join NON_METRIC_KEYS so Number(['5']) === 5 can never mint a
  bogus numeric metric
- bulkIngestBenchmarkRows persists both as jsonb lanes, NULL when absent,
  refreshed on conflict like workers
- all four read paths select the columns via to_jsonb(...) -> 'col' (the
  PR #405/#407 deploy-order lesson: bare references fail at plan time until
  the next ingest applies the migration; the jsonb lookup degrades to NULL)
- rowToAggDataEntry passes reasons through, and both chart tooltips render a
  bilingual "Measured power withheld" line with re-sanitized codes
- unofficial-run overlay rows carry both fields; the calculator view strips
  them; OpenAPI wording moves from reserved to live

The persistence input is built by spreading (ingest-ci-run.ts {...row,
configId}; run-overrides.ts applyBenchmarkPointBackfill {...point}), so the
new BenchmarkParams fields flow through with no changes there.

中文:将预留的 power_invalid_reasons / power_audit 契约字段全链路转正:迁移 014
为 benchmark_results 增加两个专用 jsonb 列并按迁移 012 的定义原样重建
latest_benchmarks;mapBenchmarkRow 对两个字段做防御性收窄(snake_case 原因码、
固定 8 键审计对象、空值映射为 undefined 以存储 SQL NULL);批量摄取以 jsonb 通道
持久化并在冲突时刷新;四条读取路径均用 to_jsonb(...) -> 'col' 容错读取(PR
携带同样字段;计算器视图剥离;OpenAPI 文案由预留改为正式。
…ies, and UI | 测试:覆盖功耗审计溯源在映射、摄取、查询与界面各层的行为

- extractPowerInvalidReasons: snake_case validation, dedupe, 32-code cap,
  64-char boundary, empty/non-array/all-invalid -> undefined (never [])
- extractPowerAudit: 8-field round-trip, Infinity/NaN/junk numerics omitted,
  negative and non-safe-integer counts rejected, sha trimming and null
  collapse, unknown keys dropped, empty husk -> undefined
- mapBenchmarkRow lands the fields on BenchmarkParams for v1/v2/agentic rows,
  stores audits from valid rows too, and never mints a numeric metric from a
  malformed ['5'] reasons array (Number(['5']) === 5)
- bulkIngestBenchmarkRows recording-mock: both columns in the INSERT list,
  two extra ::jsonb[] lanes, null lanes for absent fields, excluded.* refresh
  on conflict
- all four read paths pin the tolerant to_jsonb(...) -> 'col' form and a
  negative regex guards against bare br./lb. references (the #407 lesson)
- rowToAggDataEntry narrows null/[]/absent to undefined; tooltips render the
  bilingual withheld line only for sanitized codes (en + zh, overlay parity);
  calculator view strips both fields alongside workers

中文:为溯源字段新增全链路测试——提取函数的收窄规则(原因码校验、去重、上限、
空值处理;审计对象 8 字段往返、异常数值剔除、sha 归一化)、映射到
BenchmarkParams、批量摄取的 jsonb 通道与冲突刷新、四条读取路径的 to_jsonb
容错形式(并用反向正则钉死 #407 教训)、前端窄化与双语提示框行为、计算器
视图剥离。
…| ETL:补充数据摄取通道同样持久化功耗溯源字段

Review found ingest-supplemental.ts participates in the power
publication contract (normalize + scrub, PLAN-03) but silently dropped
power_invalid_reasons / power_audit: the persistence input carried no
provenance fields, so a supplemental entry with them would persist NULL
columns. Extract both via the shared narrowers — entry-level fields
sibling to metrics (mirroring artifact rows), with a metrics-nested
fallback since power_valid rides in metrics in this format — and delete
the keys from metrics so the persisted jsonb stays a flat numeric
record. Pin the call sequence next to the PLAN-03 supplemental tests.
中文:清理功耗溯源实现中的内部计划标签、重复测试说明和装饰性注释,同时保留迁移顺序、部署容错、数据收窄、安全与持久化约束。
中文:在 GPU 对比图提示中显示实测功耗未采信原因
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 1m 21s —— View job


Re-review: power provenance ingest — 🟢 LGTM

The branch was rebased onto origin/master (the #937/#938 stack has landed), so the base is now master directly. Five fresh commit hashes (14aa9cc2ae58fc88), but I diffed the full net delta against origin/master and confirmed it is byte-for-byte the same 21-file / +868/-20 content approved in the prior reviews, with the Bugbot GPU-tooltip fix already folded in.

  • Identify new commits since last review (rebase onto master)
  • Review the delta
  • Check CI status
  • Post findings

Migration 014 (highest risk). Re-verified: benchmark_results gets the two JSONB columns, then drop materialized view if exists latest_benchmarks → recreate. Diffed the recreated latest_benchmarks body including both indexes against the canonical migration-012 definition — fully identical. Drop-before-recreate is correctly ordered (the br.* projection otherwise freezes the column list), and create materialized view … as (no WITH NO DATA) repopulates on recreate. ✅

Deploy-order tolerance. Both read paths (executeRecursiveBenchmarkQuery inner br.* + outer to_jsonb(br) -> 'power_*', and getLatestBenchmarks to_jsonb(lb) -> …) degrade to NULL before migration 014 and are byte-identical after — avoiding the plan-time failure mode from #405/#407. Pinned by the negative-regex query-shape test. ✅

GPU tooltip (Bugbot finding). generateGPUGraphTooltipContent now renders ${powerWithheldHTML(d, locale)}, so all three generators (official scatter, unofficial overlay, GPU comparison) emit the withheld line from the same InferenceData, each inheriting the ^[a-z][a-z0-9_]*$ + ≤64-char re-validation before raw-HTML interpolation. Regression test present. ✅

ETL / ingest / supplemental lane. Ingest adds two homogeneous jsonb unnest lanes (JSON null → SQL NULL when absent) and excluded.*-refresh on conflict, mirroring workers. The supplemental lane extracts both fields (entry-level with a metrics-nested fallback) and deletes them from metrics so the persisted record stays flat/numeric. power_invalid_reasons + power_audit are in NON_METRIC_KEYS, empties collapse to undefined → NULL, and the calculator view strips both. ✅

CI. Lint & Format ✅, Tests (Unit) ✅, Code Quality ✅, PR checks ✅. Tests (E2E) still in progress; the delta is a rebase of already-green content plus the one GPU-tooltip line + test, so the outcome is unaffected. No 🔴 findings.

Runtime. The withheld-power tooltip line only renders for points carrying reason codes, which requires migration 014 applied + populated data (absent in the read-only DB) — so that specific line isn't reproducible live yet, consistent with the tolerant-read design noted across prior reviews.

Carried reminder: a human with Vercel access should hit the preview /api/v1/benchmarks once to confirm 200 with power_invalid_reasons/power_audit null before merging.
· feat/power-provenance-ingest

@blacksmith-sh

blacksmith-sh Bot commented Sep 4, 2026

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
Profit Estimator — MiniMax M3/
re-seeds the operating point and price source when switching from GLM
View Logs
Profit Estimator — MiniMax M3/
re-seeds the operating point and price source when switching from GLM
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant