feat(power): ingest and expose power audit provenance / 功率:摄取并公开功耗审计溯源字段(power_invalid_reasons、power_audit) - #939
feat(power): ingest and expose power audit provenance / 功率:摄取并公开功耗审计溯源字段(power_invalid_reasons、power_audit)#939edwingao28 wants to merge 5 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @edwingao28's task in 3m 25s —— View job Review: PLAN-07 power provenance ingest
🟢 LGTM — no blocking issues foundThis is a carefully built, well-tested change. Verified the risk areas rather than just reading the diff: Migration 014 (highest risk). Confirmed the recreated Deploy-order tolerance. Both read paths use XSS / raw-HTML tooltip. Extractor / metrics-integrity edges. Runtime. Dev server up (200); Chinese copy. The Tests. Coverage spans every layer (extractor suites, Non-blocking observation (not a finding): One reminder carried from the PR body: this is a stacked PR (#937 → #938 → this) and a human with Vercel access should hit the preview |
|
Claude finished @edwingao28's task in 2m 15s —— View job Re-review: power provenance ingest — 🟢 LGTMNo new commits since the previous LGTM review (same 4 commits:
Migration 014 (highest risk). Byte-for-byte diff of the recreated Deploy-order tolerance. All four read paths use XSS / raw-HTML tooltip. Extractor / metrics integrity. Runtime. Dev server 200; Chinese copy. Tests. Coverage spans every layer (extractor suites, Carried reminder from the PR body: this is a stacked PR (#937 → #938 → this) — merge in order, and a human with Vercel access should hit the preview |
|
Claude finished @edwingao28's task in 1m 12s —— View job Re-review: power provenance ingest — 🟢 LGTMFour new commits since the previous LGTM (
What actually changed:
Everything I verified in the two prior reviews (migration 014 byte-parity with 012, CI. Vercel deploy ✅. The dedicated Carried reminder: this is a stacked PR (#937 → #938 → this) — merge in order, and a human with Vercel access should hit the preview |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b4b956d. Configure here.
|
Claude finished @edwingao28's task in 55s —— View job Re-review: power provenance ingest — 🟢 LGTMOne new commit since the previous LGTM:
Bugbot finding — already fixed. Bugbot flagged (against the older commit
The re-validation guard is intact: Everything else unchanged. Migration 014 byte-parity with migration 012, CI. Only the Carried reminder: this is a stacked PR (#937 → #938 → this) — merge in order, and a human with Vercel access should hit the preview |
42f4768 to
2b08580
Compare
2b08580 to
bab2df9
Compare
|
Claude finished @edwingao28's task in 1m 23s —— View job Re-review: power provenance ingest — 🟢 LGTMThe branch was rebased since the last review into 5 clean commits (
Migration 014 (highest risk). Re-ran a byte-for-byte comparison of the recreated Deploy-order tolerance. Both read paths ( GPU tooltip (Bugbot finding). Extractor / metrics integrity & supplemental lane. CI. Only the in-progress Runtime. The withheld-power tooltip line only appears for points carrying reason codes, which requires migration 014 applied + populated data (absent in the read-only DB) — consistent with the tolerant-read design, so live rendering of that specific line isn't reproducible yet, as noted in prior reviews. Carried reminder from the PR body: this is a stacked PR (#937 → #938 → this) — merge in order, and a human with Vercel access should hit the preview |
Turn the reserved power_invalid_reasons / power_audit contract fields live end-to-end: - migration 014 adds dedicated jsonb columns on benchmark_results and recreates latest_benchmarks with the migration-012 definition verbatim so br.* picks up the new columns - mapBenchmarkRow narrows both fields defensively (snake_case reason codes, fixed 8-key audit shape, empty -> undefined so the columns store SQL NULL) and both keys join NON_METRIC_KEYS so Number(['5']) === 5 can never mint a bogus numeric metric - bulkIngestBenchmarkRows persists both as jsonb lanes, NULL when absent, refreshed on conflict like workers - all four read paths select the columns via to_jsonb(...) -> 'col' (the PR #405/#407 deploy-order lesson: bare references fail at plan time until the next ingest applies the migration; the jsonb lookup degrades to NULL) - rowToAggDataEntry passes reasons through, and both chart tooltips render a bilingual "Measured power withheld" line with re-sanitized codes - unofficial-run overlay rows carry both fields; the calculator view strips them; OpenAPI wording moves from reserved to live The persistence input is built by spreading (ingest-ci-run.ts {...row, configId}; run-overrides.ts applyBenchmarkPointBackfill {...point}), so the new BenchmarkParams fields flow through with no changes there. 中文:将预留的 power_invalid_reasons / power_audit 契约字段全链路转正:迁移 014 为 benchmark_results 增加两个专用 jsonb 列并按迁移 012 的定义原样重建 latest_benchmarks;mapBenchmarkRow 对两个字段做防御性收窄(snake_case 原因码、 固定 8 键审计对象、空值映射为 undefined 以存储 SQL NULL);批量摄取以 jsonb 通道 持久化并在冲突时刷新;四条读取路径均用 to_jsonb(...) -> 'col' 容错读取(PR 携带同样字段;计算器视图剥离;OpenAPI 文案由预留改为正式。
…ies, and UI | 测试:覆盖功耗审计溯源在映射、摄取、查询与界面各层的行为 - extractPowerInvalidReasons: snake_case validation, dedupe, 32-code cap, 64-char boundary, empty/non-array/all-invalid -> undefined (never []) - extractPowerAudit: 8-field round-trip, Infinity/NaN/junk numerics omitted, negative and non-safe-integer counts rejected, sha trimming and null collapse, unknown keys dropped, empty husk -> undefined - mapBenchmarkRow lands the fields on BenchmarkParams for v1/v2/agentic rows, stores audits from valid rows too, and never mints a numeric metric from a malformed ['5'] reasons array (Number(['5']) === 5) - bulkIngestBenchmarkRows recording-mock: both columns in the INSERT list, two extra ::jsonb[] lanes, null lanes for absent fields, excluded.* refresh on conflict - all four read paths pin the tolerant to_jsonb(...) -> 'col' form and a negative regex guards against bare br./lb. references (the #407 lesson) - rowToAggDataEntry narrows null/[]/absent to undefined; tooltips render the bilingual withheld line only for sanitized codes (en + zh, overlay parity); calculator view strips both fields alongside workers 中文:为溯源字段新增全链路测试——提取函数的收窄规则(原因码校验、去重、上限、 空值处理;审计对象 8 字段往返、异常数值剔除、sha 归一化)、映射到 BenchmarkParams、批量摄取的 jsonb 通道与冲突刷新、四条读取路径的 to_jsonb 容错形式(并用反向正则钉死 #407 教训)、前端窄化与双语提示框行为、计算器 视图剥离。
…| ETL:补充数据摄取通道同样持久化功耗溯源字段 Review found ingest-supplemental.ts participates in the power publication contract (normalize + scrub, PLAN-03) but silently dropped power_invalid_reasons / power_audit: the persistence input carried no provenance fields, so a supplemental entry with them would persist NULL columns. Extract both via the shared narrowers — entry-level fields sibling to metrics (mirroring artifact rows), with a metrics-nested fallback since power_valid rides in metrics in this format — and delete the keys from metrics so the persisted jsonb stays a flat numeric record. Pin the call sequence next to the PLAN-03 supplemental tests.
中文:清理功耗溯源实现中的内部计划标签、重复测试说明和装饰性注释,同时保留迁移顺序、部署容错、数据收窄、安全与持久化约束。
中文:在 GPU 对比图提示中显示实测功耗未采信原因
bab2df9 to
ae58fc8
Compare
|
Claude finished @edwingao28's task in 1m 21s —— View job Re-review: power provenance ingest — 🟢 LGTMThe branch was rebased onto
Migration 014 (highest risk). Re-verified: Deploy-order tolerance. Both read paths ( GPU tooltip (Bugbot finding). ETL / ingest / supplemental lane. Ingest adds two homogeneous jsonb CI. Lint & Format ✅, Tests (Unit) ✅, Code Quality ✅, PR checks ✅. Tests (E2E) still in progress; the delta is a rebase of already-green content plus the one GPU-tooltip line + test, so the outcome is unaffected. No 🔴 findings. Runtime. The withheld-power tooltip line only renders for points carrying reason codes, which requires migration 014 applied + populated data (absent in the read-only DB) — so that specific line isn't reproducible live yet, consistent with the tolerant-read design noted across prior reviews. Carried reminder: a human with Vercel access should hit the preview |
|
Found 2 test failures on Blacksmith runners: Failures
|

Important
STACKED PR — do not merge out of order. Base is
feat/api-power-contract(#938), which is itself stacked on #937. Merge order: #937 → #938 → this PR. This PR contains only the deltas relative to #938.Ingests and exposes the power audit provenance fields (
power_invalid_reasons,power_audit) that the producer (InferenceXaggregate_power.py, via the row-level power provenance change onfeat/power-row-provenancein InferenceX, PR queued) emits alongside thepower_validverdict. #938 reserved the two fields in the OpenAPI schema; this PR turns them live end-to-end while remaining fully tolerant of their absence (legacy artifacts, runs predating the producer change, legacy DB rows).What changed
Storage — migration
014_power_provenance.sqljsonbcolumns onbenchmark_results(power_invalid_reasons,power_audit), mirroring the migration-006workersprecedent:metricsis a flatRecord<string, number>, so structured data gets its own columns.latest_benchmarkswith the migration-012 definition verbatim (recursive append-only-curve body plus both indexes) solely sobr.*picks up the new columns — not the obsolete 006 definition.ETL —
benchmark-mapper.tsNON_METRIC_KEYS:Number(['5']) === 5, so without the guard a malformed single-element reasons array would be auto-captured as a bogus numeric metric.extractPowerInvalidReasons: keeps snake_case codes (≤ 64 chars), dedupes preserving order, caps at 32; empty result →undefinedso the column stores SQL NULL, never[].extractPowerAudit: fixed 8-key shape (window_start_unix,window_end_unix,expected_gpu_count,observed_gpu_count,sample_count,max_sample_gap_s,producer_sha,exporter_image_sha256); finite-number/safe-integer narrowing, malformed numerics omitted (partial audit beats none), shas collapse tonullper the contract'sstring|null, unknown keys dropped, empty husk →undefined.power_valid— tolerance in both directions.ingest-ci-run.tsorrun-overrides.ts: the persistence input is built by spreading ({...row, configId}/applyBenchmarkPointBackfill's{...point}), so the newBenchmarkParamsfields flow through.Ingest —
benchmark-ingest.tsunnestlanes mirroringworkers; NULL when absent;excluded.*refresh on conflict (fresh artifact is authoritative, same asworkers).Reads —
queries/benchmarks.ts(deploy-order safety, the #405/#407 lesson)to_jsonb(br) -> 'power_invalid_reasons'/to_jsonb(lb) -> ...— never bare column references. Migrations run in the ingest workflows, not at Vercel deploy; a bare reference fails at query plan time until the next ingest applies migration 014, which is exactly how PR feat(power): measured-power multinode support (workers[] + per-stage joules) #405 produced a ~63% error rate. The jsonb key lookup degrades to NULL while the column is missing and is byte-identical once it exists, so merge order and deploy timing are irrelevant. A regression test pins the tolerant form with a negative regex.API & frontend
/api/v1/benchmarksand/historyreturn the fields verbatim when stored; calculator view strips them (payload-trimmed projection excludes measured-power data by design); unofficial-run overlay rows carry both fields via the shared mapper.api-route-catalog.tsdigests refreshed for the two changed contract sources.power_auditis API-only — no UI surface (follow-up).Testing
bun run --cwd packages/db test:unit— 650 passed (extractor suites, mapper integration incl. the['5']guard, ingest lane recording-mock, query-shape assertions with the bare-reference regex guard, supplemental-path provenance sequence)bun run --cwd packages/app test:unit— 4491 passed (transform narrowing, tooltip en/zh + injection filtering + overlay parity, unofficial-run overlay parity, calculator strip, OpenAPI docs invariants, catalog guard)bun run typecheck/bun run lint/bun run fmt— cleanpackages/mcpserver.test.tsfails identically on the base branch in this environment (z.enumundefined — zod resolution, unrelated to this change).Deploy-order proof: this PR's Vercel preview serves
/api/v1/benchmarksagainst the un-migrated production DB with the two keys null — the live demonstration of the tolerant-read design. The nextstage-results/ingest run applies migration 014 and new ingests populate the columns.Review notes
Two findings from review, both addressed:
ingest-supplemental.tsparticipates in the power publication contract (the normalize + scrub from fix(etl): strip measured power metrics at ingest when power_valid=0 | ETL:power_valid=0 时在摄取阶段剥离实测功耗指标 #937) but built its persistence input withoutpowerInvalidReasons/powerAudit, so a supplemental entry carrying the fields would persist NULL columns silently. Fixed in the follow-up commit: the lane now extracts both via the shared narrowers (entry-level fields sibling tometrics, with a metrics-nested fallback sincepower_validrides inmetricsin that format) and deletes the keys frommetricsso the persisted jsonb stays a flat numeric record; the call sequence is pinned next to the fix(etl): strip measured power metrics at ingest when power_valid=0 | ETL:power_valid=0 时在摄取阶段剥离实测功耗指标 #937 supplemental tests./api/v1/benchmarkscould not be hit directly during review. Indirect evidence is strong (green Vercel check,to_jsonbreads pinned by a negative-regex test, Postgres jsonb semantics), but someone with Vercel access should hit the preview once and confirm 200 withpower_invalid_reasons/power_auditnull before merging the stack.中文说明
摄取并公开生产端(InferenceX
aggregate_power.py,对应 InferenceX 的行级功耗溯源变更feat/power-row-provenance,PR 排队中)随power_valid判定一同产出的功耗审计溯源字段(power_invalid_reasons、power_audit)。#938 已在 OpenAPI 中预留这两个字段;本 PR 将其全链路转正,并对字段缺失(旧产物、早于该生产端变更的运行、历史数据行)保持完全兼容。benchmark_results上新增两个专用jsonb列(沿用迁移 006workers的先例),并按迁移 012 的定义原样重建latest_benchmarks,使br.*覆盖新列。mapBenchmarkRow防御性收窄两个字段(snake_case 原因码去重限长限量;审计对象固定 8 键、剔除非法数值、sha 归一化为string|null、丢弃未知键;空结果映射为undefined以存储 SQL NULL);两个键加入NON_METRIC_KEYS,避免Number(['5']) === 5生成伪数值指标。to_jsonb(...) -> 'col'容错形式,列尚未迁移时读取为 NULL,迁移后逐字节等价,合并与部署顺序均无关;回归测试用反向正则钉死该形式。测试:db 包 650 项、app 包 4491 项全部通过;typecheck / lint / fmt 干净。(
packages/mcp的server.test.ts在本地环境于基础分支上即失败,与本变更无关。)评审说明:评审中发现两点,均已处理——补充数据摄取通道(
ingest-supplemental.ts)此前未持久化溯源字段,已在后续提交中修复并以单测钉住调用序列;Vercel 预览部署受访问保护(302),评审期间无法直接探测/api/v1/benchmarks,请有权限的同事在合并前访问预览确认返回 200 且两个新键为 null。Note
Medium Risk
Touches benchmark ingest, DB migration/matview recreation, and deploy-order-sensitive read paths; behavior is heavily tested and reads are migration-tolerant, but wrong ingest ordering could still affect
latest_benchmarksuntil refresh.Overview
Adds end-to-end support for producer-emitted power provenance
power_invalid_reasonsandpower_audit, alongside the existingpower_validverdict.Storage & ingest: Migration 014 adds dedicated JSONB columns on
benchmark_results(keepingmetricsnumeric-only) and recreateslatest_benchmarks.mapBenchmarkRowextracts and narrows both fields (snake_case reason codes with caps/dedup; fixed-shape audit object), excludes them from auto-metric capture, andbulkIngestpersists them with upsert refresh. Supplemental ingest now extracts the same fields from entry- or metrics-nested payloads.Reads & API: Benchmark queries select provenance via
to_jsonb(...) -> '...'so deploys stay safe before migration 014 runs on ingest. Public benchmark rows return the fields; the calculator trimmed view drops them. OpenAPI docs move from reserved to live semantics.UI: Chart tooltips (official, overlay, GPU comparison) show a bilingual “Measured power withheld” line when
power_invalid_reasonsis present, with HTML-safe re-validation of codes.power_auditis API-only (no UI yet). Unofficial-run overlays pass provenance throughmapBenchmarkRowlike the DB path.Reviewed by Cursor Bugbot for commit ae58fc8. Bugbot is set up for automated code reviews on this repo. Configure here.