Skip to content

bench: diagnose why temporal trails FSR1 in motion (+ motion-adaptive cap candidate) - #107

Draft
DennisSmolek wants to merge 1 commit into
mainfrom
claude/motion-gap-diagnosis
Draft

DennisSmolek wants to merge 1 commit into
mainfrom
claude/motion-gap-diagnosis

Conversation

@DennisSmolek

Copy link
Copy Markdown
Member

Diagnosis follow-up to #104: why does temporal trail FSR1 spatial on every Q4/Q19 motion segment (−0.01 to −2.13 dB, production RCAS) while leading on every still one? This PR changes no production shader or library default. The fix is a bench-only prototype; whether to ship it is a separate decision.

Ranked causes

Full-frame PSNR at ratio 2 on the worst segments, in the order Q4 translate / Q19 orbit-left / Q4 orbit. Positive numbers are dB recovered toward FSR1.

# cause contribution
1 Accumulation length under motion. A 24-frame chain of Catmull-Rom history resamples loses thin high-contrast features (grid lines, fence edges, silhouettes), in both frequency bands. +2.58 / +1.72 / +1.58 dB with maxAccumulation 2
2 Single-frame upsample quality (Lanczos2 vs EASU) −0.02 / −0.31 dB, measured at maxAccumulation 1
3 Auto-exposure +0.28 / −0.10 dB when off
4 Luminance locks +0.09 / 0.00 / +0.11 dB when off
5 Disocclusion, young history and the screen border band 0.07 / 0.09 dB combined
6 Shading-change detector 0.00 dB
7 Input mip bias (from the mip-bias A/B session) gap changes by +0.09 / −0.26 dB
— Harness alignment none: every mode peaks at reference offset 0

Supporting evidence:

  • History under motion is mature (21.7 of 24 frames on average), not young or rejected.
  • The signed temporal-minus-FSR1 error map is red almost only along thin lines and edges.
  • Fetching history bilinearly instead of with Catmull-Rom costs another 0.55 / 0.90 dB, so the resampling filter is on the critical path.
  • A global cap is not the fix: maxAccumulation 4 everywhere keeps still-scene PSNR but raises still-scene temporal error 6× (0.118 → 0.706/255).

Candidate fix (bench-only): motion-adaptive accumulation cap

bench/src/candidates/shaders/accumulateVariants.ts rewrites only the blend of production ACCUMULATE_SHADER:

motionCap = mix(maxAccumulation, cap, smoothstep(lo, hi, motionTexels))

It is wired through a new internal _accumulateShader option, _-prefixed and never exported, like _depthClipShader.

accumulate-motion-cap2-early-v1 uses cap 2 with a 0.05 → 0.5 px/frame ramp, the still-scene relax's own fade. Results:

  • Temporal moves from −0.01…−2.13 dB below FSR1 to +0.01…+0.50 dB above it on all 8 motion windows, with lower temporal error in each.
  • Every still window (Q1, Q12, Q18, Q19 still, Q4 hold) is unchanged to 0.01 dB, with identical temporal error.
  • Q0's animated objects gain +0.34 dB.
  • Holdout scenarios (not used to choose the constants): Q2's slow aliasing dolly goes from −1.43 to −0.46 dB against FSR1, and Q3 (object motion) from +0.29 to +0.75.

Not measured yet: GPU cost (expected negligible), ratio 3, the effect-graph scenarios, and an interactive look at slow pans for crawl.

Tooling

  • measure-reference-quality.mjs gains:
    • --reference-offset, for alignment checks;
    • --save-diagnostics, which keeps presented frames and temporal guides;
    • per-window guide stats in the summary: history age, disocclusion, motion.
  • New scripts/analyze-reference-error.mjs splits one mode's deficit against another by pixel class (border, disoccluded, young, edge by motion third, interior) and by frequency band, and writes a signed difference PNG.

Write-up with every reproduce command: bench/docs/REFERENCE-QUALITY.md, section "Why temporal trails FSR1 in motion". PAPER-NOTES.md and BENCHMARKING.md's identity table are updated.

Follow-ups (not in this PR)

  • TemporalGuides.history (in types.ts) and docs/temporal-guides.md document .a as accumulation age in frames. The shader actually stores age ÷ maxAccumulation (0–1).
  • Adoption of the cap: an ABBA timing run, ratio 3, and interactive review.

Verification

  • pnpm lint, pnpm typecheck and pnpm test pass (593 tests, including 2 new GPU-free structural tests for the candidate shaders).
  • All GPU runs on Apple Metal-3 in headless Chrome 157: no non-finite values, no browser errors, and the canvas presentation check within 1/255.

🤖 Generated with Claude Code

…ap candidate

Ranks the causes of temporal's motion deficit to FSR1 (Q4/Q19, production
RCAS) against the ground-truth reference:

1. Accumulation length under motion: +2.58 / +1.72 / +1.58 dB recovered on
   Q4 translate / Q19 orbit-left / Q4 orbit with maxAccumulation 2. The loss
   sits on thin high-contrast features in both frequency bands; bilinear
   history costs another 0.55-0.90 dB, so the resampling chain is on the path.
2. Lanczos2 vs EASU single-frame upsample: -0.02 / -0.31 dB.
3-7. Auto-exposure, locks, disocclusion/young/border, the shading detector
   and input mip bias each measure <= 0.3 dB; harness alignment is exact.

Prototype (bench-only, production shader and defaults untouched): a
motion-adaptive accumulation cap via a new internal `_accumulateShader`
option. `accumulate-motion-cap2-early-v1` puts temporal at or above FSR1 on
all 8 motion windows (+0.01..+0.50 dB, from -0.01..-2.13), leaves every still
window unchanged, and recovers +0.97 dB on the held-out Q2.

Tooling: measure-reference-quality.mjs gains --reference-offset,
--save-diagnostics and per-window guide stats; analyze-reference-error.mjs
attributes the deficit by pixel class and frequency band.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant