Skip to content

feat(bench): upscaler motion probe on a frozen copy of @pmndrs/motion-vectors - #108

Merged
DennisSmolek merged 4 commits into
mainfrom
claude/motion-velocity-optical-flow-ab8d0b
Nov 17, 2026
Merged

DennisSmolek merged 4 commits into
mainfrom
claude/motion-velocity-optical-flow-ab8d0b

Conversation

@DennisSmolek

Copy link
Copy Markdown
Member

Adds a bench probe that measures how motion vectors affect the upscaler, built on a frozen copy of the motion analysis passes. The passes themselves now live in pmndrs/motion-vectors, with the lab, the contract and the findings.

What's here

  • packages/motion/: a private, frozen copy (@pmndrs/upscaler-motion, not bundled) of @pmndrs/motion-vectors' src/. Its README says not to edit it here.
  • bench/motion-upscaler.html + scripts/measure-motion-upscaler.mjs (EXPERIMENTAL): for each upscale path, the warp error and the resets at cuts, on Q19 and Q10. Three experiments: audit-q19, warp and cut.
  • Docs: docs/README.md points to the new repo. UPSTREAM-CANDIDATES.md gains Thin-feature locks can't engage under a noisy SSGI input (follow-up to 10-ssgi-denoise) #17 (several renders per tick pin VelocityNode's previous camera) and feat!: preserve alpha through the upscale (#15) #18 (velocity ignores deformation and setProjectionMatrix jitter; three r187 fixes the projection half).
  • An eslint rule keeps packages/motion from importing the upscaler, so the copy stays a plain copy.

What it doesn't touch
The library, dist/ and the published subpaths don't change. Nothing outside the bench imports packages/motion, and three stays on r186.1.

Follow-up (later, by design)
The upscaler is frozen until motion-vectors and the downsampler finish. Once @pmndrs/motion-vectors 0.x is on npm, the bench takes it as a dev dependency, packages/motion/ is deleted, and the motion part of docs/engine-contract.md moves to the new repo's CONTRACT.md.

Verification
pnpm lint, pnpm typecheck, pnpm test (605) and pnpm build pass, and so does a frozen-lockfile install. The probe's cut experiment reproduces the recorded numbers on Metal (EMD 1.2844, auto-reset at frame 60). No shaders in the library changed.

🤖 Generated with Claude Code

DennisSmolek and others added 4 commits November 14, 2026 00:37
…tion

Adds an experimental, bench-only motion package (packages/motion,
not bundled into dist) and a GPU lab to measure what motion analysis
can do for the upscaler and our other temporal tools.

- cameraMotion: static-world motion from depth in three's velocity
  convention; matches three's velocity to four decimals on every Q19
  camera segment.
- motionResidual: scores candidate motion fields (zero, engine, depth)
  by warp residual with an occlusion test; also gives a reference-free
  warping-error metric for upscaler output.
- lumaHistogram: histogram cut detection; earth mover's distance
  separates Q10's cut 7x from Q19 camera motion.
- bench/motion-lab.html + scripts/measure-motion.mjs: deformers scene
  and Q19/Q10 experiments with exact GPU readback.

Findings in docs/research/MOTION.md (three's velocity reports vertex
deformation as motion; camera history is per animation tick; the
projection override is baked at shader build), two new upstream
candidates, and the spec + phased plan for splitting this into its own
package in docs/research/MOTION-VECTORS-PLAN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The motion package leaves for its own repo right after Phase 1, shaped
like the downsampler (one package, subpath per engine, optional peers).
Records the name (repo pmndrs/motion-vectors, package
@pmndrs/motion-vectors), replaces the split/keep gate with a 0.x
publish gate, and points the docs index and CLAUDE.md at the new repo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The motion package and its lab now live entirely under packages/motion,
ready to move to pmndrs/motion-vectors with their history.

- Oracle: every lab object writes its exact previous-frame motion (own
  camera matrices, own previous model matrices, deformations re-run at
  last frame's time) into an extra MRT output; motionCompare.ts scores
  each candidate as endpoint error in render pixels per object.
- Lab scenes of its own: deformers, move-then-hold cases, water (swell +
  glint + ripples), a courtyard camera track and a cut timeline; knobs
  for jitter, the setProjectionMatrix trap and renders per tick.
- No upscaler imports (eslint): allocator, three glue and CDP helpers
  are copied. The upscaler keeps its consumer side as
  bench/motion-upscaler.html + scripts/measure-motion-upscaler.mjs.
- Contract draft: packages/motion/CONTRACT.md.

Findings (three r186.1, MOTION.md §7): three's velocity is exact for
rigid and camera motion but errs 3-64+ px on deformation, 10.6/23.9 px
at 2/4 renders per tick and 0.36 px after a runtime jitter switch. The
luma histogram misses in-scene cuts; the depth-consistency share
separates them (44-99 % vs at most 5 %).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lab, its driver, the contract, MOTION.md and the plan moved to
pmndrs/motion-vectors with their history (Phase E). The upscaler keeps:

- packages/motion/src as a frozen copy (README says so), only for the
  bench's upscaler motion probe, until @pmndrs/motion-vectors 0.x is on
  npm and the bench takes it as a dev dependency;
- bench/motion-upscaler.html + scripts/measure-motion-upscaler.mjs.

Docs point at the new repo; upstream candidate #18 notes that three
r187 fixes its setProjectionMatrix half.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DennisSmolek
DennisSmolek merged commit 7f23149 into main Nov 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant