Repository navigation
feat(bench): upscaler motion probe on a frozen copy of @pmndrs/motion-vectors - #108
Merged
Merged
Conversation
…tion Adds an experimental, bench-only motion package (packages/motion, not bundled into dist) and a GPU lab to measure what motion analysis can do for the upscaler and our other temporal tools. - cameraMotion: static-world motion from depth in three's velocity convention; matches three's velocity to four decimals on every Q19 camera segment. - motionResidual: scores candidate motion fields (zero, engine, depth) by warp residual with an occlusion test; also gives a reference-free warping-error metric for upscaler output. - lumaHistogram: histogram cut detection; earth mover's distance separates Q10's cut 7x from Q19 camera motion. - bench/motion-lab.html + scripts/measure-motion.mjs: deformers scene and Q19/Q10 experiments with exact GPU readback. Findings in docs/research/MOTION.md (three's velocity reports vertex deformation as motion; camera history is per animation tick; the projection override is baked at shader build), two new upstream candidates, and the spec + phased plan for splitting this into its own package in docs/research/MOTION-VECTORS-PLAN.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The motion package leaves for its own repo right after Phase 1, shaped like the downsampler (one package, subpath per engine, optional peers). Records the name (repo pmndrs/motion-vectors, package @pmndrs/motion-vectors), replaces the split/keep gate with a 0.x publish gate, and points the docs index and CLAUDE.md at the new repo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The motion package and its lab now live entirely under packages/motion, ready to move to pmndrs/motion-vectors with their history. - Oracle: every lab object writes its exact previous-frame motion (own camera matrices, own previous model matrices, deformations re-run at last frame's time) into an extra MRT output; motionCompare.ts scores each candidate as endpoint error in render pixels per object. - Lab scenes of its own: deformers, move-then-hold cases, water (swell + glint + ripples), a courtyard camera track and a cut timeline; knobs for jitter, the setProjectionMatrix trap and renders per tick. - No upscaler imports (eslint): allocator, three glue and CDP helpers are copied. The upscaler keeps its consumer side as bench/motion-upscaler.html + scripts/measure-motion-upscaler.mjs. - Contract draft: packages/motion/CONTRACT.md. Findings (three r186.1, MOTION.md §7): three's velocity is exact for rigid and camera motion but errs 3-64+ px on deformation, 10.6/23.9 px at 2/4 renders per tick and 0.36 px after a runtime jitter switch. The luma histogram misses in-scene cuts; the depth-consistency share separates them (44-99 % vs at most 5 %). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lab, its driver, the contract, MOTION.md and the plan moved to pmndrs/motion-vectors with their history (Phase E). The upscaler keeps: - packages/motion/src as a frozen copy (README says so), only for the bench's upscaler motion probe, until @pmndrs/motion-vectors 0.x is on npm and the bench takes it as a dev dependency; - bench/motion-upscaler.html + scripts/measure-motion-upscaler.mjs. Docs point at the new repo; upstream candidate #18 notes that three r187 fixes its setProjectionMatrix half. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a bench probe that measures how motion vectors affect the upscaler, built on a frozen copy of the motion analysis passes. The passes themselves now live in pmndrs/motion-vectors, with the lab, the contract and the findings.
What's here
packages/motion/: a private, frozen copy (@pmndrs/upscaler-motion, not bundled) of@pmndrs/motion-vectors'src/. Its README says not to edit it here.bench/motion-upscaler.html+scripts/measure-motion-upscaler.mjs(EXPERIMENTAL): for each upscale path, the warp error and the resets at cuts, on Q19 and Q10. Three experiments:audit-q19,warpandcut.docs/README.mdpoints to the new repo.UPSTREAM-CANDIDATES.mdgains Thin-feature locks can't engage under a noisy SSGI input (follow-up to 10-ssgi-denoise) #17 (several renders per tick pinVelocityNode's previous camera) and feat!: preserve alpha through the upscale (#15) #18 (velocity ignores deformation and setProjectionMatrix jitter; three r187 fixes the projection half).packages/motionfrom importing the upscaler, so the copy stays a plain copy.What it doesn't touch
The library,
dist/and the published subpaths don't change. Nothing outside the bench importspackages/motion, and three stays on r186.1.Follow-up (later, by design)
The upscaler is frozen until motion-vectors and the downsampler finish. Once
@pmndrs/motion-vectors0.x is on npm, the bench takes it as a dev dependency,packages/motion/is deleted, and the motion part ofdocs/engine-contract.mdmoves to the new repo'sCONTRACT.md.Verification
pnpm lint,pnpm typecheck,pnpm test(605) andpnpm buildpass, and so does a frozen-lockfile install. The probe'scutexperiment reproduces the recorded numbers on Metal (EMD 1.2844, auto-reset at frame 60). No shaders in the library changed.🤖 Generated with Claude Code