Repository navigation
test: mask PDF region in visual regression snapshots - #448
Conversation
✅ Deploy Preview for graypaper-reader ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (52)
📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthroughA new ChangesPDF Viewer Masking in Snapshot Tests
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
The PDF.js canvas occasionally renders a few pixels off between otherwise identical runs. Because a positional shift moves every glyph edge, it changes a band of pixels around all text and reliably trips the 2% pixel-diff threshold, making the snapshot suite flaky. Rather than loosening the global threshold (which would also blind us to real layout regressions of the same magnitude), mask the PDF viewer(s) out of the layout/chrome snapshots. The surrounding UI keeps the tight tolerance; the inherently non-deterministic PDF region is ignored. - Add maskedPdfScreenshot(page) helper masking .pdf-viewer-root (matches both the main viewer and the split-pane viewer) - Apply it to all full-page snapshots in basic/split-screen/narrow-screen specs (clip-based notes-tab shots already exclude the PDF, left untouched) - Regenerate the linux baselines Trade-off: these tests no longer detect PDF *render* regressions (wrong page, blank canvas, broken scale) — only that it loads. A dedicated PDF-render test with its own looser tolerance could restore that coverage if wanted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ba790b1 to
552cb3a
Compare
Visual Regression Test Report ✅ PassedGithub run id: 27945268230 🔗 Artifacts: Download |
Problem
The visual regression suite fails intermittently because the PDF.js canvas occasionally renders a few pixels off between otherwise identical runs. A positional shift is the worst case for pixel-diff thresholds: it moves every glyph edge, changing a band of pixels around all text, which reliably trips the existing
maxDiffPixelRatio: 0.02threshold.Approach
Rather than loosening the global threshold (which would also blind us to genuine layout regressions of the same magnitude — and these are layout tests), this masks the PDF viewer out of the layout/chrome snapshots. The surrounding UI keeps the tight 2% tolerance; the inherently non-deterministic PDF region is ignored.
Playwright paints a solid magenta box over the masked element's bounding box. The jitter lives in the canvas content, not in the position of
.pdf-viewer-root(anabsolute-positioned container that fills its pane), so the masked comparison is deterministic.Changes
tools/snapshot-tests/tests/utils/screenshot-opts.ts—maskedPdfScreenshot(page)masks.pdf-viewer-root(matches both the main and split-pane viewers)basic-snapshots,split-screen, andnarrow-screenspecsclip-based notes-tab shots already exclude the PDF and were left untouchednpm run docker:test:updateExample regenerated baseline (homepage outline) — PDF area masked, all chrome still verified:
darwinbaselines are now stale. Only the linux baselines (used by CI) were regenerated. Anyone running the suite locally on macOS will needcd tools/snapshot-tests && npm run test:local -- --update-snapshots, or we can regenerate+commit them in a follow-up.data-pdf-ready). A dedicated PDF-render test with its own looser tolerance could restore that coverage if desired.🤖 Generated with Claude Code
Summary by CodeRabbit