Skip to content

[TEST] Golden-file regeneration test for report pages - #72

Merged
eleanorfrajka merged 3 commits into
mainfrom
rep/01-golden-files
Aug 13, 2026
Merged

[TEST] Golden-file regeneration test for report pages#72
eleanorfrajka merged 3 commits into
mainfrom
rep/01-golden-files

Conversation

@eleanorfrajka

Copy link
Copy Markdown
Collaborator

Summary

Adds a golden-file safety net for the report HTML: a test that renders every report page for the dune2 fixture and asserts the output is byte-identical to committed reference files. This is the regression net for the ongoing report-subsystem work — a change that is meant to be structural must keep this green (proving it moved no output), and a deliberate visual change re-baselines it so the intended change is the reviewable diff. Includes one small enabling fix so report generation no longer errors when writing outside the processing tree.

The golden test

tests/unit/test_report_golden.py renders the five pages produced for a single-mooring fixture (mooring summary, stack, grid, and both instrument pages) into a temp directory, then compares each against a committed golden after two deterministic normalisations:

  • the generated timestamp — the only wall-clock-varying field — is replaced with <GENERATED>;
  • each base64 PNG payload is replaced with a short content hash (base64,sha256:<16 hex>).

The PNGs are deterministic run-to-run, so the hash is stable; hashing them keeps the golden files small (~210 KB for all five, versus ~3 MB of raw base64) and their diffs readable, while still detecting any change to a rendered figure. Deployment/recovery timestamps and other data values are left untouched, so the test stays sensitive to real content changes.

Re-baseline with REBASELINE_GOLDEN=1 pytest tests/unit/test_report_golden.py. A drift prints a compact, base64-free unified diff and writes the actual output alongside the render for inspection.

The golden fixtures live under tests/fixtures/golden/dune2/. They are normalised test oracles (hashed images, placeholder timestamp), not browsable reports.

Enabling fix — output-location-independent status logging

Adds _safe_rel(path, root) to utilities.py (re-exported via report/_html_helpers.py) and uses it at the four status-log sites in _grid.py and _stack.py. Previously those logged the output path via Path.relative_to(display_root), which raised ValueError — and aborted grid/stack generation with an "ERROR generating …" message — whenever the report was written to a directory outside the processing tree (a custom output dir). The HTML was written correctly before the crash; only the logging line failed. _safe_rel falls back to the bare filename, so a logging call can no longer abort generation. This is what lets the golden test render to a temp directory cleanly.

Testing

Full suite: 761 passed, 9 skipped. ruff check clean. The golden test was verified to fail on a tampered golden and pass on a clean one, confirming it is a real net rather than a no-op.

Breaking changes

None. The new test and fixtures are additive; _safe_rel only changes a logging path (the generated HTML is unaffected — the golden fixtures capture identical output).

@eleanorfrajka
eleanorfrajka merged commit 5a13fd4 into main Aug 13, 2026
5 checks passed
@eleanorfrajka
eleanorfrajka deleted the rep/01-golden-files branch August 13, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant