[TEST] Golden-file regeneration test for report pages - #72
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a golden-file safety net for the report HTML: a test that renders every report page for the
dune2fixture and asserts the output is byte-identical to committed reference files. This is the regression net for the ongoing report-subsystem work — a change that is meant to be structural must keep this green (proving it moved no output), and a deliberate visual change re-baselines it so the intended change is the reviewable diff. Includes one small enabling fix so report generation no longer errors when writing outside the processing tree.The golden test
tests/unit/test_report_golden.pyrenders the five pages produced for a single-mooring fixture (mooring summary, stack, grid, and both instrument pages) into a temp directory, then compares each against a committed golden after two deterministic normalisations:generatedtimestamp — the only wall-clock-varying field — is replaced with<GENERATED>;base64,sha256:<16 hex>).The PNGs are deterministic run-to-run, so the hash is stable; hashing them keeps the golden files small (~210 KB for all five, versus ~3 MB of raw base64) and their diffs readable, while still detecting any change to a rendered figure. Deployment/recovery timestamps and other data values are left untouched, so the test stays sensitive to real content changes.
Re-baseline with
REBASELINE_GOLDEN=1 pytest tests/unit/test_report_golden.py. A drift prints a compact, base64-free unified diff and writes the actual output alongside the render for inspection.The golden fixtures live under
tests/fixtures/golden/dune2/. They are normalised test oracles (hashed images, placeholder timestamp), not browsable reports.Enabling fix — output-location-independent status logging
Adds
_safe_rel(path, root)toutilities.py(re-exported viareport/_html_helpers.py) and uses it at the four status-log sites in_grid.pyand_stack.py. Previously those logged the output path viaPath.relative_to(display_root), which raisedValueError— and aborted grid/stack generation with an "ERROR generating …" message — whenever the report was written to a directory outside the processing tree (a custom output dir). The HTML was written correctly before the crash; only the logging line failed._safe_relfalls back to the bare filename, so a logging call can no longer abort generation. This is what lets the golden test render to a temp directory cleanly.Testing
Full suite: 761 passed, 9 skipped.
ruff checkclean. The golden test was verified to fail on a tampered golden and pass on a clean one, confirming it is a real net rather than a no-op.Breaking changes
None. The new test and fixtures are additive;
_safe_relonly changes a logging path (the generated HTML is unaffected — the golden fixtures capture identical output).