obs_helpers: seed noise from crc32 instead of builtin hash#315
Open
Lftw wants to merge 1 commit into
Open
Conversation
The hash-seeded random helpers (cerror_hash, hashrandn, hashrand, hashmultivariaterandn) seeded numpy from the builtin hash() of a repr string. CPython salts str hashing per process (PYTHONHASHSEED), so simulated noise and gain/D-term errors were not reproducible across processes even with the same seed argument. Route all four through a new _stable_seed() helper using zlib.crc32 -- deterministic across processes, machines, and Python versions -- preserving the per-quantity independence the helpers rely on. Add tests/test_obs_helpers.py pinning the crc32 implementation and asserting identical draws across PYTHONHASHSEED via subprocesses.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev-backend #315 +/- ##
============================================
Coverage 47.44% 47.45%
============================================
Files 55 55
Lines 26984 26987 +3
Branches 4599 4599
============================================
+ Hits 12803 12806 +3
Misses 12689 12689
Partials 1492 1492 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collaborator
|
@Lftw Looks good! For the next PR, we can do local generator seeded through |
rohandahale
approved these changes
Jul 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The hash-seeded random helpers in
obs_helpers.py(cerror_hash,hashrandn,hashrand,hashmultivariaterandn) seeded numpy from the builtinhash()of a repr string built from each quantity's identity (sites/time/pol/seed). They now route through a single new_stable_seed()helper that useszlib.crc32.Why
CPython salts
strhashing per process (PYTHONHASHSEED), so the seed and all simulated thermal noise and gain/D-term errors changed on every run, even with the sameseedargument.zlib.crc32of the same text is identical across processes, machines, and Python versions, so simulated noise is now reproducible while preserving the per-quantity independence the helpers were designed for (draws keyed to a visibility's identity, independent of data ordering).Surfaced while regression-testing simulated observations: clean (noise-free) obs were bit-identical across processes but noisy obs were not; pinning
PYTHONHASHSEED=0made them identical, which pointed at the saltedhash().How to test
pytest tests/test_obs_helpers.pypins the crc32 implementation and asserts drawn noise/gain values are byte-identical acrossPYTHONHASHSEED∈ {0, 1, 999999} via subprocesses (fails on the oldhash()).test_obs_simulate,test_self_cal,test_network_cal,test_pol_cal,test_polgains_cal,test_caltable,test_obsdata): 340 passed.Notes