fix: round scru128 timestamp on pack to stop id roundtrip loss - #147
Merged
Conversation
.id unpack emits the timestamp as f64 seconds and routes it through a nanosecond datetime, then .id pack hands f64 seconds back to the Rust packer, which computed milliseconds with (timestamp * 1000.0) as u64. The `as u64` cast truncates toward zero, so any id whose reconstructed f64 millisecond value landed just under an integer (for example 1785966646157.9998 for a true 1785966646158) dropped a millisecond. That corrupted the timestamp field and repacked to a different id, about 3% of the time. Round to the nearest millisecond instead. Strengthen the roundtrip tests to catch this deterministically: pin the known failing ids and sweep 2000 fresh ids per run in both the Rust test and tests/test_xs_nu.nu, instead of checking a single random id that only tripped roughly 1 run in 35.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
.id unpack $id | .id packdid not roundtrip for about 3% of scru128 ids. The prefix matched and the low bits diverged, for example:This is what made the
.id roundtripassertion intests/test_xs_nu.nuflaky, since it minted one fresh id per run and tripped roughly 1 run in 35.Mechanism
.id unpackemits the timestamp as f64 seconds (scru128.rs, ms / 1000.0) and the nu def routes it through a nanosecond datetime..id packhands f64 seconds back to the Rust packer, which computed milliseconds with(timestamp * 1000.0) as u64. Theas u64cast truncates toward zero. When the reconstructed f64 millisecond value landed just under an integer (for example1785966646157.9998for a true1785966646158), it dropped a millisecond. That corrupts the 48-bit timestamp field and repacks to a different id.Fix
Round to the nearest millisecond instead of truncating, in both
pack_from_jsonandpack(src/scru128.rs). The nu datetime detour keeps the value within sub-nanosecond of the true millisecond, so rounding recovers it exactly.Tests
Both roundtrip tests now pin the known failing ids and sweep 2000 fresh ids per run, replacing the single random-id check. Measured loop failures: 56/2000 before, 0/2000 after. The Rust
test_scru128_round_tripand the e2etests/test_xs_nu.nuboth fail reliably on the old code and pass on the fix.cargo fmt,cargo clippy -D warnings, andcargo testare green. Thedocs(astro) step ofcheck.shrequires Node >=22.12 and is not run here (local Node is 18.19.1); that is an environment gate, unrelated to this change.