Skip to content

fix: round scru128 timestamp on pack to stop id roundtrip loss - #147

Merged
cablehead merged 1 commit into
mainfrom
fix-id-codec
Aug 6, 2026
Merged

fix: round scru128 timestamp on pack to stop id roundtrip loss#147
cablehead merged 1 commit into
mainfrom
fix-id-codec

Conversation

@cablehead

Copy link
Copy Markdown
Owner

Problem

.id unpack $id | .id pack did not roundtrip for about 3% of scru128 ids. The prefix matched and the low bits diverged, for example:

03gmmrhqljv8bnulus79wipck -> 03gmmrhqleed8w8bsl6cn17tg

This is what made the .id roundtrip assertion in tests/test_xs_nu.nu flaky, since it minted one fresh id per run and tripped roughly 1 run in 35.

Mechanism

.id unpack emits the timestamp as f64 seconds (scru128.rs, ms / 1000.0) and the nu def routes it through a nanosecond datetime. .id pack hands f64 seconds back to the Rust packer, which computed milliseconds with (timestamp * 1000.0) as u64. The as u64 cast truncates toward zero. When the reconstructed f64 millisecond value landed just under an integer (for example 1785966646157.9998 for a true 1785966646158), it dropped a millisecond. That corrupts the 48-bit timestamp field and repacks to a different id.

Fix

Round to the nearest millisecond instead of truncating, in both pack_from_json and pack (src/scru128.rs). The nu datetime detour keeps the value within sub-nanosecond of the true millisecond, so rounding recovers it exactly.

Tests

Both roundtrip tests now pin the known failing ids and sweep 2000 fresh ids per run, replacing the single random-id check. Measured loop failures: 56/2000 before, 0/2000 after. The Rust test_scru128_round_trip and the e2e tests/test_xs_nu.nu both fail reliably on the old code and pass on the fix.

cargo fmt, cargo clippy -D warnings, and cargo test are green. The docs (astro) step of check.sh requires Node >=22.12 and is not run here (local Node is 18.19.1); that is an environment gate, unrelated to this change.

.id unpack emits the timestamp as f64 seconds and routes it through a
nanosecond datetime, then .id pack hands f64 seconds back to the Rust
packer, which computed milliseconds with (timestamp * 1000.0) as u64.
The `as u64` cast truncates toward zero, so any id whose reconstructed
f64 millisecond value landed just under an integer (for example
1785966646157.9998 for a true 1785966646158) dropped a millisecond.
That corrupted the timestamp field and repacked to a different id,
about 3% of the time. Round to the nearest millisecond instead.

Strengthen the roundtrip tests to catch this deterministically: pin the
known failing ids and sweep 2000 fresh ids per run in both the Rust
test and tests/test_xs_nu.nu, instead of checking a single random id
that only tripped roughly 1 run in 35.
@cablehead
cablehead merged commit 701280c into main Aug 6, 2026
8 of 10 checks passed
@cablehead
cablehead deleted the fix-id-codec branch August 6, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant