[kriszyp]

⊙ Problem

The upgrade-impact pipeline needs a caller rule for Table.getRecordCount(): large or slow scans can return estimates, and callers may mistake those numbers for exact counts.

🤔 Does the release framing hold?

The function is identical in v5.2.15 and v5.3.0. The motivating estimator change landed separately on both release lines (#2612); the useful result is a caller-contract rule, rather than a claim that those tags differ.

💡 Solution

Eight caller tests pin default calls, the sampling floor, exactCount, forced high/low estimates, and describe_table forwarding/skipping. The invariant note and root index explain why the estimated range can miss the live count. No runtime behavior changes.

🤔 Pin the estimator undershoot?

Yes. The forced-low case characterizes current behavior for the upgrade scanner. It does not claim that an undershoot is desirable or prevent a future runtime fix from updating this expectation.

Caller impact rule and supporting research

🤔 Why keep focused test seams?

describeTable exposes no time budget: its spy verifies option forwarding and a fixed return verifies field mapping. Real sampling and clamping have separate coverage. Two local estimator fixtures keep the examples independent. LMDB uses a different statistics seam, so only the two forced RocksDB estimator cases skip there; six caller cases still run.

Caller impact rule

(a) Trigger conditions

A call's recordCount is an estimate (and estimatedRange is present) only when all three hold:

1. options.exactCount is not set. exactCount: true always scans to completion; no version of this code path has ever skipped that.
2. The value scan exceeds its time budget (options.timeLimit, default 500ms, checked each ~500ms interval) and has scanned at least MIN_ESTIMATOR_SAMPLE (1,000) entries. Below that floor, the loop always completes exactly.
3. The escape fires: at a checkpoint past the floor, either a base estimate puts the scan short of halfway through the table, or 20 budget checkpoints have passed regardless. A table whose scan is already past halfway when the budget trips, and hasn't hit the checkpoint ceiling, just finishes exactly instead. Even past the floor and the budget, a forward+reverse sample that meets in the middle still returns an exact count.

What changed in September 2026, and what did not: the data source for the halfway decision and the extrapolation base swapped from an exact key-count scan to a cheap sampled estimate. The return shape, the floor, the exactCount escape, and the fact that large/slow tables get an estimate with a range were all already true (since 2023). The numeric value of recordCount/estimatedRange for a table that lands on the sampling path can differ from what an older build would have returned for the identical data — because the extrapolation base is computed differently. The halfway gate also uses that base, so a particular call can switch between exact and estimated results. Both variants were already part of the caller-visible contract.

(b) Caller shapes

Classifications:

• breaks: a call that returned a specific value/shape now returns a different one that the caller cannot handle.
• behavior change only: the number can differ, but the shape and the caller's handling of it are the same.
• unaffected: same contract regardless of version or data size.
• undecidable from code: depends on table size/time at the call site, not on the code shape.

Rows are covered by getRecordCountCallerImpact.test.js, by tests already on main (noted), or are classified from the code alone (the "approximately N" display and the "not this API" row have no test).

caller shape	synthetic example	classification	why
default call, table below 1,000 entries	await Table.getRecordCount()	unaffected	always exact; the estimator path never engages
exactCount: true, any table size	await Table.getRecordCount({ exactCount: true })	unaffected	bypasses the estimator entirely, unconditionally
default call, table ≥1,000 entries, scan finishes within budget	await Table.getRecordCount() on a table whose scan is fast	undecidable from code	exact if the scan finishes in time; estimated otherwise. Depends on table size and runtime load, not the call site
default call, table ≥1,000 entries, scan exceeds budget	same call, large/slow table	behavior change only	estimatedRange was already possible here; its numeric value can shift between builds, and the estimated halfway gate can also change whether a particular call returns a range; both return variants were already supported
caller asserts recordCount === knownLiveCount (tests, "has data changed" checks)	assert.equal((await T.getRecordCount()).recordCount, N)	breaks, once the table is large enough to sample	was never guaranteed even before September 2026 — the point estimate has never been authoritative, and upper is only a heuristic, not a hard bound — but a build whose estimate happened to land on the true value can start landing elsewhere
pagination: Math.ceil(recordCount / pageSize) used as the authoritative last-page boundary	a "page N of M" UI computed from the estimate alone	behavior change only	always approximate once sampling engages, and upper does not even guarantee the true page count is bracketed; the specific wrong answer can shift between builds
pagination: same, but reading estimatedRange to show "approximately N"	`~${recordCount} (${lower}–${upper})`	unaffected	the meaning of the optional estimatedRange signal is unchanged — but see the migration guidance below; upper is advisory, not a hard bound (lower is an exact count of the rows actually sampled, so it is trustworthy as a floor)
describe_table/describe_all, default	{"operation":"describe_table",...}	behavior change only	record_count/estimated_record_range pass through getRecordCount() unchanged in shape
describe_table with exact_count: true	{"operation":"describe_table","exact_count":true,...}	unaffected	forwards straight to exactCount
describe_table with skip_record_count: true	{"operation":"describe_table","skip_record_count":true,...}	unaffected	getRecordCount is never called
REST GET /Table/ (describe, no id), default	GET /Product/	unaffected	recordCount stays undefined; getRecordCount is never called unless the caller opts in (already pinned in basicREST-test.mjs)
REST GET /Table/ with includeExpensiveRecordCountEstimates	same, with the context flag set	behavior change only	same reasoning as the default operations call, but note REST uses recordCount/estimatedRecordRange (camelCase), not the operations API's snake_case fields, and has no exact_count/skip_record_count equivalent
not this API at all	Map.prototype.get/size, a custom getRecordCount on a non-table Resource that never calls super	unaffected	never reaches this code

Migration guidance for flagged callers:

• Treating recordCount as exact for a table that can exceed ~1,000 rows: read estimatedRange, but treat upper as advisory, not a hard bound — pass exactCount: true/exact_count: true instead if an authoritative number is actually needed and the cost is acceptable. lower is safe to treat as a real floor (it is a direct count, not an estimate).
• A small, bounded table (test fixtures, config tables) that will never cross the 1,000-row floor: no action needed; it is always exact.
• Code that only displays "~N records" or "~N pages": no action needed; that is exactly what the contract has always promised, and always with the same upper-is-heuristic caveat.

(c) Intended contract change, not a regression

No contract changed. The internal data-source swap is an intended performance fix, not a behavior change:

evidence	what it says
921a635 commit message	"getRecordCount's escape path exists because the value scan already blew its time budget, and it answered by starting an exact getKeysCount() — a second, unbounded scan of the same key space. Measured on rocksdb-js 2.9.0 ... that is 92ms at 1M keys and 934ms at 10M ... estimateCount() is flat and under 0.05ms at both sizes."
same commit	"The base now comes from a CountEstimator checkpointed as the scan advances ... Verified across nine churn distributions that the range contains the true live count in all of them." — an empirical check over nine distributions, not a mathematical guarantee; Table.getRecordCount's own comment calls the result "a widened heuristic interval, not a guaranteed bound," and this PR's underestimate test injects fixed estimator answers and shows upper landing below the live count
tests added with 921a635	grow getRecordCount.test.js to cover checkpoint deltas, calibration under churn, and estimator-failure fallback — all internal mechanics, none change the return shape
resources/Table.ts:1020-1029 type declaration	unchanged across v5.2.15/v5.3.0/main: Promise<{recordCount: number, estimatedRange?: undefined} | {recordCount: number, estimatedRange: number[]}>

(d) Static-detection recipe

tier	pattern	meaning
1	\.getRecordCount\s*\( (any receiver)	candidate. Confirm the receiver reaches Table/a table class, not an unrelated getRecordCount
2	tier 1 without a nearby exactCount\s*:\s*true in the same call's argument object	undecidable from code unless the table is provably small (a fixture/config table). Flag for the caller shapes in (b) above: equality assertions and page-math-as-authoritative are the ones worth a human look
3	exactCount\s*:\s*true inside the call, or exact_count\s*:\s*true in an operations-API object	unaffected; drop from the candidate list
4	\brecord_count\b / \bestimated_record_range\b read from an operations response, or \.recordCount\b / \.estimatedRange\b read off a getRecordCount() result	same classification as tier 2/3 above, one level removed — flag if the surrounding code does an equality comparison or arithmetic that assumes exactness (===, Math.ceil(x.recordCount / ...) used as a hard boundary)

False positives: getRecordCount defined on an unrelated class; exactCount/exact_count passed as a variable that is provably always true at that call site (data-flow, not a line regex); a comment or test name containing the string.

False negatives: exactCount passed as a variable that is sometimes false (the regex only catches the literal true); a wrapper function that calls getRecordCount() internally and the caller only sees the wrapper; destructured imports (const { getRecordCount } = Table) renaming the call; a skip_record_count-aware caller that computes its own count elsewhere and compares it against a different record's cached estimate.

On real code: zero non-test, non-comment hits for \.getRecordCount\s*\( outside resources/Table.ts, dataLayer/schemaDescribe.ts and their tests in Harper's own tree. This API has exactly the two call sites documented above inside Harper; the scanner's job is finding it in application code.

(e) Main versus the target release

Main behaves as v5.3.0 (and v5.2.15) did:

• getRecordCount's implementation at resources/Table.ts is byte-identical across v5.2.15, v5.3.0 and main — confirmed by diffing the function body (not just surrounding context) directly.
• The caller-visible contract (return shape, 1,000-entry floor, exactCount escape, describe_table/REST pass-through) has been stable since 30f4df6 (2023-10-19).
• No drift since either release tag.

✅ Verification

• Rebased onto current main; npm run build, lint:required, Prettier and check:design-docs pass. Focused run (caller-impact, getRecordCount, eviction and LMDB close-completion suites): 32 passing, 3 skipped.
• CI on this head: every unit and integration job passes (Node 22/24/26, Windows, Bun, uWS HTTP).

PR QA: skipped @ 173b524 — tests and documentation only.

Docs companion: none; this records an existing internal contract and changes no public API.

_{🤖 Generated by OpenAI Codex; posted via @kriszyp.}

Release target: main — tests and documentation only
Related PRs: 13 others independent
Complexity: medium

_{Dispatch: task harper-upgrade-rule-record-count-530 · queued by unknown · ran by claude/sonnet/high · worker kzyp-xps-1}

Deferred findings

• a1496b2b35969f76 minor test-coverage — unitTests/resources/getRecordCountCallerImpact.test.js — Characterization requires an underestimated upper endpoint — NOT FILED
• 9c54836bfc5f4abc minor correctness — resources/Table.ts — Whole-store fallback only on zero remainder; low positive remainder yields upper below live count — NOT FILED
• 8fc12de59c6fb7b3 nit test-coverage — dataLayer/schemaDescribe.ts — describe_table sets estimated_record_range to explicit undefined on exact counts; no test pins missing-key vs undefined — NOT FILED

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex; adjudicated=domain; declined=cursor-grok,muse; rounds=12; full=3 @ 173b524}

_{Review-Attention: skim ~2m (decisions: pin-current-undershoot, contract-in-design-note) @ 173b524}

[gemini-code-assist]

Code Review

This pull request documents the heuristic nature of getRecordCount in DESIGN.md and introduces a comprehensive test suite in getRecordCountCallerImpact.test.js to verify its behavior under various conditions. The review feedback recommends optimizing the test setup by batching put operations instead of executing thousands of concurrent requests via Promise.all, which can cause performance overhead or timeouts in CI environments.