Repository navigation
Verify complete static-sharding reads and pin local-only filtering fix - #1130
Merged
Merged
Conversation
Dispatch-Task: harper-pro-main-red-static-sharding-20261022 Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Dispatch-Task: harper-pro-main-red-static-sharding-20261022 Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Keep the strengthened static-sharding assertion and include the coordinated runtime fix from HarperFast/harper#3571. Dispatch-Task: harper-pro-main-red-static-sharding-20261022 Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Include the computed-sort completeness repair in the coordinated core draft HarperFast/harper#3571. Dispatch-Task: harper-pro-main-red-static-sharding-20261022 Co-Authored-By: GPT-5 Codex <noreply@openai.com>
There was a problem hiding this comment.
Code Review
This pull request updates static sharding integration tests and documentation to verify that limit queries load complete rows from pooled stubs without changing placement. It introduces a SettledReads resource and enhances LocalRows with additional metadata flags and residency information. Two issues were identified in the test fixture changes: first, new ArrayBuffer(4) is used where new SharedArrayBuffer(4) is required for Atomics operations to avoid a TypeError; second, existingEntry in _storeReplica can be nullish, which would cause a TypeError when accessing existingEntry.key in the finally block.
Preserve the current core ancestry and upstream replication repairs while retaining the reviewed local-only fix and strengthened static-sharding test. Dispatch-Task: harper-pro-main-red-static-sharding-20261022 Co-Authored-By: GPT-5 Codex <noreply@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
⊙ Problem
The static-sharding test still expects an empty pooled nonowner query after #1087 made invalidation stubs searchable and ordinary queries hydrate their full records. Strengthening it exposed a runtime defect: local-only queries returned partial
{id, tag}stubs. The pooled PR job also omitted this suite.🤔 Keep complete results and placement guarantees
Keep #1087: pooled reads return three distinct complete matches; unpooled and local-only reads return
[]. Both nonowners retain their stub flags, payloads and residency.💡 Solution
Pin the coordinated core fix and retain the strengthened static-sharding assertions. Core's
Table.transformEntryForSelectnow checks invalidation flags after lazy primary-entry loading. Core #3527's rewrite must preserve this filter and the local-only SKIP handling inTable.transformToOrderedSelect, plus the null-safe field access ingetAttributeValueand theTable.searchflag enabling the collection filter before sort-key resolution.The fixture observes raw metadata and waits for returned rows' replica-storage attempts to settle. The existing pooled PR job now includes this suite. The design note describes indexed-stub hydration and its cross-shard limit.
⚖️ Alternatives
🤔 Preserve the local-only assertion
Returning partial stubs is the defect. Weakening the assertion or changing invalidation indexing is unsuitable. Cross-shard merged limits remain #198/#436. This test checks complete results from existing local candidates, not a shard-wide search or speculative-fetch budget.
🤔 Keep the current PR runtime selection
Main/nightly already run Node 22/24/26. Add this suite to the pooled PR job without tripling its five unrelated suites; validate all twelve Node/pool/engine combinations locally. The fixture's completion fence follows the current admission call and must change if that call is removed.
Planning decision and CI capacity
Framing-Verdict: better-alternative-exists, resolved by adopting the narrower CI change and deterministic completion fence, then coordinating the required runtime repair. The updated pooled PR job passed in 4m02s, including setup and this suite (43 passing, two skips). Retain the 25-minute cap.
✅ Verification
The final twelve-leg matrix ran on Pro1201ab2b/corec1e192c7: 11/12 legs pass. Node24/LMDB/pool0 has replication-barrier timeouts also observed on current main. Core compiled/source regressions pass61 RocksDB /60 LMDB plus one existing skip; rebuilt baseline fails25 cases per engine.
Matrix details and qualifications
Node22.23.1/24.21.0/26.9.0 × pool0/2 × RocksDB/LMDB: eleven legs pass all8 tests. The remaining leg passed the strengthened limit case but timed out waiting for same-shard database rows; its targeted rerun timed out in setup. Bounded controls reproduce replication-barrier timeouts on main38897539/core9b8542 and the pinned pair, at different barriers; restoring only baseline Table.ts passes8/8. These probes call table.get, outside the changed collection filter. Retain all failed runs; no further retry or assertion change.
The initial local bd7587f/c1e192c7 matrix passed12/12 before merging Pro main. Compiled regression proof rebuilt both baseline and fix. Broad Pro CI build/unit/type/lint/integration jobs pass; companion-check awaits core#3571. Core's draft records its WebSocket mock, MQTT, design-budget and transient uWS CI qualifications. Both remain draft.
PR QA: skipped @ 1201ab2 — kept within this single-task run; author-run matrix exercises the real four-node route.
Docs: no companion; core restores local-only completeness and Pro adds regression coverage.
Refs #436.
Depends-on: harper#3571
🤖 Generated by OpenAI Codex; posted via @kriszyp.
Release target: v5.4 — main-line static-sharding regression; PR base is main.
Related PRs: #1116 overlaps, #1120 overlaps, #1086 overlaps, #1101 overlaps, #1092 independent, #1103 independent, #1112 independent, #1038 independent
Complexity: medium
Deferred findings
6b32ed31e4301e29nit test-coverage —integrationTests/cluster/staticSharding.test.mjs— Mixed-shard limit semantics remain uncharacterized — NOT FILEDOrigin — the dispatch brief this PR was written from
Fix Harper-Pro main static-sharding limit regression after #1087
Dispatch: task
harper-pro-main-red-static-sharding-20261022· queued by unknown · ran by codex/gpt-6-astra/xhigh · worker kzyp-xps-1Review-Coverage: authored=codex; ran=gemini,claude; adjudicated=domain; blocked=cursor-composer(failed); declined=cursor-grok,muse; rounds=6; full=3 @ 1201ab2
Review-Attention: read ~3m (decisions: pooled-query-contract, settlement-observation, do-less-alternative; raised: degraded review) @ 1201ab2