[kriszyp]

Fix Date chaining, retire obsolete schema accessors, and preserve write validation and live bindings across reloads.

⊙ Problem

Date chains lost timestamp getters. Re-declaration retained stale accessors and timestamp slots, exposed sealed-write bypasses, and cleared live bindings when expression compilation failed.

🤔 Bounded reload contract

Accessor retirement must preserve sealed-write admission and live bindings after a rejected expression reload; durable catalog rollback remains a follow-up.

💡 Solution

Keep Date builders complete, reconcile generated accessors with the current schema, include own fields in the validated local mutation delta, and prepare expression resolvers and derived bindings before publishing them.

⚖️ Alternatives

🤔 Validate edits rather than the merged record

Merged-record validation rejects untouched legacy data. Combine tracked and own-field edits into one validated delta when loadAsInstance !== false, preserving unsealed own-only edits; non-object incremental roots (null, primitives, arrays, binary) fail root validation with 400 rather than being silently dropped.

🤔 Failed reload scope

Precompile expressions before touching live bindings; catalog rollback (#3781) and generated resolver ownership (#3782) remain deferred. Overwrite retained getters and delete absent names.

🤔 Reload and per-write cost

Deleting retired accessors leaves prototypes in V8 fast mode (%HasFastProperties). Object.keys(this) runs once per loadAsInstance !== false tracked update (tracked in #3785); Date descriptors are built once per declaration.

🤔 Edge inputs kept as on main

A caller-sealed patch plus an own field throws, as at the updated-time stamp. An own field defined with Object.defineProperty under a timestamp name still overrides the stamp. A source-fill stub reads the live created-time attribute (#3616).

🤔 Existing Date policies

Date policies (#3584, #3713), builder serialization (#3784) and fixture cleanup (#3596) stay separate.

🔧 Changes

• resources/defineTable.ts: complete Date construction and fluent DateField types preserve every chain.
• resources/Table.ts: own-field mutation capture, root admission before delta construction, expression preparation, derived binding publication, absent-only retirement, and source-fill created-time capture. Look hardest at validation before no-change skipping and compilation before live mutation.
• resources/tracked.ts: null-prototype descriptor map and retirement of obsolete generated accessors preserve application-defined members.
• resources/DESIGN.md: enforced reload and validation contract.

Validation order

Own-key enumeration avoids the inherited schema walk. Own fields merge into the change set itself, so coercion and timestamp stamps reach the instance and a put-only source; a read-only own field that needs converting rejects the write. Both initialized and lazy updates require capture.

Table.ts:5621

write.captureChanges?.()
assertValidRecordRootBeforeCombining(recordUpdate)
if (tracksInstance) {
  ownKeys = collapseOwn ? Object.keys(this) : []
  recordUpdate = collapseChanges(this, instanceChanges, ownKeys)
  ownMerged = ownKeys.length > 0 && recordUpdate !== instanceChanges
}
if (fullUpdate || (tracksInstance ? recordUpdate !== undefined : recordUpdate && hasChanges(recordUpdate))) {
  this.validate(recordUpdate, !fullUpdate)
  if (ownMerged) applyOwnChanges(this, recordUpdate, ownKeys)
}

Product and architecture tour

Sealed tracked writes

Which edits does the commit validator inspect?

Before	After
A retired own field plus a declared edit could persist an unknown value.	Validation rejects the transaction; unrelated edits preserve untouched stored legacy fields.

Expression preparation and live publication

What remains usable when a raw expression is invalid?

Prepare before publishing

Compilation errors leave the previous live derived bindings and generated getters available.

flowchart TD
    A[Raw table declaration] --> B[Compile all expressions]
    B -->|Syntax error| C[Throw with old live bindings]
    B -->|Success| D[Assemble next resolver map and slots]
    D --> E[Publish derived bindings]
    E --> F[Overwrite retained getters]
    F --> G[Delete retired names]

Loading

Complete Date builders

What can a component declare after another modifier?

Example: Indexed creation timestamp

createdAt: types.date.indexed.createdTime retains both index and server-managed metadata. Each modifier returns a fresh complete Date builder.

Outcome: The table registers and stamps a Date; changing one chain leaves its source unchanged.

✅ Verification

• Write admission and reload recovery cover real transactions/storage; pre-fix failures reproduce on 17832a3, 01f458e and 8c92da6.
• Registration, QA-1951, tracked, schema, source-fill, caching and write/transaction suites: 161 passing on RocksDB; 112 on LMDB. Type fixtures, build, lint and TypeScript checks pass.

Full-gate limits and isolated checks

• Core: 1,646 recorded passing checks before timeout, seven deployment-test failures reported. All six named cases, including the associated cleanup, pass in isolation.
• Resources: 5,129 recorded passing checks before timeout, one source-apply snapshot-retry failure. That case and the subscription-resume case at the cutoff pass in isolation.
• Integration: timeout with one TTL clean-window premise failure and four local Ollama generation failures. The complete TTL file passes 5/5 alone, with all 10 windows exact. Ollama requests the absent llama3.2 alias (404), while llama3.2:3b exists (200); no backend code changed.
• Performance diagnostic: four paired runs against incoming 17832a3 and this head; 33 attributes, 16 rows, audit off, 500 warmup and 2,500 timed writes per mode. Lazy/initialized/own-field median times were 102.2→112.0µs (+9.6%), 161.4→172.4µs (+6.8%) and 167.5→163.6µs (−2.4%), with overlapping ranges. Separate 1,500-write heap samples changed approximately +2.3%, +1.3%, +1.0%. The modest measured cost is accepted for matching the validated and persisted delta; narrowing to empty changes would reintroduce the initialized-update bypass. This synthetic single-process RocksDB check excludes native allocation and production concurrency.
• Downstream Next.js Node24 initially timed out waiting for a cross-node cached nonce (42 passed, 1 failed, 5 not run). The targeted retry passed; Nodes20/22 passed initially. The cause was not established, and no source change was made for this observation.
• CI mapping: test:unit:main:core, :components, :server, three resource shards and all integration matrix shards cover the full gates in separate processes.

PR QA: Do tracked handles enforce sealed-field retirement across HTTP-worker reloads? — passed @ 2e73efb, qa-pr-harper-3594-2e73efb

All 14 arms and a cold rerun passed on two HTTP workers; pre-fix controls failed. Later commit 002b0cc only condenses DESIGN notes.

Docs: none needed — these repairs enforce existing schema and sealed-table contracts.

_{🤖 Generated by Claude (Opus), maintained by OpenAI Codex; posted via @kriszyp.}

Release target: v5.4 — older release lines only on demonstrated customer need
Related PRs: #3586 overlaps, #3610 overlaps, #3653 overlaps, #3655 overlaps, 3 others independent
Framing-Verdict: chosen-approach-sound
Complexity: complicated

Tracked follow-ups

Deferred findings

• f3c239815d68ff76 minor data-integrity — resources/databases.ts — Failed schema binding leaves catalog and version ahead of live bindings — filed Reject raw table declarations atomically when computed binding fails #3781
• bf10b0cc9768fd8a minor correctness — resources/defineTable.ts — Recursive enumerable field modifiers overflow serialization — filed Define finite serialization for fluent schema field builders #3784
• 8dece860c91d5dcb minor correctness — resources/Table.ts — Reload retains obsolete schema-generated computed resolvers — filed Retire schema-generated computed resolvers when an expression is removed #3782
• 11c203b4f8d371ef minor correctness — resources/Table.ts — Date primary key that is also a timestamp: getNewId returns a number and the primary-key correction overwrites the Date stamp — filed A Date primary key marked createdTime gets a numeric id and no created time #3713
• 25e65aa81a684ee4 minor correctness — resources/defineTable.ts — date.createdTime.updatedTime binds both timestamp slots to one field; created-time retention overwrites the updated stamp so it never advanc — filed A field with both timestamp flags never updates, and a table stamps only one created and one updated field #3584
• ce464ff0a127b5ba nit performance — resources/tracked.ts — retire loop scans every own prototype key per reload — filed Remove the never-instantiated Updatable class from makeTable #3585
• 9d62bb87ee1920c1 nit performance — resources/tracked.ts — collapseChanges enumerates Object.keys(target) on every loadAsInstance!==false tracked write — NOT FILED
• 6c66820b53f50fb5 nit documentation — resources/DESIGN.md — document dual-timestamp stamping next to the source-fill note once A field with both timestamp flags never updates, and a table stamps only one created and one updated field #3584 lands — NOT FILED
• b5cd68d7e588d613 nit performance — resources/Table.ts — collapseChanges enumerates Object.keys(target) on every loadAsInstance!==false tracked write — NOT FILED
• 7413c79f06cb1466 nit data-integrity — resources/Table.ts — record-root shape check skipped for context.source writes — NOT FILED

Late Grok findings

4 surviving (highest minor) — Grok reviewed cc66360, ruled against 886b844

• minor — resources/defineTable.ts:96 — date.createdTime.updatedTime sets both flags on one attribute, but a table has one created-time slot and one updated-time slot (resources/Table.ts:1538-1539). On every write after the first, the updated-time stamp at resources/Table.ts:5130 is overwritten with the original value at `resources/
• nit — unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js:63 — added comments still narrate the steps or restate test names (qa-1951…test.js:1-2, 63, 198, 262, 308, 318, 366); resources/Table.ts:9044 repeats the note at resources/DESIGN.md:289
• nit — resources/DESIGN.md:289 — the note names its enforcing test, so under the rule at DESIGN.md:1 it should shrink to one line naming that enforcement. It is still three sentences and links no issue or PR
• nit — resources/tracked.ts:225 — the retire loop scans every own prototype key on each schema declaration. This runs only at declaration time (resources/Table.ts:9037-9038), not per request, so the cost does not matter

Not standing: 1 merged, 3 resolved. Grok ran 20m (20m from launch to report); the fold adjudication (cursor-grok@cc66360be478-7c917eff, claude) took 2m.

Origin — the dispatch brief this PR was written from

defineTable (resources/defineTable.ts) Date modifier getters (.indexed, .nullable, .primaryKey) return plain makeField, dropping .createdTime/.updatedTime when chained first (types.date.indexed.createdTime crashes). Also on schema evolution: updatedAttributes() never resets hasRelationships and never deletes old accessors from structPrototype; assignTrackedAccessors (resources/tracked.ts) never deletes removed isAttribute accessors. Make every Date modifier return makeDateField and fix the DateField interface; reset hasRelationships and delete the previous resolvedAttributeNamesList names from structPrototype; in assignTrackedAccessors delete own prototype properties whose getter is marked isAttribute and are absent from the new descriptor set. Include the QA regression suite unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js (7 tests: fails 6/7 without the fix, 7/7 with it) in this PR.

Acceptance

The 7-test QA-1951 suite passes (and 6/7 fail on origin/main); existing defineTable tests green. Show each new test failing on origin/main sources (forced rebuild) before the fix.

_{Dispatch: task harper-definetable-date-modifiers-accessors · queued by finding-triage/wave-40 · ran by claude/opus/xhigh · worker kzyp-xps-1}

Late Grok findings

5 surviving (highest minor) — Grok reviewed d9ad0c5

• minor — resources/defineTable.ts:96 — date.createdTime.updatedTime (and the other orders) is accepted, but Table.ts:1688-1689 binds both timestamp slots to the same attribute. On an update, Table.ts:5491 stamps the updated time and then Table.ts:5504-5506 writes the original created time over it, so the field never changes after
• minor — resources/Table.ts:5491 — a Date field marked both primary key and timestamp (date.primaryKey.createdTime, now pinned valid at unitTests/resources/defineTable-registration.test.js:300): create({}) gets a numeric id from getNewId (Table.ts:3104 skips the UUID branch for Date, Table.ts:3202 returns a number). The
• nit — unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js:8 — Arms 1–3 repeat defineTable-registration.test.js cases (Arm 1 passes on main). Only Arm 4 adds coverage of its own.
• nit — resources/DESIGN.md:275 — two problems: (1) the timestamp-slot rebind is folded into the RecordObject-prototype bullet, where a reader looking for it won't search; (2) it cites #3594, which can't be checked because the branch has no PR yet.
• nit — resources/Table.ts:11724 — added comments speak to the diff rather than to the next reader: Table.ts:11724 ("the attribute, not a flag:") and unitTests/types/defineTable.type-test.ts:125 (a fixture label that says the field "stands alone" but not which slot wins at runtime).

Not standing: 2 merged. Grok ran 13m (13m from launch to report); the fold adjudication (cursor-grok@d9ad0c563568-92aad3f5, claude) took 1m.

Late Grok findings

1 surviving (highest nit) — Grok reviewed 91efa6a

• nit — resources/Table.ts:12472 — a source fill picks the created-time attribute once, at resources/Table.ts:11725-11726. If the table is re-declared before that fill commits, and that re-declaration moves created-time to a different attribute, then on a sharded table with secondary indexes where this node does not own the record,

Not standing: none. Grok ran 15m (15m from launch to report); the fold adjudication (cursor-grok@91efa6afcba0-84444436, claude) took 1m.

Late Grok findings

7 surviving (highest major) — Grok reviewed 17832a3

• major — resources/Table.ts:10423 — Reload deletes and recreates retained resolver getters; their lookup-performance impact remains unmeasured
• major — resources/tracked.ts:225 — After dropping a sealed-table field, assigning it alongside a declared-field edit persists an unknown property without validation
• major — resources/Table.ts:10195 — A raw table() reload whose computed-expression compilation throws before timestamp and relationship attributes leaves subsequent writes unstamped and relationship-selection checks disabled
• nit — resources/defineTable.ts:94 — Every Date modifier constructs additional descriptor objects during schema declaration
• nit — unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js:90 — Date and relationship fixtures duplicate registration-suite coverage, requiring parallel maintenance; consolidate while retaining unique tracked-update cases
• nit — unitTests/types/defineTable.type-test.ts:125 — The added comment narrates runtime timestamp behavior that this type test does not exercise
• nit — unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js:2 — The new assert import omits the required node: prefix

Not standing: none. Grok ran 15m (15m from launch to report); the fold adjudication (cursor-grok@17832a3ba55a-0fe469dd, codex) took 4m.

Late Grok findings

9 surviving (highest major) — Grok reviewed 8c92da6

• major — resources/Table.ts:5523 — Every local incremental instance commit allocates an own-key array, including writes without own fields; this path has no measurement
• major — resources/Table.ts:5526 — An incremental update with a Buffer root followed by record.extra = 1 converts the root into indexed properties before validation, allowing an invalid record mutation
• minor — resources/databases.ts:5531 — A schema-binding exception leaves persisted catalog changes and the incremented version ahead of live bindings
• minor — resources/Table.ts:10398 — Reloading a schema-generated computed field without its previous function or expression leaves the old resolver callable, producing stale values
• minor — resources/defineTable.ts:84 — Serializing a Date builder exhausts the stack through recursive enumerable modifiers; both reported serialization examples already fail on the base branch
• nit — resources/defineTable.ts:94 — Every Date modifier step installs two additional getter descriptors during schema construction
• nit — unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js:90 — Overlapping Date and accessor-retirement fixtures require parallel maintenance alongside the registration suite
• nit — unitTests/resources/qa-1951-define-table-evolution-and-date-chaining.test.js:2 — The new suite imports a Node builtin without the required node: prefix
• nit — unitTests/types/defineTable.type-test.ts:125 — The added comment describes runtime timestamp precedence beside assertions that establish only types

Not standing: none. Grok ran 23m (23m from launch to report); the fold adjudication (cursor-grok@8c92da6e48f1-9681b034, codex) took 3m.

Late Grok findings

7 surviving (highest minor) — Grok reviewed 43a5440, ruled against 525d09a

• minor — resources/Table.ts:5552 — Every local incremental instance write, including a plain Table.patch(id, { label }) with no own fields, now runs Object.keys(this). The probe measured a median of 0.539 µs on HEAD vs 0.440 µs on base for the validation block.
• minor — resources/Table.ts:5580 — An own field that carries the created-time or updated-time attribute's name is copied over the server stamp at commit, because resources/tracked.ts:490-497 copies own properties after the change set. Also, the change set is left half-merged when the update is rejected.
• minor — resources/Table.ts:5583 — An unchanged non-writable own field holding NaN fails the !== check, Reflect.set returns false, and the update is wrongly rejected with a 400.
• minor — resources/Table.ts:5557 — A sealed but not frozen patch object is reused as the change set, so adding a captured own field throws TypeError and aborts the write.
• minor — resources/Table.ts:12788 — residencyStub reads the live createdTimeProperty while the source fill pinned createdTimeToAssign (resources/Table.ts:12040-12042). If a schema reload lands between the fill and a non-resident commit, the stub is stored without the pinned created-time stamp.
• nit — resources/Table.ts:10449 — assignTrackedAccessors(Updatable, this, true) reinstalls accessors and runs the retirement scan on every reload, for a class that is never instantiated. The cost is per reload only.
• nit — resources/Table.ts:5555 — New comments at resources/Table.ts:5555-5556, :5581, :12040 and unitTests/types/defineTable.type-test.ts:125 mostly narrate the code.

Not standing: 1 separate-issue. Grok ran 15m (15m from launch to report); the fold adjudication (cursor-grok@43a5440d371c-1e418129, claude) took 1m.

_{Review-Coverage: authored=gemini; ran=cursor-composer,codex,cursor-grok,claude; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=27; full=8 @ c914e37}

_{Review-Attention: deep ~35m (critical: Table.ts, tracked.ts; decisions: readonly-own-field-rejects, v2-instance-own-props-excluded, null-incremental-root-400, applyownchanges-rederives-merged-keys, reload-atomicity-in-memory-only, do-less-alternative, release-target-vs-evidence) @ c914e37}

[gemini-code-assist]

Code Review

This pull request enhances schema evolution and re-declaration handling in Harper. It ensures that when a table schema is re-declared, stale accessors from the previous declaration (such as relationships, computed properties, and timestamps) are retired and deleted from the prototype to prevent them from shadowing newly declared fields. Additionally, it refactors types.date to support chaining modifiers like .indexed, .nullable, .createdTime, and .updatedTime in any order, and updates timestamp assignment during transactions to use the attribute object directly to avoid race conditions during schema reloads. Comprehensive unit, integration, and type tests have been added to validate these changes. There are no review comments, and I have no additional feedback to provide.