Skip to content

Grant the primary key and keep protected attributes for attribute-restricted roles on REST, GraphQL and MQTT - #3297

Merged
kriszyp merged 21 commits into
mainfrom
fix/resource-path-translated-permissions
Nov 4, 2026
Merged

kriszyp merged 21 commits into
mainfrom
fix/resource-path-translated-permissions

Conversation

@kriszyp

@kriszyp kriszyp commented Oct 19, 2026 •

Copy link
Copy Markdown
Member
A worker presses a green shield button

⊙ Problem

For attribute-restricted roles, the resource path omits an unlisted primary key and a full PUT or retained MQTT publish erases stored attributes the role cannot update. This fixes Resource path authorizes against untranslated role permissions: no primary-key auto-grant, and a full put by an attribute-restricted role erases unlisted attributes.

🤔 Retained replacement policy and limits

Preserving protected attributes is the agreed policy; rejecting every restricted replacement would break retained MQTT use. Array bodies reaching a single Table authorization check are refused. Existing PATCH limits remain: operation-shaped values are operations, and put-only sources with cold or stale caches can lose protected values. Delete/retry can create partial rows.

💡 Solution

Super users, unrestricted replacements and the operations API retain existing rules.

⚖️ Alternatives

  • Full operations translation drops undeclared properties that open tables can store.
  • Copying protected values into payloads exposes them to messages and writes stale bases into logs.
  • Keeping older hooks duplicates main’s body-resolution and permission helpers.

Framing-Verdict: chosen-approach-sound

🤔 Direct instance PATCH/create inputs

Custom thenables can be checked after resolution but written from their original fields. This reproduces on main. Direct callers must supply records; protocol dispatch resolves bodies first. Separate hardening is needed.

🔧 Changes

  • resources/Table.ts: permission lookup adds the cached implicit grant; schema refresh invalidates it. allowUpdate keeps the checked map on the receiver. Legacy full update and put stage restricted patches; staging retains main’s pending-write/error handling. Source write-through selects implemented patch or materialized put for restricted writes. The replacement helper rejects invalid roots before staging. Validation requires a complete record when creating a missing row.
  • security/permissionsTranslator.ts: implicitPrimaryKeyPermission uses the existing grant/timestamp primitives and last-entry rule. Operations-API translation is unchanged.
  • security/DESIGN.md records the invariant, lifetime, source limits and untested commit/replay/replication paths; DESIGN.md indexes it. Existing index summaries and the lockdown note were shortened to stay within their byte ceilings, retaining the sealing performance constraint and enforcement pointers.
  • resources/MAP.md points resource authors to the cross-layer permission invariant.
  • Resource permission tests cover key denial, timestamps, sealed/undeclared fields, tracked payloads, override retries and audit entries. New owned-database cases exercise source method selection, hook rejection and invalid direct roots in both instance modes. Own-key assertions pin the accepted omission representation. The delivery deadline is cancellable, the override test follows main’s resolved-body contract, and direct-root checks assert specific failures and unchanged storage.
  • Precondition regression test verifies discarded restricted updates report stale-condition failures through save without unhandled rejections.
  • Protocol integration test and its schema / config cover REST, GraphQL and MQTT with unrestricted and operations-API controls. The then-field control verifies unrestricted body resolution and persistence through the operations API.

🤔 Rechecks and documentation ownership

A second denied or pending allowUpdate clears the instance restriction. This retained lifetime policy requires trusted overrides to honor failed or unfinished rechecks. The cross-layer invariant stays in security’s design note, linked from the resource map, to preserve both directories’ existing byte budgets. Compression retains the existing enforcement rationale; restoring it requires different pruning.

🤔 Navigation and event deadlines

The index links detailed exceptions rather than repeating them. The subscription test waits for its completion event with a cancellable deadline; polling retains the same timeout. Legacy PUT signatures and their existing shift comment stay unchanged. One older non-record test accepts any rejection; newer direct-root cases assert specific statuses.

🤔 Accepted writable-key representation

Omitted writable fields keep own enumerable keys valued undefined, matching PATCH; JSON omits them while JavaScript and MessagePack retain them. This compatibility difference from unrestricted PUT is accepted for this maintenance rebase. Own-key assertions cover both engines and instance modes. Physical deletion needs a separate change coordinated across commit, transaction logs, replay and replication.

🤔 Serializer evidence scope

Primary-store checks establish the observed table shape. The shared-codec round-trip establishes MessagePack serialization of that shape, not table-encoder persistence. Restart, replay and replication remain explicit coverage gaps.

🤔 Existing denial return contract

Direct allowUpdate can return undefined rather than literal false when update permission is absent. Main has the same behavior; dispatch rejects falsy verdicts. An explicit-boolean cleanup remains separate from this maintenance work.

✅ Verification

  • Build and all static checks pass.
  • Permission suite: 121 passing per source/compiled × RocksDB/LMDB run.
  • CI: all checks passed on 69435ff93ea7.
Full gate results and base comparisons

Before the accepted-shape test/doc change, eight complete files passed with 339 RocksDB (1 pending) and 278 LMDB (20 pending); protocol integration passed 10/10. Production modules are unchanged.

Base: df882a311be6. The final base refresh preserves all 19 branch patches exactly. Full gate results below tested 7bd0cb707a50 on base e58d105ee3f6; current focused checks appear above.

Check Observed result
Build, types, lint, full formatter, design budgets, resource partition Pass
Focused permissions/request-body/preconditions, RocksDB 161 passing
Same focused selection, LMDB 146 passing; 15 skipped
REST/GraphQL/MQTT integration 10 passing
Main core CI group 5317 passing (4m); 172 pending
Main components CI group 2434 passing (17m); 7 pending; 4 failing
Main server CI group 2404 passing (6m); 13 pending; 2 failing
Full resources, RocksDB 6581 passing (21m); 243 pending; 1 failing
Full resources, LMDB 4971 passing (7m); 1393 pending
Full integration, concurrency 2 ℹ tests 3366; ℹ pass 3342; ℹ fail 4; ℹ cancelled 0; ℹ skipped 18

Mocha verified the three main CI groups partition all 485 files exactly. The monolithic main command hit its 30-minute deadline and is not reported as passing. The two installation-guard failures reproduce identically on exact main production code. Full-scope component retry/probe failures also occur on the base, with variable watcher/recovery cases; focused comparisons pass. The full-scope origin-floor log-key equality failure reproduces at the same assertion on exact main; 23 new permission cases fail there and pass on the branch. These unrelated tests were not changed. The four real-Ollama integration failures reproduce in isolation; direct requests return 404 because the generative model is missing. No model or service was changed.

Negative controls: the tested base’s production modules make 5 protocol cases fail while all 5 controls pass. Removing only the recovered staging fix makes the new precondition test fail on unhandled 412 rejections. Every temporary base build is restored byte-for-byte afterward.

The related-PR scan is partial: open candidates were checked, but merged history beyond its first 50 entries was not scanned. Candidate patches were compared separately for permission, source and publish interactions.

The stacked Apply attribute read permissions to subscription and MQTT deliveries remains untouched and needs its separate rebase after this parent settles.

Docs: this maintenance rebase adds no API or configuration.

🤖 Generated by OpenAI Codex; posted via @kriszyp.

Release target: v5.4 — existing PR milestone; task base remains main

Related PRs: #3485 overlaps (Compartment mode: enforce allowedDirectory on plain file imports and support named imports from dynamic CommonJS packages), #3126 overlaps (Decode multipart forms for REST resources and stream uploaded Blobs), #3316 overlaps (Apply related-table role permissions to @relationship reads), #3229 overlaps (HTTP preconditions with optional locks), #3307 overlaps (Apply attribute read permissions to subscription and MQTT deliveries), 4 others independent

Complexity: complicated

Deferred findings

  • a296b96988cfe551 major correctness — resources/Table.ts — Restricted full PUT retains omitted writable keys as own undefined properties — NOT FILED
  • c7f3f26b28bcb0a8 major security — resources/Table.ts — false-mode PUT/PATCH with no body skips allowUpdate and writes RequestTarget fields as the record — NOT FILED
  • 1c06d86d6598a09d minor data-integrity — resources/Table.ts — full-vs-patch validation decided once; retry after delete or LMDB same-transaction create — NOT FILED
  • 3c7dab65a3e9c048 minor security — resources/Table.ts — REST PUT creating a record is authorized by update permission, not insert — NOT FILED
  • cc9ffdaded7d17ef minor correctness — resources/Table.ts — restricted PUT applies {op}-shaped values as CRDT operations — NOT FILED
  • 075082de79d55b0e minor security — resources/Table.ts — Denied or pending authorization clears the prior instance restriction — NOT FILED
  • f2433a9c31837cdb minor correctness — resources/Table.ts — Explicit id-less PUT targets lack a body-primary-key fallback — NOT FILED
  • df355c4d6f7b31e8 minor data-integrity — resources/Table.ts — Cold or stale cache loses protected values through source PUT fallback — NOT FILED
  • d6cab136a64b609d nit security — resources/Table.ts — restricted array PUT refused outright; numeric-attribute-name bypass — NOT FILED
  • de2e7130035a0fac nit correctness — resources/Resource.ts — function-valued then field on a record body treated as thenable — NOT FILED
  • +13 more in the review receipt

🤔 defer this PR's major correctness finding a296b96988cfe551 (Restricted full PUT retains omitted writable keys as own undefined properties) to a separate issue instead of fixing it here?

Origin — the dispatch brief this PR was written from

Fix harper#3284: the resource path (REST, GraphQL, MQTT) authorizes against the untranslated role permission, so the documented primary-key auto-grant is missing and a full put by an attribute-restricted role erases attributes the role cannot access. Apply translated permissions (permissionsTranslator.getRolePermissions) on the resource path via getTablePermissions or userView. Full-put policy: PRESERVE every attribute the role cannot write (keeps MQTT retained publish usable for restricted roles); if you find a reason refusal is required instead, ask before implementing.

Acceptance

Integration tests on REST and MQTT for a role with attribute_permissions [title, owner]: REST/GraphQL reads include id; MQTT publish carrying id is accepted; full PUT and retained MQTT publish without secret leave the stored secret unchanged; a payload that sets an attribute the role cannot write is still refused; super user and ops API unchanged. Shown failing on origin/main.

Dispatch: task harper-3284-resource-path-translated-permissions · queued by finding-triage/wave-36 · ran by claude/opus/high · worker kzyp-xps-1

Late Grok findings

1 surviving (highest minor), 7 previously adjudicated — Grok reviewed 7bd0cb7
  • major — resources/Table.ts:12041 — rejected bodies receive an allowed authorization verdict (previously adjudicated)
  • major — resources/Table.ts:4497 — restricted PUT selects an unimplemented source PATCH (previously adjudicated)
  • minor — resources/Table.ts:5702 — direct thenables fall through to full replacement
  • minor — security/DESIGN.md:308 — the Table permission invariant remains outside its owning directory (previously adjudicated)
  • minor — unitTests/resources/permissions.test.js:723 — fixture tables remain in the configured test database (previously adjudicated)
  • nit — security/DESIGN.md:298 — unrelated documentation compression broadens review scope (previously adjudicated)
  • nit — integrationTests/security/resource-attribute-permissions.test.ts:243 — the then-field test bypasses restricted replacement (previously adjudicated)
  • nit — unitTests/resources/permissions.test.js:1174 — publish delivery retains a five-second deadline (previously adjudicated)

Not standing: none. Grok ran 21m (22m from launch to report); the fold adjudication (cursor-grok@7bd0cb707a50-0437d1df, codex) took 3m.

Late Grok findings

none surviving, 7 previously adjudicated — Grok reviewed 7ee00fc
  • nit — security/DESIGN.md:308 — the Table permission note remains outside its owning directory (previously adjudicated)
  • nit — security/DESIGN.md:298 — unrelated security prose remains rewritten (previously adjudicated)
  • nit — integrationTests/security/resource-attribute-permissions.test.ts:244 — the HTTP fixture proves acceptance without checking persistence (previously adjudicated)
  • nit — unitTests/resources/permissions.test.js:1413 — the five-second delivery deadline remains disputed (previously adjudicated)
  • nit — security/DESIGN.md:312 — invalid-root wording leaves the full-replacement boundary implicit (previously adjudicated)
  • nit — DESIGN.md:154 — the index omits the note's protected-value exceptions (previously adjudicated)
  • nit — resources/Table.ts:5681 — the dual PUT signatures could use additional explanation (previously adjudicated)

Not standing: 1 accepted-known. Grok ran 23m (23m from launch to report); the fold adjudication (cursor-grok@7ee00fc0e22f-0676edf6, codex) took 2m.

Late Grok findings

1 surviving (highest major), 1 previously adjudicated — Grok reviewed 702d08c
  • major — resources/Table.ts:12157 — restricted full PUT retains omitted writable keys as own properties
  • nit — unitTests/resources/permissions.test.js:1223 — non-record-root assertions accept unrelated exceptions (previously adjudicated)

Not standing: none. Grok ran 15m (16m from launch to report); the fold adjudication (cursor-grok@702d08cbc15f-c0fe6a1e, codex) took 2m.

Review-Coverage: authored=codex; ran=muse,gemini,cursor-composer,cursor-muse,cursor-grok,claude; late=cursor-grok:adjudicated; adjudicated=domain; declined=cursor-kimi; rounds=25; full=13 @ 69435ff

Review-Attention: deep ~45m (critical: Table.ts; sensitive: permissionsTranslator.ts; decisions: do-less-alternative, replacement-shape, restriction-lifetime, deferred-a296b969) @ 69435ff

@kriszyp kriszyp added this to the v5.4 milestone Oct 19, 2026
gemini-code-assist[bot]

This comment was marked as resolved.

kriszyp and others added 19 commits November 3, 2026 23:43
… resource-path full puts

The resource path (REST, GraphQL, MQTT) authorized attribute-restricted roles against the stored
role as written. An unlisted primary key got no access, so reads omitted `id` and an MQTT publish
carrying it was refused; and a full put (REST PUT, retained MQTT publish) erased every attribute
the role does not list.

- getTablePermissions grants an unlisted primary key the translator's implicit access
  (implicitPrimaryKeyPermission, shared with permissionsTranslator), memoized per stored entry.
- allowUpdate no longer writes stored values into the payload it authorizes; it records the
  role's update map, and a full put by such a role is staged as a patch: the payload, the key,
  and a removal of each omitted updatable attribute. Its log entry, replay, replication and
  source write therefore leave protected attributes as stored. With no base it validates as a
  full record.
- PUT bodies are resolved before allow* runs, as QUERY bodies already were, so the attribute
  check sees the real keys (closes the PUT half of #1434).

Fixes #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
- Refuse an array body for attribute-restricted roles in allowCreate/allowUpdate: its keys are
  indices, and its elements are later written without a check of their own.
- Clear the instance's update map on every write, so it authorizes only the write it was
  checked for.
- Never generate a removal for the created/updated time attributes or for an inherited
  Object.prototype name.
- Scope the Bun/Windows skip to the MQTT tests, and bound the test waits.
- DESIGN.md: state the put-only source and racing-delete limits.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…ep created time on restricted puts

- PATCH and POST bodies now reach allowUpdate/allowCreate resolved, as PUT and QUERY bodies do, so
  an attribute-restricted role can no longer write a protected attribute through them (#1434).
- A restricted full put drops supplied created/updated times from its patch, so the stored created
  time survives as it does on an unrestricted put.
- The update map is cleared by the patch and publish paths only, so an override's own write
  (lock(), addTo()) before super.put no longer drops it.

Refs #3284, #1434

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…put; no key grant on keyless tables

- The update map recorded by allowUpdate now stays on the request's instance until a full put takes
  it, so an override's own update() before super.put can no longer turn the put into an
  unrestricted replace. A later full put in the same request is restricted too.
- A keyless table gets no implicit primary-key entry (it named an `undefined` attribute, which broke
  restricted reads and admitted a body key "undefined").
- publish bodies are also resolved before authorization.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…es with a then field

- A full put takes the update map only when its root is a record, so a non-record write cannot
  consume it unapplied; omitted attributes are detected by own keys.
- On a sealed table, no removal is generated for a listed attribute the schema does not declare.
- A resolved write body is run as data even when it has a non-function `then`, which a REST
  PUT/PATCH body could carry once bodies are resolved before authorization.
- The body-first method check is computed once per wrapper, not per call.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…materialize tracked payloads

- allowUpdate's update map is no longer taken by the first full put: it restricts every full put on
  the request's instance, so a failed and retried super.put cannot become an unrestricted replace.
- A tracked payload is materialized before the replacement patch is built, so values behind
  getRecord() are written rather than removed.
- Sealed-table membership is a Set built once per restricted put.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…ed attribute names

- allowUpdate/allowCreate check the keys of a tracked payload's materialized record, the same
  record the restricted replacement writes.
- Declared attribute names are kept per table, refreshed in updatedAttributes(), instead of being
  built on each restricted put to a sealed table.
- Removal markers are own data properties; trimmed comments that restated the code.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
A JSON body or MQTT message with a `getRecord` field was taken for a tracked object, so a role
allowed to write that field got a 403. Also trims comments that restated the code.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
… update bodies too

- A promised body re-dispatched without a context now carries the ambient context explicitly, so
  a falsy body is not re-read as the (id, context) form.
- The static `update` entry also resolves a promised body before authorization.
- The replacement patch defines the primary key as an own property.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…eeping the public contract

The dispatcher again passes allow* the body as given (a promise for REST), as their public
signature declares, so an override that calls body.then() keeps working. Table's own allowUpdate and
allowCreate resolve a promised body before checking its keys. Resource.ts is back to main.

Also: on a sealed table, only declared attributes count toward the implicit primary-key grant, and
that memo is reset when the schema changes.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…ep a body's parse error

- After resolving a promised body, Table's allowUpdate/allowCreate continue with Table's own method,
  so a subclass override that calls super is not run a second time.
- A body that fails to resolve passes the hook and fails the write with its own error, rather than
  being reported as an authorization failure.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
A thenable is not a record root, so a promise handed straight to put/update by app code takes the
same path it does on main (the write fails) instead of becoming a patch that removes every
writable attribute.

Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
Refs #3284

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: harper-3284-resource-path-translated-permissions
…/DESIGN.md

resources/DESIGN.md is at its line budget on main, so the note moves to the directory that owns
permissions, indexed from the root.

Refs #3284

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
They return a promise when the body is a promise, so the declarations say so, and the callers that
await the result are typed to match.

Refs #3284

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ED1ufN1nEZ76qGD8BRKYEX
Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
…tion helpers

Preserve translated key semantics and restricted replacement patches while reusing the current body-resolution helpers. Cover readonly-field races and conditional staging, and update tests for resolved request bodies.

Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Keep invalid roots from falling back to full replacements while an attribute
permission map is installed. Use the existing source put fallback when an
inherited patch has no implementation, and propagate direct body rejections.
Cover source shapes and invalid inputs with real-module tests; isolate the
new fixtures and document the remaining source and direct-caller limits.

Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Cover body-only ids and legacy full updates in both instance modes, and check
then-field persistence through the operations API. Qualify the design index
and record the existing trusted-caller input boundary explicitly.

Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
@kriszyp
kriszyp force-pushed the fix/resource-path-translated-permissions branch from 75e734c to 702d08c Compare November 4, 2026 07:34
kriszyp and others added 2 commits November 4, 2026 05:51
Assert own undefined keys in both instance modes and their MessagePack
representation, retaining JSON and protected-field checks. Document the
accepted PATCH representation without changing runtime deletion semantics.

Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Use the shared package binding so the permissions suite stays in the
type-strip lane and uses the same codec entry as the runtime.

Dispatch-Task: pr-maint-1739871cdd61c444abd7c428c452d4e3
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
@kriszyp
kriszyp marked this pull request as ready for review November 4, 2026 14:32
@kriszyp
kriszyp merged commit 056c4ac into main Nov 4, 2026
75 checks passed
@kriszyp
kriszyp deleted the fix/resource-path-translated-permissions branch November 4, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant