Skip to content

Operations API returns null for @relationship attributes (or rejects them): the thread serving it never loads the component schema #2296

Closed

Description

@Devin-Holland

Summary

Confirmed by a local repro on a clean, freshly-installed single node, on both Harper 5.1.15 and 5.2.4 (the latest release), with identical results — see the repro comment on this issue. No restart, no stale state, and no multi-node setup is required to hit this; both relationship directions and both get_attributes forms are affected.

The operations API cannot resolve @relationship attributes, because the thread that serves it never loads the component's GraphQL schema. Depending on what stale attribute metadata a node happens to hold, the same query either fails loudly with unknown attribute or — much worse — silently returns null for the relationship while reporting HTTP 200.

The same relationship, on the same node, at the same moment, resolves correctly over the component/REST surface. So this is specific to the operations API, not to the resolver.

Observed

Two nodes of one cluster, same version (5.2.0), same component, same records:

surface thread node A node B
component REST port http/N (loads component) resolves correctly resolves correctly
operations API main/0 (no component) unknown attribute 'location' (HTTP 400) "location": null (HTTP 200)

describe_table shows the divergence directly — node A lists 32 attributes, node B lists 34, and the delta is exactly the two @relationship attributes. On node B they carry schema-derived types ("type": "Location", "type": "array"), so that node's list came from the GraphQL schema, not from storage.

There is no operations-API form that works. The object form ({"name":"location","select":["cloudProvider"]}) skips the unknown-attribute check in validation/searchValidator.ts — so it passes validation and then returns null anyway.

A full process restart of node A did not restore the attributes; it came back with the identical 32. Before the restart, this same operations-API query had been resolving relationships correctly for months, which suggests the working state was inherited in-memory schema that a restart does not rebuild.

Mechanism

  1. components/componentLoader.ts gates component loading to non-main threads:

    if ((config.extensionModule || config.pluginModule) && (!isMainThread || config.runOnMainThread)) {

    So the main thread never runs table() against the component's GraphQL schema.

  2. resources/databases.ts deliberately never persists relationship attributes to attributesDbi:

    if (attribute.relationship || attribute.computed) {
        hasChanges = true;
        if (attribute.relationship) continue;
    }

    They are runtime-only state, present solely in a thread that loaded the schema.

  3. The operations API is served from the main thread (errors log as [main/0]). Its attribute list is rebuilt from attributesDbi, which by (2) can never contain relationships.

  4. validation/searchValidator.ts rejects any get_attributes entry absent from the table's attribute list, producing unknown attribute '<rel>'. The object/nested form skips that check, so it passes validation and yields null instead — this needs no stale state and reproduces on a fresh install. Separately, on a node whose attribute list does contain the relationship, the bare form also passes validation and returns null; see the note on the two-node divergence below.

Why this is worth a tier above its blast radius

The null path reports success. A caller cannot distinguish "this host has no location" from "this node cannot resolve relationships at all". In our case it surfaced only because a downstream consumer happened to call .get() on the null and crashed — anything that merely stores or forwards the value would have propagated wrong data indefinitely with nothing in the logs. Nothing is written to the log at load time, because the relationship is never refused; it is simply never registered.

Versions

Observed on 5.2.0 in a deployment, and reproduced from scratch locally on 5.1.15 and 5.2.4 with identical results — so the whole 5.1.x-5.2.x range is affected, latest release included. Not fixed as of current main (past 5.2.4), verified by reading:

  • the componentLoader main-thread gate — present and unchanged (introduced 2026-03-17)
  • the databases.ts relationship-persistence exclusion — present and unchanged (introduced 2025-10-10)
  • validation/searchValidator.ts — unchanged since 5.2.0, unknown attribute check still in place

No commit between 5.2.0 and current main touches relationship resolution on this path.

Repro

  1. Deploy a component whose GraphQL schema declares an @relationship attribute.
  2. Query it over the component REST port with select(rel{field}) — resolves.
  3. Query the same attribute via the operations API search_by_conditions with get_attributes: ["rel"] — returns unknown attribute, or null if a stale attribute entry survives.
  4. Restart the node and repeat step 3 — the attribute is now absent from describe_table entirely.

The question behind the bug

Is the operations API intended to resolve @relationship attributes at all, given the thread serving it never loads the component?

  • If yes, this is a straightforward bug: that surface needs access to schema-derived attributes.
  • If no, the surface should reject them consistently, so callers get a loud error rather than a silent null on whichever nodes happen to retain stale metadata.

Either resolution is fine from our side. The silent null is the part that should not survive either way.

Activity

  1. added this to the v5.2 milestone on Aug 24, 2026
  2. Devin-Holland commented on Aug 24, 2026

    @Devin-Holland
    MemberAuthor

    Confirmed by a local repro — and one correction to the issue above

    Reproduced from scratch on a clean, freshly-installed single node on both 5.1.15 and 5.2.4 (rocksdb default, no restart, no stale state, no prior schema history). The same driver script was run against each version; every probe produced identical results on both. The component below is the only thing deployed.

    This is not a 5.1-only issue — 5.2.4 is the latest release at time of writing and behaves exactly the same.

    Correction to the original report: I wrote that the silent null arises "where a stale in-memory attribute survives". That is not required. The object/nested form of get_attributes produces the silent null on a completely fresh node, because it bypasses the unknown-attribute check in validation/searchValidator.ts and then finds nothing to resolve. The silent path is therefore trivially reachable on any install, not a rare artifact of node history — which makes this materially easier to hit than the issue first implied.

    Fixture

    schema.graphql:

    type Location @table @export {
    	id: ID @primaryKey
    	cloudProvider: String @indexed
    	hosts: [Host] @relationship(to: locationId)
    }
    
    type Host @table @export {
    	id: ID @primaryKey
    	name: String @indexed
    	status: String @indexed
    	locationId: ID @indexed
    	location: Location @relationship(from: locationId)
    }

    config.yaml:

    graphqlSchema:
      files: '*.graphql'
    rest: true

    Seed: one Location (loc-1, cloudProvider: linode) and one Host (host-1, locationId: loc-1).

    Results (identical on 5.1.15 and 5.2.4)

    1. Component REST port — resolves correctly

    GET /Host/?status=RUNNING&select(id,locationId,location{cloudProvider})
    200 [{"id":"host-1","locationId":"loc-1","location":{"cloudProvider":"linode"}}]
    

    2. Operations API, bare relationship name — rejected

    {"operation":"search_by_conditions","table":"Host","get_attributes":["id","locationId","location"], ...}
    400 {"error":"unknown attribute 'location'"}

    3. Operations API, object/nested form — HTTP 200 with a silent null

    {"operation":"search_by_conditions","table":"Host","get_attributes":["id","locationId",{"name":"location","select":["cloudProvider"]}], ...}
    200 [{"id":"host-1","locationId":"loc-1","location":null}]

    4. describe_table omits every relationship attribute

    attributes: ["id","locationId","name","status"]
    

    location is absent, on a node that has never been restarted and whose component declares it.

    5. Reverse relationships fail identically — Location.hosts (@relationship(to:)) resolves over REST ("hosts":[{"id":"host-1"}]) and returns 400 unknown attribute 'hosts' over the operations API. So this is not specific to the forward from: direction.

    What this pins down

    • Not restart-dependent, not stale-state-dependent, not deployment-specific — a fresh install reproduces it.
    • Not version-specific: reproduced on 5.1.15 and 5.2.4, with the original report on 5.2.0 — so the whole 5.1.x → 5.2.x range is affected, including the newest release. Consistent with the componentLoader main-thread gate (2026-03-17) and the databases.ts relationship-persistence exclusion (2025-10-10) both being unchanged through current main.
    • Both relationship directions affected, over both get_attributes forms — one loud, one silent. There is no operations-API form that resolves a relationship.

    On the two-node divergence — unexplained, and not what I first thought

    The original report has one node rejecting the bare form and another returning null for it. I initially attributed that to stale in-memory metadata. That is now falsified: the node holding the relationship attributes was fully restarted and came back with them still present (34 attributes, location and instances intact, bare form still returning null). So it is persistent node-local state, not stale in-memory state.

    Worth noting which node is the odd one: a fresh install matches the rejecting node — no relationship attributes at all. The node that has them, with schema-derived types ("type": "Location", "type": "array"), is the anomaly, and I can't currently account for how its operations-API thread acquired them.

    This does not affect the core defect, which reproduces from a clean install on both versions above and needs no divergence to demonstrate.

  3. Devin-Holland commented on Aug 24, 2026

    @Devin-Holland
    MemberAuthor

    Follow-up on the two-node divergence: thread topology ruled out

    Two further results narrow where the divergence comes from, and both contradict what I guessed earlier.

    1. Both nodes serve the operations API from the same thread. The node that returns null never errors on the relationship, so it had logged nothing identifying its thread. Forcing an error with a deliberately bogus attribute settles it:

    [main/0] [error]: unknown attribute 'zzznotarealattribute'
    

    Same main/0 as the node that rejects the relationship. So this is not one node serving the operations API from a component-loading worker — thread topology is identical on both.

    2. The state survives a full process restart. The node holding the relationship attributes was fully restarted and came back unchanged — 34 attributes, location and instances still present, bare form still returning null. It did not converge on the other node's shape.

    What that leaves

    The two nodes differ only in the contents of the main thread's attribute list, and that difference is durable across a restart — so it is persisted state, not in-memory drift.

    Worth noting how the attributes present themselves on that node:

    {"attribute":"instances","type":"array"}
    {"attribute":"location","type":"Location"}
    {"attribute":"locationId","type":"ID","indexed":{}}

    The two relationship attributes carry no indexed marker, unlike the real FK beside them. Since non-indexed attributes are persisted to attributesDbi — that is what RE-7 (#1183) was about — the most likely place to look is persisted attribute records in that node's local store, rather than anything in the component-loading path [speculative; I have not read that node's attributesDbi].

    A fresh install has no such records and matches the rejecting node, so the node that holds them is the anomaly, not the norm.

    None of this affects the core defect. It reproduces from a clean install on 5.1.15 and 5.2.4 with no divergence involved. I'm recording it only so the two-node observation in the original report isn't mistaken for the mechanism.

  4. self-assigned this
    on Aug 24, 2026
  5. kriszyp commented on Aug 26, 2026

    @kriszyp
    Member

    Related QA report:
    SQL SELECT omits @computed scalars (subtotal=undefined) while an explicit column list surfaces them correctly (subtotal=10), and REST's default no-select GET plus GraphQL both work — so the wildcard path was missed by #1484/#1601 (c899a82), whose stated scope was "default reads across REST/GraphQL/SQL". Two days post-merge is the cheapest moment to close it. Distinct code path from F-190, which drops dynamic* attributes on schema-less tables; these two only share a symptom surface.

  6. added a commit that references this issue on Aug 27, 2026
    cab72e5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

P1

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions