Skip to content

feat(ir)!: identify operation groups and declare ID namespaces - #801

Merged
OmarAlJarrah merged 19 commits into
mainfrom
feat/ir-operation-group-id
Oct 9, 2026
Merged

OmarAlJarrah merged 19 commits into
mainfrom
feat/ir-operation-group-id

Conversation

@OmarAlJarrah

@OmarAlJarrah OmarAlJarrah commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Two gaps in how the IR names things, closed together because both ride the same schema generation.

Operation groups had no identity. emitter-design.md §3.4 keys sub-clients by a GroupKey "derived from OperationGroup identity", but ir.OperationGroup serialized as a name, docs and operations. A sub-client could therefore be keyed only by its name, which §15 rules out for anything that has to survive a rename, and which two groups can share:

tags: [{name: default}]
paths:
  /tagged:   {get: {operationId: tagged,   tags: [default], responses: {"200": {description: ok}}}}
  /untagged: {get: {operationId: untagged,                   responses: {"200": {description: ok}}}}

This compiled to a group named {source: default} and a fallback group named {hint: default}: two groups that render the same, told apart only by the naming channel.

A lost separator in an ID was invisible for most classes. An ID is kind/namespace/path, and only its first two separators say where the namespace ends, so a path glued onto its namespace is a well-formed ID in a namespace nobody meant (op/openapipaths/~1pets/get). irverify could see that only where a node records the pointer its ID was derived from. An operation, a service and a group do not, so it verified clean.

Closes #673. Closes #802.

Change

Group identity

  • ir.GroupID is a new ID class (kind prefix g), and OperationGroup gains ID. It is unique across the document, nested groups and every service included.

  • The OpenAPI compiler mints it from the rule that formed the group, in a namespace named for the rule:

    Group ID
    first tag of the operation g/tags/<tag name>
    first path segment (path-prefix strategy) g/path-prefix/<segment>
    untagged operations g/default
    webhook operations g/webhooks

    The tag name and segment are RFC 6901 escaped, so a/b and a~1b stay distinct. The accumulator that gathers operations into groups keys by this ID and stamps it, so a group's key and its identity are one value.

  • emitter-design.md uses ir.GroupID where it had GroupKey, with the two optional references (GroupPlan.Parent, ShapeHint.Remount) spelled as nil pointers, as §3.1 requires of a reference that may be omitted.

Declared namespaces

  • Document gains IDSpaces: the namespaces its IDs live in, keyed by kind prefix, stated by the producer as it stamps IRVersion.
  • irverify reports an ID whose namespace the document does not declare for its kind (ir/id-space-undeclared), whatever the class and whether or not its node records a pointer. The primitive namespace is ir's and needs no entry. A document declaring nothing while carrying IDs is reported once (ir/id-spaces-absent), as a missing irVersion is. The declaration must itself be usable: keyed by real kinds, a kind with no namespace left out, each list sorted by the byte order of its UTF-8 spelling and without repeats, no namespace empty or carrying the separator (ir/id-spaces-unknown-kind, ir/id-spaces-not-canonical, ir/id-space-invalid).
  • It is a vocabulary and not a usage report, so a namespace no ID uses is not an error in a document. The compiler's own test holds its list to what its corpus uses, in both directions.
  • compile.Namespaces builds the declaration from the Space constants a compiler already derives IDs with. The OpenAPI compiler names its namespaces once and stamps them beside IRVersion.
  • The namespaces a document's IDs use are derived in one test helper, irtest.SpacesUsed, which irverify's tests and the compiler's vocabulary test share.
  • The check reads the declarations irverify already collects, so it adds no walk.

The rest of the verifier

  • Operation, service, group and property IDs are now held to the ID grammar (well-formed). Properties are also held to agreement with their recorded pointer, as types and security schemes already were. Operations are not, because their provenance records where the body is declared and the ID where it is mounted, and the two differ for an operation reached through a $ref'd path item (openapi: referenced non-schema components hoist once per use site under fabricated pointers #107); holding them to it fails component-reuse.yaml and operation_id_remounts.yaml.
  • All of that lives in checkDeclaredIDs, which now owns a node's own ID in one place (present, well-formed, in agreement), with one table mapping every ID class to its prefix.
  • The empty-reference classification of a node's own ID (idElsewhere) had a case for Operation.ID only. Service.ID and Property.ID had no guard at all, and OperationGroup.ID had one only because shared fixtures happened to carry an ID-less group. Each now has its own case.
  • Comments that listed the classes with no registry map (operations and services) include groups.

The architecture rule

A compile.Space may only be written as a string literal, because a namespace assembled from data can carry the path in with it. The rule also resolves each file's own imports, so an aliased import cannot hide a violation and an unrelated package called compile cannot cause one. It is syntactic, so two constants glued together are not seen by it; the declaration sees that as an undeclared namespace.

Docs

ir-design.md records the Document member, the g prefix, the derivation rule, what irverify holds IDs to and why, and the 0.7.0 history entry. architecture.md lists the new enforced rule.

Design choices

An ID, not a "name path". A name-derived key moves when a group is renamed, and ShapeHint.Remount is authored alongside the SDK and keyed by IR ID, so it would break on exactly the edit it exists to survive. Names stay presentation (invariant 3).

Not derived from the Tag Object. A Tag Object is metadata about a name, not the group: an operation may use a tag no Tag Object declares, a document may declare the same tag twice, and /tags/N shifts when the list is reordered. The grouping rule mints the group, so each rule gets a namespace of its own, which also keeps g/default and g/webhooks from colliding with a tag called default or webhooks. Name-keyed IDs extend without change to nested groups, such as the 3.2 tag parent chains in #613 and the dotted group keys in #582.

A per-kind declaration, required like irVersion. A flat set could not tell a namespace some other kind declares from one this kind may use. Making the declaration optional would let a producer that forgot opt out silently, the failure the version stamp's policy exists to avoid.

Now, not later. This PR already moves the schema to 0.7.0, so the declaration costs no further generation. Adding it afterwards would cost every consumer another break for a hole that is cheap to close here.

Alternatives rejected (#802 has the full list): recording the derivation pointer on Provenance (format-specific, widens every Provenance, and does nothing for services or groups); an unambiguous delimiter between namespace and path (rewrites every ID in every document); a prefix heuristic (false positives).

Out of scope, stated in groupFor

The issue also asks that the fallback group's name not collide with a declared one. That part is not done: the fallback keeps the hint default. Any hint can equal some tag's name, so renaming it only moves the collision. Emitters resolve name collisions by ID (emitter-design.md §4.12), which this change makes possible for groups. Telling a document author that two groups share a name is #647.

What catches a lost separator

Defect Caught by
A group, operation or other ID built by converting a string the architecture test
A namespace assembled from data (compile.Space("tags"+name)) the architecture test, and the declaration
Two constants glued together the declaration
A producer outside this repository the declaration
A property or type ID whose path disagrees with its pointer agreement

Planted before this change, the second and third both passed the architecture test, irverify and every harness oracle. Both now fail TestVerify_Corpus and the other corpus oracles.

Test plan

  • On the branch head with main (build: move to Go 1.27.2, golangci-lint v2.14.0, openapi v1.25.5 #804) merged in: gofmt, go vet, golangci-lint v2.14.0 and the whole suite, every package at 100% statement coverage. The race-detector coverage run, the fuzz search and the bench smoke run are the CI gate's.
  • The harness over the whole testdata tree gives every spec the same outcome as main, the deliberately broken negative fixtures included. The only difference is the new fixture.
  • A conformance case (group-identity) writes the group IDs out for every rule, including a declared tag default, a tag webhooks, an undeclared tag, and a/b against a~1b. Unit tests cover the path-prefix strategy, a tag declared twice, a two-order diff of one document with paths and tags reversed, and an exhaustive check that tag and path-prefix IDs never collide over every string of up to four characters from the alphabet RFC 6901 escapes.
  • The namespace check is tried on all six ID classes: a lost separator, a namespace declared for another kind only, the primitive namespace, a forgotten declaration (reported once), a malformed ID (not reported twice), a declared-but-unused namespace (clean), each way a declaration can be unusable (a kind with an empty or null list included), and the order a list is sorted in, with a pair of namespaces that UTF-8 and UTF-16 order differently.
  • The compiler's vocabulary is held to the namespaces the corpus uses, in both directions and under both grouping strategies; a path-prefix namespace is invisible to the default-strategy tests.
  • Every defect below was planted, compiled, and turned at least one named test red before being reverted. Group IDs: escaping / but not ~, the fallback group taking a tag's ID, the accumulator not stamping the ID, path-prefix groups using the tag rule, path-prefix segments left unescaped, the webhook group taking a tag's ID, IDs derived from first-seen position. Verifier: swapped prefixes, a class dropped from the table, empty IDs not skipped, the pointer never read, operations held to agreement (which fails the real corpus), properties not held to it, each own-ID classification wrong. Namespace check: one flat set instead of one per kind, the primitive namespace not exempt, the primitive exemption widened beyond types, the forgotten declaration reported per ID, and the check not wired in. Declaration: an unknown kind also judged as a kind, a kind with no namespace not reported, a UTF-16 comparison in place of byte order, and a class dropped from, or sharing a kind in, the prefix table. Vocabulary: a declared namespace nothing uses, a used one undeclared, the path-prefix one dropped, the stamp removed. Shared helper: the primitive namespace not left out, repeats not removed, no sort, no kind filter, malformed IDs not skipped. Architecture rule: each clause of the namespace rule and of the import resolution.
  • Pre-existing goldens change only in the irVersion line, the added group id lines and the new idSpaces block, and regenerating them changes nothing further. One golden is new, for group-identity.
  • Cost: Verify makes about a thousand more allocations than on main (3,160 to 4,235 on enum-string.yaml, a small conformance spec; 38,511 to 39,600 on param-style-matrix.yaml, the largest committed one), because every reflection walk visits the new member. It is a per-document cost that does not follow the number of IDs.

Breaking

IRVersion moves to 0.7.0. Every group gains an id key and every document gains idSpaces, so a consumer pinned to 0.6.0 must be rebuilt against this generation; there is no migration path (ir-design.md §2.1).

#734 and #735 also set IRVersion to 0.7.0 from the same base. Whichever of the three merges later must merge main in, take the next number, re-anchor the three version literals in ir/document_test.go, add its history entry under the new number, and regenerate the goldens.

emitter-design §3.4 keys sub-clients by a GroupKey "derived from
OperationGroup identity", but an OperationGroup carried none: it
serialized as a name, docs and operations. A sub-client could then be
keyed only by its name, which the emitter design rules out for anything
that must survive a rename, and which two groups can share. A declared
tag called "default" and the group untagged operations fall into render
from the same words.

OperationGroup gains ID (a new GroupID class, kind prefix g). The
OpenAPI compiler mints it from the rule that formed the group, in a
namespace named for that rule, because no OpenAPI construct is a group:
a Tag Object is metadata about a name and an operation may use a tag no
Tag Object declares.

    g/tags/<tag name>             first tag of the operation
    g/path-prefix/<segment>       first path segment
    g/default                     untagged operations
    g/webhooks                    webhook operations

The tag name and segment go through RFC 6901 escaping, so "a/b" and
"a~1b" stay distinct. The group accumulator now keys by this ID, which
makes the key and the identity one value. irverify already reads ID
classes by reflection, so an absent or repeated group ID is reported as
ir/empty-group-id and ir/duplicate-group-id; its class tables list the
new type. emitter-design now uses ir.GroupID where it had GroupKey.

The fallback group keeps the hint "default" beside a tag of that name.
Any hint can equal some tag's name, so renaming only moves the
collision; emitters resolve name collisions by ID (emitter-design
§4.12).

BREAKING CHANGE: IRVersion moves to 0.7.0. Every group in every
compiled document gains an id key, so a consumer pinned to 0.6.0 must
be rebuilt against this generation; there is no migration path
(ir-design §2.1). Goldens change only in the irVersion line and the
added group IDs.
checkIDs holds a type's and a security scheme's ID to the ID grammar:
well-formed, and carrying the pointer the node records. It reaches them
through their registries. An operation, a service, a group and a
property live in the service tree or a model, so none of them was
checked: each could carry any string and still verify. A property ID
that lost the separator between its namespace and its path, the shape
of #141, passed.

checkDeclaredIDShapes walks the document and holds each of those four
classes to its kind prefix, reusing the violation checkIDs reports so
the two cannot disagree on wording. Properties are also held to
agreement with their recorded pointer: both construction sites derive
the ID and the provenance from one pointer, and the merge of a
redeclared property keeps the two together.

Operations are held to shape only. An operation's provenance records
where its body is declared and its ID where it is mounted, and they
differ for an operation reached through a $ref'd path item or callback
(#107); holding operations to agreement fails component-reuse.yaml and
operation_id_remounts.yaml in the corpus. A service records no pointer
and a group no provenance. A lost separator in an operation, service or
group ID therefore stays invisible, which ir-design §3.1 now says.

Empty IDs stay with checkDeclaredIDs and are skipped here, so one
defect gets one report. Channels and messages have no kind prefix yet
and are not held.

Four test fixtures spelled an operation ID "o/x/S/op", which no
compiler could mint; they now use the op prefix.
idPositions classes every ID-typed field, and the entry for a node's own
ID must be idElsewhere, or an empty one is reported twice: once by the
rule that owns it and once as an empty reference. Only Operation.ID had
a case in TestVerify_EmptyIDsAnotherCheckOwnsAreNotReportedTwice.
Setting OperationGroup.ID, Service.ID or Property.ID to idRequired
failed no test for the last two, and failed the group's only because the
shared fixtures happened to carry a group with no ID.

Give each of the three a case that plants the empty ID and asserts the
owner's code. Each misclassification now fails exactly its own case.

With the guard explicit, the fixtures can stop depending on that noise.
The ones that reach the full Verify now carry a valid group ID, so
they are valid documents apart from the defect under test. Fixtures
that run a single check, or that plant an empty ID on purpose, are left
as they are. The pass fixtures are left too: they use short opaque IDs
by design, pass.Validate has no opinion on an ID's grammar, and the one
test that also runs Verify filters by a single code.
Document has no map for an operation, a service or, since the group ID,
a group, so their registries and their empty and duplicate checks come
from the identities the nodes declare. Five comments listed the first
two as if they were the whole set: the Registries doc, the declarations
zero value, checkDeclaredIDs, checkDuplicateIDs and the dangling-
reference walk in pass. They now include groups, so a reader following
one of them to find which classes are handled by declaration does not
conclude that groups are not.

Comments only.
The class tables the irverify ID tests iterate were maps, so subtests ran
in a different order each time, and the property and operation cases
found their class by slice position, which a reordered table would have
pointed at the wrong one. They are ordered slices now and classes are
looked up by name.

The collision test stopped the whole test with t.Fatalf; it uses require
like its neighbours, and ranges over its two rules in a fixed order.

Escaping a path-prefix segment was pinned only by the exhaustive
collision test. Removing it left the spelled-out table green, because no
row there had a character that needs escaping. Add a tilde and a slash.
The framework supplies the separator between an ID's namespace and its
path, but only when the two arrive as separate arguments. The
architecture test forbade converting a string into an ID type and said
nothing about compile.Space, so a compiler could assemble a namespace
from data and carry the path in with it. That is how "t/anonaddr" lost
its separator (#141), and the finished ID is well-formed whatever
produced it.

Planting it showed the gap: a group ID built as
compile.GroupID(compile.Space("tags"+name), "") yields "g/tagspets",
which the architecture test, irverify and every harness oracle accepted.
Only tests that pin literal IDs noticed. irverify cannot see it for an
operation, a service or a group, which record no pointer to agree with.

The sweep now reports a compile.Space converted from anything but a
string literal, or declared with a value that is not one. Every
namespace in the compilers today is a typed literal constant, so nothing
changes there. The planted source covers the declaration, a glued
conversion, a conversion of a variable and of a number, and checks that
literal namespaces and a declaration with no value stay clean; each
clause of the rule fails it when removed.

The rule is syntactic, like its neighbours: it assumes the conventional
package names, and a namespace glued from two constants is not caught,
though that is a decision visible in source and not data reaching an ID.

The comments that said a lost separator in these IDs is invisible now
say what keeps one out.
ir-design §3.1 says an ID held as a reference is never empty, and that a
reference a position may omit is a nil pointer; the emitter design
already writes PollOp and FinalOp that way. Replacing GroupKey with
ir.GroupID left GroupPlan.Parent and ShapeHint.Remount as IR ID
references that use "" for none, which the rule forbids. They are
*ir.GroupID now, nil for top-level and for keep.

The ir-design sentence saying irverify cannot see a lost separator now
cites the issue tracking the way to close it, and the pathAgrees comment
says irverify's only way, since the architecture test is another.
The shape check for operation, service, group and property IDs was its
own check, walking the same nodes as checkDeclaredIDs and having to agree
with it on who owned an empty ID: it skipped them, and the other check
reported them. Both are facets of one rule, that the ID a node declares
for itself is present and well-formed, so checkDeclaredIDs now holds
both and the coordination is gone.

It also removes a full walk of the document. Verify allocated about 8%
more with the separate check, measured over the largest corpus
document, and allocates the same as it did before the shape check now.

The class tables were two as well: one mapped the four non-registry
classes to a kind prefix, and checkIDs spelled the other two inline.
kindPrefixes maps all six, and pointerDerived names the classes whose
path is the pointer the node records, separately, since that is a
different fact about a class than which prefix it has.

Behaviour is unchanged; every existing test passes without edits. A
direct test covers the guard for a class with no prefix, which no
document reaches while channels and messages are registry-keyed.
The rules that keep compilers from building an ID, or a namespace, out
of data matched the conventional names ir and compile. A file importing
either under an alias was not seen, and one importing an unrelated
package that happens to be called compile was reported for it.

Each file's own imports now decide the names, with a file that imports
nothing read as a snippet under the conventional ones. The planted
source covers both an aliased import and an unrelated package, and each
clause of the resolution fails it when removed.

The namespace rule stays syntactic: two constants glued together are
not seen here. The declared-namespace check in irverify sees them.
An ID is kind/namespace/path, and only its first two separators say where
the namespace ends. A producer that glues the path onto the namespace
emits a well-formed ID in a namespace named for the concatenation:
op/openapipaths/~1pets/get. irverify could see that only where a node
records the pointer its ID was derived from, which a type, a scheme and
a property do. An operation records where its body is declared and its
ID where it is mounted, a service records no pointer, and a group has no
provenance, so for those three it verified clean, and so did a producer
outside this repository for any class.

Document gains IDSpaces, the namespaces its IDs live in, keyed by kind
prefix and stated by the producer as it stamps IRVersion. irverify
reports an ID in a namespace the document does not declare for its kind,
whatever the class and whether or not its node records a pointer. The
primitive namespace is ir's own and needs no entry. A document that
declares nothing while carrying such IDs is reported once, not per ID,
as a missing irVersion is. The declaration must itself be usable: keyed
by real kinds, each list sorted without repeats, no namespace empty or
carrying the separator. It is a vocabulary and not a usage report, so a
namespace no ID uses is not an error in a document.

compile.Namespaces builds the declaration from the Space constants a
compiler already derives IDs with. The OpenAPI compiler names its
namespaces once and stamps them beside IRVersion. A test holds that list
to the namespaces the corpus really uses, compiled under both grouping
strategies since the path-prefix groups exist only under the second, and
fails on a namespace used and not declared and on one declared and never
used.

The check reads the declarations irverify already collects, so it adds no
walk. The tests that build a document by hand and assert a clean verify
declare the namespaces their IDs use through one helper, as their
fixtures already stamp irVersion; the tests of the declaration build
their own.

Planting the lost separator showed what this closes. A group ID built as
compile.GroupID(compile.Space("tags"+name), "") passed the architecture
test, irverify and every harness oracle. Gluing two constants, which the
syntactic architecture rule cannot see, did too. Both now fail
TestVerify_Corpus and the other corpus oracles.

BREAKING CHANGE: Document gains the idSpaces key, so every compiled
document changes and a consumer pinned to 0.6.0 must be rebuilt. This
rides the 0.7.0 bump that adding OperationGroup.ID already makes, so it
costs no further generation. There is no migration path (ir-design
§2.1).
@OmarAlJarrah OmarAlJarrah changed the title feat(ir)!: give each operation group a stable ID feat(ir)!: identify operation groups and declare ID namespaces Oct 9, 2026
Three comments and §3.1 said agreement with the recorded pointer is the
only way irverify catches an ID that lost the separator between its
namespace and its path. checkIDSpaces catches it in every class, so the
claim was false.

The rest of the same pass:

- Name the collation of Document.IDSpaces lists, the byte order of the
  UTF-8 spelling, which is what irverify enforces and what a producer
  sorting by UTF-16 code units would otherwise get wrong.
- Say the primitive-namespace exemption is for types only.
- §3.1 no longer says a group's name is part of its ID right after
  defining IDs such as g/tags/<tag name>; the group's Naming is what is
  not part of it.
- §2.1 names declaring IDSpaces as the second obligation of a compiler
  beside stamping IRVersion.
- Rewrap the architecture.md paragraph and realign the ShapeHint.Remount
  line in the emitter-design.md snippet.
sortedKeys and the SortFunc(strings.Compare) call spelled out what
slices.Sorted(maps.Keys(m)) and slices.Sort already say, which the rest
of the repo uses. Drop both, and the strings import that only fed one.

The doc comments of checkIDSpaces and declarationViolations now say the
primitive exemption is for types and name the canonical order.
Each of these held only by accident; widening or dropping it left every
test green:

- The primitive namespace is exempt for types alone. An op/prim/x or
  g/prim/x ID is in a namespace nobody declared.
- kindPrefixes must name each kind in ir.IDKinds exactly once. A kind
  added to the list without a class would have its IDs skipped by the
  grammar check and the namespace check alike, because a class with no
  prefix is left alone.
- A declaration key that is no kind is reported once, as that, and its
  list is not also judged as a kind's would be.
The canonical form of Document.IDSpaces leaves a kind with no namespace
out, as compile.Namespaces.Declaration does. A key holding an empty or
null list verified clean, so a second producer could write one and the
two declarations of one vocabulary would differ byte for byte, which the
sorted-without-repeats rule exists to prevent (invariant 7).

It is now reported as ir/id-spaces-not-canonical at the kind. The kind
switch also replaces the continue after the unknown-kind report, so each
defect in a declaration gets exactly one report by construction.
irverify's tests and the OpenAPI compiler's vocabulary test each computed
the namespaces a document's IDs live in from ir.DeclaredIDs, with slightly
different filters. That derivation is the basis of the guarantee that a
declaration equals what is used, so two copies could drift apart.

irtest.SpacesUsed is the one copy, with its own tests. It leaves out the
primitive namespace, IDs of no known kind and malformed IDs, and returns
each kind's namespaces sorted without repeats.
The package now holds compile.Namespaces, which builds Document.IDSpaces
from the Space constants a compiler derives IDs with, but its comment
listed only the ID grammar. Name it, and drop a clause to stay under the
100-word cap.
The canonical form says sorted by the byte order of the UTF-8 spelling.
A producer sorting by UTF-16 code units, the default in JavaScript and
Java, orders a character outside the BMP before U+FF5E, so the two
orders disagree on exactly the input a non-Go producer could emit. No
test said which one irverify enforces; replacing the comparison with a
UTF-16 one left every test green. It now reddens this one.
irtest gained SpacesUsed, the namespaces a document's IDs live in, so
the three one-line descriptions of the package that called it a golden
helper were incomplete: the README package table, the layout in
CLAUDE.md and the tree in architecture.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant