Skip to content

refactor: replace hand-written code with standard library calls - #825

Merged
OmarAlJarrah merged 20 commits into
mainfrom
refactor/use-stdlib-helpers
Oct 12, 2026
Merged

OmarAlJarrah merged 20 commits into
mainfrom
refactor/use-stdlib-helpers

Conversation

@OmarAlJarrah

@OmarAlJarrah OmarAlJarrah commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Go 1.27's standard library now covers a good share of the small loops and helpers this repo wrote by hand. This replaces them, so there is less code to maintain. One output ordering changes, described below; no other behaviour change is intended. Each commit is one kind of replacement:

  • the rewrites go fix and the modernize analyzer propose, including reflect.TypeAssert, with the names they produced tidied by hand
  • sort.* to slices.Sort* (all but one keep their semantics, see below), and hand-written clone and key-collecting loops to slices / maps helpers
  • small local helpers the standard library already provides, for example defs.parentPointer is jsontext.Pointer.Parent
  • ast.Inspect callbacks that never prune to ast.Preorder, and NumField / MapRange loops in tests to Fields / Seq2 (reflect loops in production code stay indexed, see below)
  • "does any element match" loops to slices.ContainsFunc / IndexFunc
  • index-then-slice splits to strings.Cut, append-onto-nil copies to slices.Clone, and an unchecked reflect assertion to a comma-ok reflect.TypeAssert
  • test helpers that list a package's non-test source files use go/build.ImportDir instead of filtering os.ReadDir
  • locals that exist only so a field or return can take their address use new(value): constants and literals of constants rewritten mechanically, plus five computed values whose initializer is a pure read that nothing between it and its use can change
  • errors.AsType where a typed local was declared only for errors.As, strings.CutLast for the LastIndex-then-slice pair, Keys() where a loop took only the key of a sequencedmap, and a range over an int where a counted loop only counts
  • intrange, modernize, usestdlibvars and usetesting are enabled in .golangci.yml, since nothing in the gate ran go fix or the modernize analyzers. They hold the forms the analyzers propose: counted loops, slices.Backward, SplitSeq, reflect.Type.Fields outside navigation.go, Index splits that keep both halves, LastIndex splits, errors.AsType, slices.Contains loops, any, http.Method* and the t. helpers. The rest of this list has no linter: sort.*, the clone and collect loops, ContainsFunc with a predicate, ast.Preorder, new(value), go/build, and an Index that only truncates
  • TestMerger_ReconcileReportsDisagreementAndKeepsAWinner now asserts the winner its name promises; it checked only the diagnostic

The one output change

coordinatesBelow, which lists what interned beneath a coordinate in the --explain report, sorted with sort.Slice. It now uses slices.SortStableFunc. Nodes that share a coordinate list in the registry's ID order, which is how nodeAtPointer reads them. The unstable sort left such ties in an order that depended on the list: insertion sort is stable up to twelve elements, and past that ties can come out in any order. On the main binary, compile testdata/conformance/openapi/allof-oneof-cooccurrence.yaml --explain /components/schemas lists the two nodes at /components/schemas/Combo/oneOf/0 as composed then anon; it now lists anon then composed. It is the only spec under testdata/ whose report differs. TestCoordinatesBelow_TiedPointersListInIDOrder pins the order, and no fixture had enough entries to do so before.

Left as they are on purpose

  • cmd/morphic execute still copies args with append([]string{}, …): slices.Clone(nil) is nil, and cobra then reads os.Args[1:].
  • JSON Pointer escaping stays hand-written (defs.pointerEscaper, openapitest.escape): jsontext.Pointer.AppendToken rewrites a byte that is no UTF-8 as U+FFFD, a key the library never compares. defs.pointerEscaper now says so; openapitest.escape already did.
  • nodeview.InternalPointer keeps strings.Split(ref, "#"); Cut reads a second # differently from the resolver it mirrors.
  • annotation/decimal.go keeps its own exponent handling, since math/big cannot build 1e2000000.
  • "every element passes" loops stay loops: there is no slices.All, and a negated ContainsFunc reads worse.
  • The remaining ast.Inspect calls prune subtrees with return false, which Preorder cannot do.
  • The four ts.PrimRef locals in schema.go keep their form: PrimRef interns a primitive, so moving it past scalarEncoding would change the order types intern in.
  • errors.As in cmd/morphic/exit.go stays: its condition also tests the match, so AsType would add a statement.
  • Reflect loops in production code keep indexing and MapRange: ir.WalkValues, four irverify checks, and navigation's embedsAnswer and keyed. A range over Value.Fields, Seq, Seq2 or Type.Fields cannot be inlined, so each loop puts its body on the heap. In WalkValues that took every caller's visit closure with it, and BenchmarkValidate_Petstore went from 5669 to 9031 allocations per op; keyed paid four allocations a call where indexing pays none. TestWalkValues_StructsAndMapsCostNoAllocation and TestKeyed_CostsNoAllocation pin both, and .golangci.yml declines modernize's stditerators for navigation.go, the one file where it proposes the range.
  • harness TestCheckPath_UnreadableFileIsError keeps os.MkdirTemp: a unix socket path is capped near 104 bytes, and t.TempDir spells the test name into its path. usetesting has a path-and-call exclusion whose comment says so.

Not in this PR

JSON Pointer unescaping (defs.go, load/treereads.go, navigation.go) could become jsontext.Pointer.Tokens(), which matched the hand-written form on every input tried, invalid UTF-8 included. Those reads deliberately mirror how the resolver reads a pointer, and two of them unescape one token rather than a pointer, so the change needs its own review: #827.

Test plan

  • go build ./..., go vet ./... and go test ./... pass on the final tree
  • go fix -diff ./... and the modernize analyzer report only keyed's loop, the range declined above; applying it with go fix ./... turns TestKeyed_CostsNoAllocation red (4 allocations)
  • The overlay test helper's old and new bodies return identical maps over every compilable spec under testdata/ (130 documents, 218 entries)
  • jsontext.Pointer.Parent matched defs.parentPointer on 2M random inputs
  • The pinned golangci-lint release (v2.14.0, built with the repo's toolchain) reports 0 issues on the final tree with the four new linters enabled
  • Planted violations show the linters reach: intrange, usestdlibvars and usetesting fire on them, modernize on strings.Index splits and interface{}
  • For each rewritten production site (including NormalizeMediaType, parseDecimalBound, both reach.go copies and the overlay test helper), a planted defect turns its package's tests red, and a no-op control stays green
  • The main and branch binaries print the same --explain report for every spec under testdata/ at six coordinates, except the tied pair above
  • TestCoordinatesBelow_TiedPointersListInIDOrder fails when coordinatesBelow goes back to an unstable sort, and passes with a comment-only edit
  • BenchmarkValidate_Petstore makes 5669 allocations per op on main and on the final tree
  • TestWalkValues_StructsAndMapsCostNoAllocation fails when either walk loop ranges over Fields or Seq2 (5 against 14 and 20 allocations), also under -race, and passes with a comment-only edit
  • TestKeyed_CostsNoAllocation fails on the Type.Fields range (4 allocations) and passes with a comment-only edit; without the .golangci.yml exclusion, stditerators flags keyed and nothing else
  • TestMerger_ReconcileReportsDisagreementAndKeepsAWinner fails when the later declaration's type wins
  • CI gate (coverage, lint)

Replace hand-written loops and assertions with the standard library's
iterator and generic forms:

- three-clause integer loops range over the integer
- reverse index loops use slices.Backward
- NumField/Field loops over a reflect.Type use Type.Fields
- errors.As with a declared target uses errors.AsType
- ranging over strings.Split or strings.Fields uses SplitSeq and FieldsSeq
- v.Interface().(T) on a reflect.Value uses reflect.TypeAssert, which
  skips the interface boxing

No behaviour change.
Replace the sort package and hand-written copy and collect loops with
their slices and maps equivalents:

- sort.Strings, sort.Slice and sort.SliceStable become slices.Sort,
  slices.SortFunc and slices.SortStableFunc, ordered with cmp.Compare
- append([]T(nil), s...) and make-and-copy become slices.Clone
- appending two slices into a fresh one becomes slices.Concat
- loops that gather map keys or values, or the tokens of a JSON pointer,
  become slices.Collect, slices.AppendSeq and maps.Keys or maps.Values

The explain command's coordinate sort now uses a stable sort: nodes that
share a pointer arrive ordered by ID, and only a stable sort guarantees
they stay in that order.
- defs: drop parentPointer, which is jsontext.Pointer.Parent; the test
  that pinned its bound now pins the method's
- openapi detect: drop trimBOM in favour of bytes.TrimPrefix
- operation: drop schemaHasType, a wrapper around slices.Contains
- load: compare pointers with cmp.Compare instead of converting both
  sides to string for strings.Compare
- tests: t.Chdir replaces a manual os.Chdir and cleanup, benchmarks use
  b.Loop (which also resets and stops the timer), bytes.NewReader
  replaces strings.NewReader(string(b)), and bytes.Equal replaces a
  string comparison of two byte slices
- constant error messages use errors.New rather than fmt.Errorf
- len(s) > 0 on a string is s != ""
- ast.Inspect callbacks that only ever return true become range loops
  over ast.Preorder
- struct field loops that paired reflect.Type.Field(i) with
  reflect.Value.Field(i) range over reflect.Value.Fields
- map loops built on MapRange or MapKeys range over reflect.Value.Seq2,
  or Seq where only the keys are read

No behaviour change.
Five tests listed a package's production Go files by reading the
directory (or globbing it) and filtering out names ending in _test.go.
go/build.ImportDir returns exactly that set, already sorted, and fails
when there is none, so the filter and the emptiness check go away.

Unlike the filter it honours build constraints, which is what these
tests want: they read the sources the package builds from.
Loops whose only job is to report whether, or where, an element
matches a predicate become calls to slices.ContainsFunc and
slices.IndexFunc. This covers the diagnostic lookups in openapitest and
the tests, DeclaresAny, the $id ancestor check, the exit-code helpers
and ir.FirstError.

No behaviour change.
Index-then-slice splits become strings.Cut, append-onto-nil copies become
slices.Clone, and the unchecked assertion in the overlay test helper becomes a
comma-ok reflect.TypeAssert, which also drops its explicit type comparison.

pointerEscaper now says why it is not jsontext.Pointer.AppendToken: that
rewrites a byte that is no UTF-8 as U+FFFD, a key the library never compares.
A local holding a constant or a literal of constants, read once as &local,
is the pointer `new` now builds from the value. Rewriting only those, and
only where the single use sits in the same block with no loop or closure
between, leaves the value read exactly where it was written.

Locals that hold a computed value stay: moving a call to its use site would
reorder whatever runs between them.
Five locals each exist only so a field or return can take their address.
Each initializer is a pure read made on the line before its use with
nothing in between that could change it, so the call moves into new(...)
without reordering anything.

The four ts.PrimRef locals in schema.go stay: PrimRef interns a primitive,
and moving it past scalarEncoding would change the order types intern in.
errors.AsType returns the matched error and a bool, so the two sites that
declared a typed local only to hand errors.As its address take the pair
directly. strings.CutLast splits around the final separator, which is what
the LastIndex-then-slice pair in lastSegment and in the mappings test did
by hand.

errors.As in cmd/morphic/exit.go stays: its condition also tests the match,
so AsType would add a statement rather than remove one.
Two loops range over a sequencedmap's All() and use only the key; Keys()
yields exactly that. One of them copied the keys into a slice, which
slices.Collect does. A fixture loop counted j up to n/6+1 with no other use
of the bound; ranging over the int says so.
The loop counts i up to len(pointer) and indexes with it, touching neither
the bound nor i inside. Ranging over the length keeps the byte index that
percent-encoding needs; ranging over the string would decode runes and
split an invalid UTF-8 byte differently.
go fix and the modernize analyzers report nothing on this tree, but nothing
in the gate kept it that way: golangci-lint ran none of them, so the next
hand-written strings.Index split or counted loop would pass review and CI
alike.

intrange and usestdlibvars catch the counted loop and the spelled-out HTTP
method. modernize adds the stdlib forms go fix proposes, such as strings.Cut
and any. usetesting prefers t.TempDir, t.Chdir and t.Setenv over their os
spellings.

One site is excluded by path and by call: TestCheckPath_UnreadableFileIsError
builds a unix socket under os.MkdirTemp, because t.TempDir spells the whole
test name into a path the socket limit (about 104 bytes) will not take.
coordinatesBelow now sorts with a stable sort, so nodes that share a
coordinate list in the registry's ID order, the order nodeAtPointer
reads them in. The unstable sort it replaced left ties in an order
that depended on the list: insertion sort is stable up to twelve
elements, and past that the order was scrambled. The conformance spec
allof-oneof-cooccurrence.yaml lists thirteen coordinates and showed it.

No fixture had enough entries for the difference to show, so reverting
to an unstable sort left the suite green. Eight pointers of eight nodes
each fail on it.
The helper had three callers. Replacing it with the standard library
call at each of them spelled the same conversion and Collect three
times on already long lines. Keep the helper and let its body be the
single Collect call, so the call sites are untouched.
A range over reflect.Value.Fields, Seq or Seq2 cannot be inlined, so each
loop puts its body on the heap. In ir.WalkValues the struct loop also
moved the walker itself there, which made the visit closure of every
caller escape too. pass.Validate, which every compile runs, went from 5669
to 9031 allocations and took about a fifth longer on the Petstore
benchmark.

The walk, the four irverify checks that range over a reflect value and
navigation's embedsAnswer go back to indexed loops and MapRange, which
restores 5669. navigation's keyed keeps Type.Fields: modernize flags its
indexed form, and it allocates once per call rather than once per value.

TestWalkValues_StructsAndMapsCostNoAllocation walks one struct and a chain
of four, each embedding an empty map, and requires both to allocate alike.
Either iterator form fails it.
The stable sort made the order of nodes that share a pointer part of the
explain report, and a test pins it, but the doc comment still promised
only pointer order.
TestMerger_ReconcileReportsDisagreementAndKeepsAWinner checked the
diagnostic but not the property, so letting the later declaration's type
win left it green. It names the merged property again and asserts the type
it keeps.
keyed runs as pointer reads step through models. Ranging over
reflect.Type.Fields costs it four allocations a call, where the indexed
loop costs none, so it goes back to indexing as the value walks did. The
previous commit kept the range on the reading that it cost one allocation
a call; measured, it is four.

modernize's stditerators proposes that range for this loop, so
.golangci.yml declines it for navigation.go alone, the way it declines
usetesting for one harness test. TestKeyed_CostsNoAllocation fails on the
range form.
The walk test's doc described the iterator form as having raised
Validate's allocations, a state the merged history never holds; it now
states the cost in the present tense. The keyed test's doc named only
modernize, but go fix proposes the same rewrite and is not stopped by the
lint exclusion, so a go fix run is the change this test exists to catch.
@OmarAlJarrah
OmarAlJarrah merged commit 9867eab into main Oct 12, 2026
1 check passed
@OmarAlJarrah
OmarAlJarrah deleted the refactor/use-stdlib-helpers branch October 12, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant