Nothing tracks build-tool versions in operator/deps.mk: 10 of 15 are behind, ginkgo and setup-envtest have drifted

[lockwobr]

Summary

Nothing tracks the build-tool versions in operator/deps.mk. Fifteen tools are pinned by hand there; ten are currently behind, two have drifted out of sync with versions declared elsewhere in the repo, and one is pinned to a git branch rather than a version, which is neither trackable nor reproducible.

#520 adds Renovate but scopes it to gomod, pep621, dockerfile, and github-actions. A custom.regex manager plus annotation comments would cover deps.mk with no structural change to any module.

Follows #520. Related to #521.

Current drift

Pinned values from operator/deps.mk, latest resolved from the Go module proxy and the GitHub releases API on 2026-08-24:

Variable	depName	Pinned	Latest
KUSTOMIZE_VERSION	sigs.k8s.io/kustomize/kustomize/v5	v5.4.1	v5.8.1
GINKGO_VERSION	github.com/onsi/ginkgo/v2	v2.28.1	v2.32.1
GOVULNCHECK_VERSION	golang.org/x/vuln	v1.3.0	v1.7.0
YQ_VERSION	github.com/mikefarah/yq/v4	v4.44.3	v4.53.6
HELMIFY_VERSION	github.com/arttor/helmify	v0.4.12	v0.4.20
HELM_VERSION	helm/helm	v4.1.4	v4.2.4
MOCKERY_VERSION	github.com/vektra/mockery/v3	v3.7.0	v3.7.4
GOLANGCI_LINT_VERSION	golangci/golangci-lint	v2.12.2	v2.13.1
GOCOVER_VERSION	github.com/boumenot/gocover-cobertura	v1.4.0	v1.5.0
(inline, no variable)	sigs.k8s.io/controller-runtime/tools/setup-envtest	release-0.22	v0.24.1
CONTROLLER_TOOLS_VERSION	sigs.k8s.io/controller-tools	v0.21.0	v0.21.0
GO_LICENSES_VERSION	github.com/google/go-licenses	v1.6.0	v1.6.0
ADDLICENSE_VERSION	github.com/google/addlicense	v1.2.0	v1.2.0
CHAINSAW_VERSION	kyverno/chainsaw	v0.2.15	v0.2.15
CTLPTL_VERSION	tilt-dev/ctlptl	v0.9.4	v0.9.4

govulncheck being four minors behind is the one worth calling out on its own: it is the vulnerability scanner, and its detection quality is a function of its version.

Three concrete defects behind the drift

1. The ginkgo CLI and the ginkgo library have diverged

operator/go.mod:7 requires github.com/onsi/ginkgo/v2 v2.32.1. operator/deps.mk:58 installs the ginkgo CLI at v2.28.1. Four minors apart, and ginkgo emits a version-mismatch warning when the CLI and the compiled suite disagree. Every make unit-tests and every ./bin/ginkgo run --focus invocation runs this mismatch today.

2. setup-envtest is pinned to a branch, two minors behind controller-runtime

operator/deps.mk:117:

test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-0.22

operator/go.mod:19 is on sigs.k8s.io/controller-runtime v0.24.1, so the envtest binary is two minors behind the controller-runtime the suites link against.

The branch pin is also not reproducible: go install ...@release-0.22 resolves the branch tip to a pseudo-version at install time, and the test -s guard means the result is then cached indefinitely in bin/. Two developers who first ran make envtest months apart are on different binaries from an identical checkout, as is any CI runner with a cold bin/.

This is now fixable. sigs.k8s.io/controller-runtime/tools/setup-envtest is an independently tagged module:

$ curl -s https://proxy.golang.org/sigs.k8s.io/controller-runtime/tools/setup-envtest/@v/list
v0.24.0
v0.24.1

@latest is v0.24.1, cut 2026-05-11 from refs/tags/tools/setup-envtest/v0.24.1, the same commit and timestamp as controller-runtime v0.24.1. The submodule tags are cut in lockstep with controller-runtime releases. Tagging only began at v0.24.0, which is why the kubebuilder-scaffolded branch form is still what is in the tree.

3. setup-envtest has no variable to annotate

The version is inline in the recipe, so converting it also means introducing ENVTEST_VERSION.

Proposed approach

Annotate deps.mk, change nothing structural

# renovate: datasource=go depName=github.com/onsi/ginkgo/v2
GINKGO_VERSION ?= v2.32.1
# renovate: datasource=github-releases depName=kyverno/chainsaw
CHAINSAW_VERSION ?= v0.2.15

datasource=go for the eleven go install tools, datasource=github-releases for golangci-lint, chainsaw, ctlptl, and helm. One custom manager reads both:

customManagers: [{
  customType: "regex",
  managerFilePatterns: ["/^operator/deps\\.mk$/"],
  matchStrings: ["# renovate: datasource=(?<datasource>\\S+) depName=(?<depName>\\S+)\\s+\\w+_VERSION \\?= (?<currentValue>\\S+)"],
}]

"regex" has to be added to enabledManagers in .github/renovate.json5, currently ["dockerfile", "github-actions", "gomod", "pep621"].

These PRs are one-line diffs. The custom manager is not the gomod manager, so no go mod tidy, no vendoring, and no make notices post-upgrade task fires for them.

Convert setup-envtest to a version pin

# renovate: datasource=go depName=sigs.k8s.io/controller-runtime/tools/setup-envtest
ENVTEST_VERSION ?= v0.24.1
...
	test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@$(ENVTEST_VERSION)

Validate with make unit-tests: ENVTEST_K8S_VERSION is 1.36.0 from operator/versions.yaml, and moving from release-0.22 to v0.24.1 changes which binary index setup-envtest reads, so confirm 1.36.0 still resolves.

Group the pins that must move together

Two pairs are cut in lockstep and should never drift again:

• ginkgo: the deps.mk CLI pin and the go.mod library dep share depName: github.com/onsi/ginkgo/v2, so a groupName: "ginkgo" rule on matchDepNames catches both in one PR.
• setup-envtest + controller-runtime: sigs.k8s.io/** already matches the existing kubernetes group.

Gotcha: the kubernetes group in .github/renovate.json5 is matchManagers: ["gomod"], so a custom.regex pin will not join it. Grouping across the two managers requires adding "custom.regex" to matchManagers on those rules.

Gotchas for whoever picks this up

• A wrong depName fails silently. Renovate proposes nothing and reports no error. For several tools the module path differs from the go install package path: controller-gen is sigs.k8s.io/controller-tools (not .../cmd/controller-gen), helmify is github.com/arttor/helmify (not .../cmd/helmify), govulncheck is golang.org/x/vuln (not .../cmd/govulncheck), ginkgo is github.com/onsi/ginkgo/v2 (not .../ginkgo). Every path in the table above was verified against proxy.golang.org/<module>/@latest; re-verify any that change.
• Confirm with a dry run before merging. .github/workflows/renovate.yaml supports workflow_dispatch with dryRun.
• setup-envtest has only two tags. The series is young. Confirm Renovate's go datasource resolves it rather than assuming.
• Bumping ten tools at once will produce real diffs (regenerated mocks from mockery, new golangci-lint findings, regenerated helm output from helmify). Landing the annotations and the bumps as separate PRs will keep those reviewable.

Rejected alternatives

Recording these so they are not re-litigated.

Move all pins into operator/versions.yaml and let it drive everything. No, for three reasons. It is circular: versions.sh reads the file with yq, and YQ_VERSION is itself one of the pins, so yq's version cannot live in the file that requires yq to read it. It buys Renovate nothing, since a customManager regex is needed either way and it does not care whether the pin is in a .mk or a .yaml. And the two files hold different kinds of value: versions.yaml is a support-matrix policy (ci.kindNodeImages is a deliberate four-version window tied to docs/operations/kubernetes-support.md), whereas deps.mk pins are "keep me current." Mixing "automation may bump this" with "a human decides this" into one file makes it harder to reason about which lines are safe to touch.

Convert the go install tools to Go tool directives in operator/go.mod. No. The main module's require graph is shared, so adding kustomize, controller-tools, mockery, and yq puts their dependencies into MVS with the operator's, raising k8s.io/* and golang.org/x/* to whatever the tools need. For a controller deliberately pinned to a Kubernetes minor that is unacceptable, independent of the vendor-tree growth it would also cause.

Introduce a tools.mod (or operator/tools/go.mod) holding the tool requirements. No, though it does solve the vendoring and MVS objections above. Two reasons it still loses to annotations. First, go install <pkg>@<version> is already module-independent: in module-aware mode it ignores the surrounding module entirely, so deps.mk is already the separate-module solution, just expressed in Make. A tools file adds a go mod tidy surface and a second dependency graph to maintain for no behavior change. Second, it does not even buy free Renovate coverage: Renovate's gomod manager matches (^|/)go\.mod$, so a file literally named tools.mod is not picked up without custom managerFilePatterns anyway. Only the directory form, operator/tools/go.mod, would match, and that is the heavier of the two shapes. Either way a custom manager or extra config is required, which is exactly what the annotation approach already does with less moving structure.

Acceptance criteria

• Every version pin in operator/deps.mk carries a # renovate: annotation, or a comment explaining why it cannot.
• "regex" is in enabledManagers and make renovate-config-check passes.
• A workflow_dispatch dry run proposes updates for the ten out-of-date tools and resolves every depName.
• setup-envtest is pinned to a version, not a branch, and make unit-tests passes against ENVTEST_K8S_VERSION 1.36.0.
• The ginkgo CLI pin and the go.mod ginkgo library are grouped into a single PR and are at the same version.

[mohityadav8]

/assign