Skip to content

chore(agents): add test-audit skill for contributor test-quality gating - #3697

Open
politerealism wants to merge 1 commit into
NVIDIA:mainfrom
politerealism:test-audit-skill
Open

politerealism wants to merge 1 commit into
NVIDIA:mainfrom
politerealism:test-audit-skill

Conversation

@politerealism

Copy link
Copy Markdown
Contributor

Summary

Adds .agents/skills/test-audit, an internal contributor skill for gating new tests and auditing existing ones for junk patterns, mock-only coverage of real dependencies, and low-value duplication across OpenShell's test surfaces (Rust unit tests, e2e/rust, the conformance suite, the tmachine/Ansible harness, and the Python/TypeScript/Go SDKs). Modeled on openclaw's test-audit skill, adapted to this repo's actual tooling.

Related Issue

N/A — internal contributor tooling, no product behavior change. (Precedent: PR #293, which introduced .agents/skills/ and sync-agent-infra itself, used the same "N/A — internal initiative" framing.)

Changes

  • .agents/skills/test-audit/SKILL.md (new): three modes (authoring gate, audit, campaign) built around one value bar — does a test protect an independent contract, not just mirror the implementation. Includes two sections grounded in lessons from test(podman): close CI coverage gaps in Podman compute driver testing #3663/test(podman): close rootful userns, resource-limit, and daemon-failure CI gaps #3690:
    • Unit tests: verify the contract, not the implementation — the diagnostic question "where did this expected value come from," with the podman_resource_limits.rs cgroup-value cross-check as a worked example of deriving expected values independently instead of tracing them from the code under test.
    • Integration boundaries: test what crosses, not just what's inside — four real boundaries (process, privilege/identity, declaration-to-enforcement, our-code-to-external-tool) where this repo's own bugs actually lived, each only caught because a test crossed the boundary for real instead of mocking across it.
    • A section on checking for duplicate/overlapping issues and PRs before filing or starting new work.
  • CONTRIBUTING.md: added the skill to the "Agent Skills for Contributors" table under the Contributing category.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated — N/A, this is a documentation/skill file, no code changed
  • E2E tests added/updated — N/A
  • Verified via sync-agent-infra's consistency checklist: skill name matches directory, metadata.internal: true set, name globally unique, no other file (AGENTS.md, README.md, issue templates, workflow-chain references) needed updating.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable) — not applicable; internal skill documentation, not architecture

🤖 Generated with Claude Code

OpenShell had no internal skill for auditing test quality across its
five distinct test surfaces (Rust unit tests, e2e/rust, the conformance
suite, the tmachine/Ansible harness, and the Python/TypeScript/Go SDKs).
Modeled on openclaw's test-audit skill, adapted to this repo's actual
tooling (mise tasks, cargo nextest, uv, driver feature gates) instead of
Vitest, with two OpenShell-specific additions grounded in lessons from
NVIDIA#3663/NVIDIA#3690: a section on deriving expected values from an independent
source rather than tracing the implementation itself, and a section on
testing the handoff between components (process, privilege, and
declaration-to-enforcement boundaries) rather than only what's inside
each one.

Wired into CONTRIBUTING.md's contributor skills table per sync-agent-infra's
maintenance map; no other cross-references needed updating.

Signed-off-by: politerealism <burdcat17@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant