Skip to content

feat(std): std.mutation — adopt aeocha's mutation-testing driver - #1588

Merged
paul-hammant merged 2 commits into
mainfrom
feat/std-mutation-testing
Aug 15, 2026
Merged

feat(std): std.mutation — adopt aeocha's mutation-testing driver#1588
paul-hammant merged 2 commits into
mainfrom
feat/std-mutation-testing

Conversation

@paul-hammant

Copy link
Copy Markdown
Collaborator

Lands the asks/aeocha-mutate-in-std-mutation-testing.md ask: adopts
contrib/mutate from the sunsetting aeocha repo — the one piece of real
technology left there with no stdlib home, and the stated blocker to
archiving that repo. Once this merges and releases, aeocha can delete
contrib/mutate and grow its tombstone README.

Shape

  • std/mutation (import std.mutation) — the naming note in the ask
    resolved to the plain segment: no hyphen (lexes as minus), and no
    mutation/testing tail that would collide with std.os.testing for
    co-importers. Library with one entry point,
    mutation.run(sut, test, lib_dir) -> int (survivor count, -1 on
    abort), so a build script can gate on it.
  • examples/mutation-testing/mutate.ae — the runnable front-end
    (ae run … -- <sut.ae> <test.ae> [lib_dir]), with the worked calc
    example in lib/ (ported from aeocha's README demo to std.spec).
    A future ae mutate subcommand remains the natural evolution, per the
    ask explicitly not required here.
  • docs/mutation-testing.md — usage, operator table, honest Tier-1
    limits, and the AST-level upgrade path (the reason in-tree is the
    right home).

What carried over unchanged

Outer-loop text mutation (padded-operator + string-literal mutants,
string-boundary aware), the three-way verdict via the structured report
(AE_SPEC_FORMAT=aeocha + AE_SPEC_REPORT, reading failed=N — a
non-compiling mutant never masquerades as a kill), the error[-grep
no-compile gate (#953 workaround; fixing #953 in-tree later simplifies
it), and byte-identical SUT restore.

What changed in the port

  • Facade-free: the driver and all fixtures/examples import std.spec
    directly (aeocha was already a pure forwarder).
  • Sub-builds honour AE_BIN (defaults to ae on PATH) so the
    in-tree harnesses can point at build/ae.
  • os.system / os.getenv via std.os instead of a raw extern; probe
    binary is pid-suffixed so concurrent runs can't clobber each other;
    sentinel renamed AE_MUTANT.

Acceptance (the ask's bar)

Both deterministic aeocha fixtures ported to
tests/integration/mutation_testing/ and passing locally against
build/ae:

  • operator fixture: baseline pass, exactly 1/2 mutants killed — 50%,
    SURVIVED sut.ae:11 MUL->DIV, killed sut.ae:9 ADD->SUB, md5-identical
    SUT restore
  • strings fixture: exactly 1/3 … 33%, STR->EMPTY killed at sut.ae:10 /
    survived at sut.ae:12, and no ADD->SUB mutant from the +
    inside a string literal

Fixture dirs are in tests/ae_sweep_prune.txt (driven by their own
test_*.sh; the main-less SUT modules would otherwise break the bulk
sweep). The examples-target compile+link of mutate.ae is verified
(calc SUT/test live under lib/, which the examples build skips by
convention).

🤖 Generated with Claude Code

paul-hammant and others added 2 commits August 15, 2026 11:31
Adopts contrib/mutate from the sunsetting aeocha repo — the last piece
of real technology there with no stdlib home, and the blocker to
archiving that repo.

- std/mutation/module.ae: the Tier-1 text-based driver as a library
  with one entry point, mutation.run(sut, test, lib_dir). Padded
  operator swaps + string-literal mutants, string-boundary aware;
  killed/survived/no-compile verdicts read from std.spec's structured
  report (AE_SPEC_FORMAT/AE_SPEC_REPORT env-file contract) so a
  non-compiling mutant never counts as a kill; SUT restored
  byte-identical. Per-mutant sub-builds honour AE_BIN (harnesses point
  it at build/ae) and use os.system/os.getenv via std.os.
- examples/mutation-testing/: runnable front-end (mutate.ae) plus the
  worked calc example (lib/calc.ae + lib/calc_test.ae, ported from
  aeocha's README demo to std.spec).
- tests/integration/mutation_testing/: both deterministic aeocha
  regression fixtures ported — operator fixture pins 1/2 - 50% with a
  MUL->DIV survivor at sut.ae:11 and md5-identical restore; strings
  fixture pins 1/3 - 33% and that an operator inside a string literal
  is never mutated as code. Fixture dirs added to the .ae sweep prune
  list (they are driven by their own test_*.sh).
- docs/mutation-testing.md: usage, operator set, honest Tier-1 limits,
  and the AST-level upgrade path that is the reason the tool now lives
  in-tree.

Both harnesses pass locally against build/ae; the examples-target
compile+link of mutate.ae verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@paul-hammant
paul-hammant merged commit a4bbeff into main Aug 15, 2026
25 checks passed
@paul-hammant
paul-hammant deleted the feat/std-mutation-testing branch August 15, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant