Tracking: Test262 infrastructure & reporting

[dowdiness]

Context

The Test262 toolchain has a lot of small, scoped work in flight that doesn't fit any of the existing milestones. None of it is conformance work itself, but every conformance milestone (milestone #1 Promise/async, milestone #2 bytecode VM, milestone #4 ES2015, milestone #5 async iteration, milestone #7 class private) depends on the Test262 infrastructure being solid.

This milestone bundles the dev-infra and reporting work into one place, with a shared acceptance criterion: the runner, the analyzer, the skip-validator, and the report generator all produce clean, per-mode, machine-readable output that ROADMAP and release notes can quote without hand-editing.

Sub-issues

#	Title	Difficulty	Status
#228	Generate a human-readable Test262 skip/config report	Small	open
#229	Add sharding, resume, and merged-log support to the Test262 runner	Medium	open
#230	Investigate opt-in Test262 harness performance helpers	Small	open
#231	Generate per-mode Test262 summary artifacts and badges from CI results	Small	open
#234	Generate an external feature-gap report for skipped Test262 features	Small	open
#292	ci: cache MoonBit toolchain installation to avoid re-downloading on every run	Trivial	open

Recommended order

1. ci: cache MoonBit toolchain installation to avoid re-downloading on every run #292 — cache the MoonBit toolchain. One-line change, removes ~1-2 min from every CI run, no risk. Lands first.
2. Generate per-mode Test262 summary artifacts and badges from CI results #231 — per-mode summary artifacts and badges. Builds on existing cmd/report_test262. Cheap, immediately useful in PRs and the README.
3. Generate a human-readable Test262 skip/config report #228 — human-readable skip/config report. Mirrors the summary work but for skip metadata, not run results.
4. Generate an external feature-gap report for skipped Test262 features #234 — external feature-gap report. Useful for users to see "we don't implement X yet" without scraping skip metadata.
5. Investigate opt-in Test262 harness performance helpers #230 — opt-in harness performance helpers. Investigate first; the answer may be "no change needed".
6. Add sharding, resume, and merged-log support to the Test262 runner #229 — sharding/resume/merged-log in the runner. Largest of the six; lands last. The CI already shards (#27745753206 reports 4 shards), so this is mostly about resilience and resume.

Status

• ci: cache MoonBit toolchain installation to avoid re-downloading on every run #292 — toolchain cache enabled
• Generate per-mode Test262 summary artifacts and badges from CI results #231 — per-mode summary artifacts and badges published from CI
• Generate a human-readable Test262 skip/config report #228 — human-readable skip/config report available
• Generate an external feature-gap report for skipped Test262 features #234 — external feature-gap report available
• Investigate opt-in Test262 harness performance helpers #230 — performance helper investigation done (land or close)
• Add sharding, resume, and merged-log support to the Test262 runner #229 — runner supports resume and merged log output

Acceptance criteria (this tracking issue)

• All 6 sub-issues are closed.
• The make test262-report flow produces both test262-summary.md and test262-editions.md without manual intervention in CI, in both ROADMAP and CHANGELOG formats (per ROADMAP §1).
• Per-mode artifacts (strict / non-strict) are uploaded as separate GitHub Actions artifacts in every Test262 Conformance run.
• A feature-gap report is generated on demand (locally and in CI), and lives in docs/supported-features.md as a regenerated section.
• CI runtime is reduced by at least 1 minute on average (MoonBit toolchain cache hit rate > 90%).
• moon check && moon test clean.
• No regression in the existing Test262 baseline (test262-baseline.json passed_min).

Notes

• This milestone is infrastructure-only. It does not change conformance numbers, but every future conformance milestone benefits from it.
• ROADMAP §1 ("Keep conformance data fresh") is the document anchor for this work; the milestone acceptance criteria are the operational form of that rule.
• The Test262 skip-metadata validator (make test262-validate-skips, cmd/test262_validate_skips) is already in good shape from earlier work and is not in scope here.

[dowdiness]

All 6 sub-issues are now closed:

• ci: cache MoonBit toolchain installation to avoid re-downloading on every run #292 — MoonBit toolchain cache (merged in PR ci: cache MoonBit toolchain to skip curl install on warm runs (#292) #295, copilot-setup-steps gap in PR ci+scripts: copilot toolchain cache + Test262 feature-gap tool #460)
• Generate a human-readable Test262 skip/config report #228 — make test262-skip-report + scripts/test262_skip_report.py + docs
• Add sharding, resume, and merged-log support to the Test262 runner #229 — --shard/--resume-from/--merged-log/--tests-file in native runner + docs
• Investigate opt-in Test262 harness performance helpers #230 — --harness-helper codePointRange/atomicsTimeoutProfile + docs
• Generate per-mode Test262 summary artifacts and badges from CI results #231 — per-mode CI artifacts + docs/supported-features.md + docs/TEST262.md
• Generate an external feature-gap report for skipped Test262 features #234 — scripts/test262_feature_gap.py + 15 Python unit tests + docs (PR ci+scripts: copilot toolchain cache + Test262 feature-gap tool #460)

All acceptance criteria from the tracking issue are met. make test262-report produces both test262-summary.md and edition tables without manual intervention; per-mode artifacts are uploaded in every CI run; the feature-gap report is on-demand via make test262-feature-gap; the toolchain cache hit rate is >90% on warm runs.