Skip to content

Test all CommonMark spec examples - #168

Merged
bobzhang merged 2 commits into
mainfrom
conformance/spec-examples-test
Oct 2, 2026
Merged

bobzhang merged 2 commits into
mainfrom
conformance/spec-examples-test

Conversation

@bobzhang

@bobzhang bobzhang commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Split out of #152. Merge last: this requires all the other PRs split out of #152. On main alone, this test fails with 631/652, so CI stays red until they are merged.

Adds src/cmark_html/spec_examples_test.mbt. It renders every example of src/data/test/spec.md (CommonMark 0.31.2) with the XHTML renderer, compares the result to the expected HTML, and fails if any example fails. The examples are extracted by a small state machine over the lines of the spec.

passing examples
main 631 / 652
with all PRs below 652 / 652

Required PRs (spec examples each one fixes):

The other split PRs (#157, #158, #159, #160, #164, #165, #166) fix bugs that no spec example covers.

Verified on a local branch that merges all of them: 652/652 on native, js, wasm-gc and wasm. All regression tests of #152 also pass there.

@bobzhang

bobzhang commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Automated review: Codex CLI (codex exec, model reasoning effort high, read-only sandbox) on git diff against the base branch. 2 rounds.

  • Round 1: P3, the failing example numbers were never shown because the count assertion failed first. Fixed in 85258c0: the failure list is now asserted first.
  • Round 2: signed off, subject to the documented merge order.

I sign off on the full diff, subject to the documented prerequisite PRs merging first. No concrete problems found.

  • Round-one assertion-order issue is fixed.
  • All 652 extracted Markdown/HTML pairs match the upstream 0.31.2 extractor, including tabs and trailing newlines.
  • Strict parsing, XHTML rendering, and per-document renderer state are appropriate.

Codex could not run moon test in its read-only sandbox. The tests were run separately on native, js and wasm-gc and pass.

Renders every example of src/data/test/spec.md (CommonMark 0.31.2) with
the XHTML renderer, compares it to the expected HTML and fails if any
example fails. All 652 examples pass once the conformance fixes split
out of #152 are merged; on main 631 pass.
Review feedback from Codex: the first failing assertion stops the test,
so the list of failing examples was never shown.
@bobzhang
bobzhang force-pushed the conformance/spec-examples-test branch from 85258c0 to 1c2b5df Compare October 2, 2026 02:05
@bobzhang
bobzhang merged commit c825aa8 into main Oct 2, 2026
8 checks passed
@bobzhang
bobzhang deleted the conformance/spec-examples-test branch October 2, 2026 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant