Skip to content

Suite CI runs no engine: pin a reference machine commit, run hc-conformance on every PR, and mark rows that wait on an engine fix as pending #32

Open

Description

@johnml1135

Suite CI runs no engine, so a fixture that machine's runner rejects can land and be tagged: v0.1.0 shipped one. edge-cases/copy-agreement-featureless-redup had no DOCTYPE and no coverage rows; machine's --check-manifest and 3 of its 50 gate tests failed on the v0.1.0 tree, while this repo's checks job passed.

What we know:

Proposed change:

  1. reference.commit at the repo root names a machine commit; a CI job builds machine's hc-conformance at that commit and runs it on every PR (conformance, stress, --check-manifest).
  2. An expected row may carry pending: sillsdev/machine#N. The job accepts a failure only on pending rows and reports a pending row that now passes; machine's runner reports pending rows as pending, not failed.
  3. A PR that clears pending markers bumps reference.commit to the machine commit that fixed them.

Not known: build time of machine's runner in this repo's CI (locally: build 20 s, run 12 s + 16 s).

Related: #14 (rows record no engine build), #30, #31, sillsdev/machine#615.

Next: add the job with reference.commit once sillsdev/machine#615 is on machine master.

Claude-drafted on 2026-11-03.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions