Skip to content

fix: match preflight CI check names to the Checks API - #749

Merged
yvonnedevlinrh merged 3 commits into
complytime:mainfrom
SatriaBagusNurhadianto:fix/preflight-check-names
Nov 19, 2026
Merged

yvonnedevlinrh merged 3 commits into
complytime:mainfrom
SatriaBagusNurhadianto:fix/preflight-check-names

Conversation

@SatriaBagusNurhadianto

@SatriaBagusNurhadianto SatriaBagusNurhadianto commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

What changed:

  • Auto-discovery for ci_local.yml now records the bare job name instead of "<workflow name> / <job name>".
  • The hardcoded ci_checks.yml linter check is now Standardized CI / Run linters instead of CI / Standardized CI / Run linters.
  • Added tests/test_preflight_check_discovery.sh and wired it into ci_test_preflight.yml.

Why:

  • The Checks API reports the job's own name for direct jobs and "<calling job> / <called job>" for reusable calls. The extra workflow-name prefix made the exact-match lookup miss every check, so releases were blocked when ci_checks was left empty.

Tests:

  • bash tests/test_preflight_check_discovery.sh passes 4/4.
  • actionlint on both changed workflows is clean.
  • yamllint on normalized copies is clean. The local Windows checkout stores CRLF, so the raw working-tree files report a newline error; the committed blobs are LF.
  • tests/test_preflight_token.sh was not run because yq is not installed in this environment. It is unrelated to this change.

Fixes: #684

@marcusburghardt marcusburghardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check-name corrections address the reported mismatch, and the change is appropriately scoped. Before merging:

  1. Fix the PR-caused ShellCheck failures in the new test while preserving its intended literal matching.
  2. Add behavioral regression coverage. Source-string assertions do not verify discovery output. Execute the actual discovery logic against fixtures covering direct jobs with explicit names, job-ID fallback, and the corrected shared linter check name. Avoid duplicating the implementation inside the tests.
  3. Update openspec/specs/release-preflight/spec.md, whose discovery scenarios still describe the incorrect workflow-prefixed names.

The test’s runtime and dependencies are appropriately lightweight; the concern is regression protection, not performance.

This review was generated by /uf.review-pr (AI-assisted).

Comment thread tests/test_preflight_check_discovery.sh Resolved
Comment thread tests/test_preflight_check_discovery.sh Resolved

@yvonnedevlinrh yvonnedevlinrh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing this fix I also reviewed the changes and agree with Marcus’s feedback, the naming correction looks right, and behavioural tests would help protect it from regressions.

One small addition: please update the “CI Check Auto-Discovery” section in docs/RELEASE_WORKFLOWS.md alongside the spec, as its examples still use the old workflow-prefixed names.

@SatriaBagusNurhadianto

Copy link
Copy Markdown
Contributor Author

Addressed both reviews in 5d99da6.

ShellCheck SC2016 — gone with the source-text assertions. The test file now passes shellcheck cleanly.

Behavioral regression coverage — tests/test_preflight_check_discovery.sh extracts the live Discover CI checks step body from reusable_release_preflight.yml with yq and executes it inside throwaway fixture repositories, so the discovery implementation is not copied into the test:

  • direct jobs with explicit names (Unit Tests, Integration Tests)
  • job key fallback for a job without a name (smoke)
  • the corrected shared linter check (Standardized CI / Run linters), plus an explicit guard that CI / Standardized CI / Run linters is never produced
  • combined discovery of all three files, and confirmation that an explicit ci_checks input still bypasses auto-discovery

bash tests/test_preflight_check_discovery.sh reports 6/6 passing. yq is installed in the preflight test job with a pinned version and SHA-256 checksum so these cases actually execute in CI.

Spec and docs — openspec/specs/release-preflight/spec.md and the "CI Check Auto-Discovery" section of docs/RELEASE_WORKFLOWS.md now describe how GitHub names checks: the job name (or job key) for direct jobs, <caller job> / <called job> for reusable workflow calls, and never the workflow's top-level name.

Checks on this push need a maintainer approval to run (fork PR), so I could not watch them complete. Locally: shellcheck, actionlint, yamllint, and markdownlint are clean on the changed files, and SonarCloud passes on 5d99da6.

@marcusburghardt marcusburghardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix the ineffective prefix assertion before merging. The check-name corrections and other regression assertions are verified.

This review was generated by /uf.review-pr (AI-assisted).

Comment thread tests/test_preflight_check_discovery.sh Resolved

@marcusburghardt marcusburghardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check-name fixes address #684, and my earlier review requests are resolved. Both preflight test suites passed locally.

Several GitHub Actions workflows are awaiting approval; please ensure they pass before merging.

This review was generated by /uf.review-pr (AI-assisted).

Comment thread docs/RELEASE_WORKFLOWS.md
For `ci_local.yml`, the preflight uses `yq` to extract:
1. The workflow `name:` field
2. Each job key and its optional `name:` field
- A direct job is reported as its `name`, or as the job key when the job declares no `name`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] Matrix jobs can include matrix values in their check-run names, so auto-discovery may not match them. Please mention using the ci_checks override for matrix jobs.

The preflight auto-discovery prepended the calling workflow's name to
each discovered check. The Checks API reports the bare job name for
direct jobs and "<calling job> / <called job>" for reusable workflow
calls, so the exact-match lookup never found the checks and blocked
releases.

Use the bare job name for ci_local.yml jobs and correct the hardcoded
ci_checks.yml linter name. Add a regression test covering both paths.

Fixes complytime#684

Assisted-by: Hermes Agent (longcat-2.5)
Signed-off-by: SatriaBagusNurhadianto <kenzaarthur@gmail.com>
tests/test_preflight_check_discovery.sh was missing from the path
filters, so a change to it alone would not run the preflight test job.

Assisted-by: Hermes Agent (longcat-2.5)
Signed-off-by: SatriaBagusNurhadianto <kenzaarthur@gmail.com>
The source-text assertions in test_preflight_check_discovery.sh could pass
with the expected statements commented out or unreachable, and they failed on
harmless renaming. They also tripped ShellCheck SC2016.

Run the real "Discover CI checks" step against fixture repositories instead,
covering named jobs, the job key fallback, the bare shared linter check and
the explicit ci_checks override. yq is installed in the preflight test job so
the ci_local.yml cases actually execute in CI.

Document how GitHub names checks: the job name (or job key) for direct jobs,
"<caller job> / <called job>" for reusable workflow calls, and never the
workflow's top-level name. Update the release-preflight spec and
RELEASE_WORKFLOWS.md accordingly.

Assisted-by: Hermes Agent (longcat-2.5)
Signed-off-by: Satria Bagus Nurhadianto <kenzaarthur@gmail.com>
@yvonnedevlinrh
yvonnedevlinrh force-pushed the fix/preflight-check-names branch from 3cea9c7 to f38111e Compare November 19, 2026 17:15

@yvonnedevlinrh yvonnedevlinrh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings found

@sonarqubecloud

Copy link
Copy Markdown

@yvonnedevlinrh
yvonnedevlinrh merged commit 49704ca into complytime:main Nov 19, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: preflight auto-discovery constructs incorrect check-run names

4 participants