ci: add live test suite workflow - #681
Draft
0xisk wants to merge 31 commits into
Draft
Conversation
Signed-off-by: 0xisk <0xisk@proton.me>
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
0xisk
marked this pull request as draft
July 15, 2026 13:00
8 tasks
0xisk
force-pushed
the
test/multisig-live-harness
branch
2 times, most recently
from
July 17, 2026 16:14
a45bda7 to
7f6e09e
Compare
0xisk
force-pushed
the
ci/live-test-workflow
branch
from
July 20, 2026 09:37
4ea60c7 to
0ffe584
Compare
The live workflow needs two decisions that are not the runner's: which targets to fan out into matrix jobs, and what a nightly result does to the `live-nightly` tracking issue. Both were `run:` shell inside the workflow, where nothing can test them and nothing can run them locally, so a mistake in either stays hidden until a nightly reports the wrong thing. `scripts/live-ci.ts` takes them over, following `test-live.ts`: node runs the .ts directly and it imports only node builtins, so the jobs calling it need a checkout and a Node, not an install. * `ci/matrix.ts` resolves a requested scope against the runner's own target list, so the workflow keeps no copy of it. An unknown name is rejected in the plan job, naming the valid ones, instead of dying hours later inside the runner. * `ci/nightly.ts` decides open / comment / close, including the case the shell version also had to special-case: a suite job the plan job never reached reports `skipped`, and a nightly that tested nothing is a failed nightly. * `ci/gh.ts` builds every `gh` call as argv, so a markdown issue body is never re-interpreted by a shell. Extracted as node-run TS rather than shell scripts: the repo has no bash test harness, and this reuses the vitest project the live orchestrator's own tests already run under. That project had no runner of its own, though: no package script, no turbo task, no CI step. Wire it up as `yarn test:scripts` and run it in the test workflow, so these tests and `scripts/live`'s execute on every PR.
The live suite runs against a real local stack with real ZK proofs, so a run takes hours rather than minutes. Give it its own workflow instead of a place in the required PR checks: * nightly on main, as the regression net (a failure opens or comments on a `live-nightly` tracking issue, which closes on the next green run) * on demand via workflow_dispatch, optionally scoped to one target or to a file filter within it * on a PR, opt-in, by applying the `live-tests` label A plan job asks the runner which live targets exist and fans out one job per target, so each gets its own runner, its own stack, and its own 6-hour job budget. Never a required check. The `target` input is a free-form string rather than a `choice`: the valid set is whatever `scripts/live/targets.ts` discovers, and a dropdown enumerated here would go stale as categories join. The plan job validates the value and fails in seconds with the valid names. Every `run:` is a one-liner. The two that are not runner invocations call `scripts/live-ci.ts`, which is unit tested. Split out of #673 per review: the workflow is large enough to land on its own rather than alongside the test harness.
0xisk
force-pushed
the
ci/live-test-workflow
branch
from
August 24, 2026 10:23
0ffe584 to
19e275b
Compare
Five items from the review of this branch. The first two share `ci/matrix.ts`, its tests, and `live.yml`, so they land together. * A file filter now narrows the matrix, not just the run. A filter is a substring of a path, so it matches some targets and not others, and a target that runs no file is an infrastructure abort in the runner rather than a pass. `target: all` with `filter: Forwarder` used to queue a red job for every target the filter misses, each paying for `env-up` and a compile first. A filter that matches nothing anywhere now fails the plan job in seconds. `ci/specs.ts` lists a target's spec files for that check. * The `live-tests` PR label fanned out every target, which is 7 jobs of up to 6 hours for one label. `live-tests:<target>` scopes it to one, and the bare label still means all of them. * `scripts/` was never type-checked: `yarn types` covers only `contracts/src/**/test/witnesses`, and both ways these files run (node's type stripping, vitest's transform) discard types without checking them. Add `scripts/tsconfig.json` and a `types:scripts` step to `yarn types`, so a type error in the orchestrator or in these CI helpers is caught on every PR. * An unreadable `gh issue list` response threw a bare `SyntaxError` naming neither the command nor the output. It should not happen (`gh` keeps warnings on stderr), which is exactly why the message has to say what did. * The `setOutput` tests mutated `process.env` directly. `vi.stubEnv` keeps the change scoped to the test.
Three findings from the second review, all about how a file filter is matched. The runner passed a name filter to vitest as-is, and vitest matches a positional filter against the whole project include, which for `unit-live` is every `src/**` spec. So `yarn test:live token Forwarder` ran the multisig Forwarder spec too, and in CI, where each target is its own job, every target the filter matched ran the same union of files and reported it under its own name. Resolve the filter against the target's own spec files instead and hand vitest explicit paths. A target the filter matches nothing under is skipped rather than run: an empty filter list would run the target's whole include glob. `ci/specs.ts` moves to `live/specs.ts` for that: spec discovery belongs to the runner, and the plan job and the runner now share one matching rule (`filterSpecFiles`). That rule is vitest's own, which is case-insensitive (`TestProject.filterFiles` lowercases both sides). The plan job matched case-sensitively, so `filter: forwarder` ran locally but was rejected in CI as matching nothing. A `live-tests:` label with nothing after the colon passes the workflow's `startsWith` gate and used to resolve to the empty scope, meaning every target: a malformed label would have queued the full fan-out. It is rejected now, naming the valid targets.
0xisk
marked this pull request as ready for review
August 24, 2026 12:14
Fork-only merge, so the live workflow can be exercised on real runners from 0xisk/compact-contracts. The fork's main carried a one-line .gitattributes predating upstream's; upstream's version is a superset of it (same linguist-language line, plus linguist-detectable and the reasoning), so the conflict resolves to upstream's file.
0xisk
marked this pull request as draft
August 24, 2026 13:12
ci: add live test suite workflow
A full fan-out on a cold compiler cache loses jobs to setup, not to tests. Observed on a 7-target run: `live-security` died in `Setup Environment` with "Failed to update ... Error while fetching compact releases / Serde Error: expected value at line 1 column 1", then "No default compiler set", while the other six went on to run. `setup-compact-action` installs the compiler only on a cache miss, but runs `compact update` unconditionally, and that query hits the GitHub API. On a cache hit the query may fail harmlessly, since the action swallows it and the compiler is already installed. On a miss there is nothing to fall back on, so the same throttle kills the job. Seven jobs starting at once means seven cold installs racing that one API. Add a short `warm-compiler-cache` job between `plan` and the matrix that installs the compiler once, so every suite job starts from a cache hit and stays on the tolerant path. It is `continue-on-error`: if it fails, the matrix still runs and each job installs its own compiler, exactly as before. The composite setup action gains a `compact-only` input for it, which keeps the compiler version in one place. It also skips the turbo cache step: that entry is keyed on the commit, so saving it from a job that compiles nothing would leave every later job in the run restoring an empty cache instead of falling back to a populated one from an earlier commit.
`compile:integration` declared no compile dependency and took its inputs from `test/integration/**/*.compact` alone, but the mocks under `test/integration/_mocks/` import `src/` sources directly: ConfidentialFungibleToken, its PublicSupply extension, FungibleToken, NonFungibleToken, crypto/ElGamal, security/Initializable. So a change to any of those left the task hash untouched and turbo served the previous mock artifacts from cache. The composed contract under test was then built from stale sources while the plain src artifacts beside it were current. Depending on `compile` fixes the invalidation: turbo folds a dependency's task hash into the dependent's, so any src `.compact` edit now reaches `compile:integration` through the category task that owns it. Naming the aggregate rather than the individual categories also keeps a future mock import from silently reintroducing the gap. Consequence for the dry path: `yarn test:integration` now compiles all of `src` first. It runs under `SKIP_ZK=true`, so this is source compilation with no key generation, and every category is already a cache hit from `yarn test`.
Every live run compiled the whole repo with real proving keys, so each CI matrix job paid ~20 minutes of key generation for six categories it never deploys: on the 7-target fan-out that is 7 full-key compiles per run, and locally `yarn test:live crypto` spent 1 minute on its own artifacts and 10+ on everyone else's. Resolve the plan into a compile scope instead (`compileScope` in targets.ts) and hand that to the ArtifactCompiler: * a scoped category runs `compile:<category>`; turbo's `dependsOn` builds the categories the slice imports, and nothing else. Cross- category invalidation is sound since the compile:integration dependency fix (previous commit); the remaining category edges were verified against the actual `.compact` imports. * the integration target runs `compile:integration` alone, which now pulls the full `compile` through turbo. * an unscoped run keeps today's behavior (`compile`, verify `src/`). The truncated-key scan narrows with it, to the trees the plan's specs deploy: composition is compile-time, so dependency artifacts are build byproducts nothing deploys, and scanning them from an unrelated target could only false-positive on another slice's stale keys. The one cross-category deploy is encoded (token fixtures deploy the crypto mocks MockElGamal/MockEcdhMask), verified by grepping every category's artifact references. Per-target compile also keeps each matrix job's failure isolated: a compile error in one category no longer costs the other six jobs their compile step, and each job starts testing as soon as its own slice is built, with no cross-job barrier.
One job per target meant one long serial run per target: the 2026-08-24 nightly spent 5.75h in `live-token`, and its verdict named a category rather than a file. The spec files do not need to share a job — every one of them resets the stack anyway — so run them apart. Three layers now: * plan resolves both matrices, publishing `targets` (7) and one leg per spec file (46). Legs carry the full spec path, which matches exactly one file, and a name unique within the target: the shared directory prefix is stripped, so `multisig` gets `ForwarderPrivate` and `presets-ForwarderPrivate` rather than one name for two jobs. * compile builds each target's slice once with real ZK keys and uploads it. Without this layer the per-file fan-out would repeat a target's key generation once per file, 16 times over for `token`. * the suite runs one spec file per job against the downloaded tree. Runner-side, `--compile-only` builds the plan's scope and stops, and `--prebuilt` verifies a tree it did not build and refuses to build one. Prebuilt verification adds a completeness check (`missingKeyArtifacts`) next to the truncated-key scan: a partial download leaves contracts absent rather than 0-byte, which the key scan alone reads as clean, and both end as a deploy failure in `beforeAll` that vitest turns into a silent whole-suite skip. Also here, because the fan-out forced them: * Turbo cache keys are per target. One key shared by seven compile jobs is a write race: they all save at the end, one wins, and everyone else's slice is lost to whoever restores next. * The nightly report reads `needs.compile.result` too. A compile failure skips the suite, and a skipped suite under a green plan used to report as `skipped` — a broken build silent until someone opened the Actions tab. * `warm-compiler-cache` is gone: the compile layer installs the compiler once for the whole run, which is what that job existed to do. Its `compact-only` setup input goes with it, replaced by `turbo-cache-key`. `needs` is job-level in Actions, so the suite matrix waits for the whole compile matrix rather than for its own target's job. That barrier is the slowest compile against a suite measured in hours, and it buys every job byte-identical artifacts.
One compile matrix feeding one suite matrix made every spec file wait
for the slowest compile of every target, because `needs` in Actions is
job-level and there is no per-matrix-leg dependency edge. Run
32810453735 shows the cost: `utils` compiled in 41s, `crypto` in 67s and
`security` in 82s, while `token` and `integration` ran for many minutes,
so 15 spec jobs sat idle behind targets they share nothing with.
Pair each target instead: `compile-<target>` feeds `live-<target>`, a
matrix over that target's spec files. A target's specs now start the
moment that target is built, and a target whose compile fails skips its
own spec jobs and no others.
The pairs have to be literal YAML, so the steps move into two reusable
workflows (`live-compile.yml`, `live-suite.yml`) and the 14 jobs stay
declarations rather than 14 copies of a job body. The plan job publishes
one `legs-<target>` output per target, since output names must be
literals too.
The one property this gives up is the fully dynamic target list: a new
`src/` category no longer joins the live run on its own. `KNOWN_TARGETS`
in the workflow is what it must be added to, and the plan job now fails
in seconds naming any discovered target with no pair, so the gap cannot
be silent.
Also here:
* `worstResult` collapses the seven per-target results the nightly
report now receives as a joined list. Severity order matters: one
failure outweighs six passing targets, cancelled only wins when
nothing failed, and a mix of success and skipped is a success, since
a scoped run legitimately skips the other six targets.
* Hyphenated names inside expressions use bracket notation
(`needs['compile-token']`): `-` is also the subtraction operator, so a
hyphenated name after a `.` is ambiguous by grammar.
* Each `live-<target>` job carries `${{ matrix.name }}` in its name. A
caller job with a fixed name renders every leg of its matrix
identically, which would hide which spec a red job ran.
* Reusable workflows do not inherit the caller's `env`, so
`TURBO_TELEMETRY_DISABLED` is set in both of them.
`unit-live` excludes `src/**/test/witnesses/**` (LIVE_EXCLUDE in vitest.config): those specs build a fabricated WitnessContext and assert on the private-state helpers directly, so a live backend changes nothing about them. `specFiles()` did not know that. A target-wide run never noticed, since vitest applied its own exclude and filtered them out silently, but CI now gives each spec file its own job — and a job whose only file is excluded matches nothing, which the runner reports as an infrastructure abort: No test files found, exiting with code 0 no test file matched across token (filter: src/token/test/witnesses/ ConfidentialFungibleTokenWitnesses.test.ts) — nothing ran, so there is no result to report. ##[error]Process completed with exit code 2 Observed on run 32811648017, which would have failed all seven of them (3 under access, 4 under token). Apply the exclude where the file list is produced, so the matrix can only ever name a file its project runs: 46 legs become 39.
`compile-integration` died in `Setup Environment` on run 32812144831: API rate limit exceeded for 20.25.10.69 Error: Failed to update ... Error: Failed to run compactc Caused by: No default compiler set Not a cache miss. The job restored `compact-0.31.0-Linux-X64` fine. `setup-compact-action` passes `github.token` to its install step but not to its update step, so the `compact update` it runs on every job goes out unauthenticated: 60 requests an hour per runner egress IP, shared by every job starting at once. A cache hit is the fragile path here, not the safe one. It skips the install step, so nothing has set the default compiler, and `compact update` is what sets it. `|| true` swallows the update's own failure and the following `compact compile --version` is what fails the job. Six targets got their API call in before the ceiling; the seventh did not. Exporting `GITHUB_TOKEN` on the compile job authenticates that query (1000 requests an hour) — a composite action's steps inherit the calling job's environment, which the failing log confirms by echoing this workflow's `TURBO_TELEMETRY_DISABLED` inside the action's own step. The upstream fix is for the action's update step to take `github.token` the way its install step already does. The suite jobs need nothing here: they pass `skip-compact` already, so only the compile jobs touch that API at all.
`setup-node`'s `cache: yarn` restores `.yarn/cache` alone, so every job
unpacked it into `node_modules` and rebuilt the native modules from
source. Run 32815780786 shows it in each compile job:
Cache hit for: node-cache-Linux-x64-yarn-...
yarn install --immutable
msgpackr-extract must be built because it never has been before
classic-level must be built ...
cpu-features must be built ...
ssh2 must be built ...
Completed in 23s 516ms
24s of a 38s setup, repeated in all 46 jobs the live run now fans out
into, plus every job of checks.yml and test.yml.
Cache `node_modules` (and the workspace's) alongside the zips instead, so
the install has nothing to do. Measured with zstd, the level
actions/cache uses: `node_modules` is 137 MB against `.yarn/cache`'s
138 MB, so the entry roughly doubles in size to buy back the whole
install. That trade is clearly right — the turbo cache is already 275 MB
and restores in about 3s.
The zips stay in the entry because they are what a miss needs: dropping
them would send every job of a lockfile-changing run to the network. The
key is exact, with no `restore-keys`, since a tree restored from a
different lockfile is worse than no tree — `--immutable` would only have
to undo it. A miss therefore behaves exactly as it did before.
`yarn install --immutable` still runs on a hit. It is a fast no-op there,
and it is what makes a partial restore fail in setup rather than inside a
spec.
A commit-keyed entry is written on every push whether or not anything it caches changed, and these entries are large: `token`'s slice is ~275 MB of ZK keys and `multisig`'s ~213 MB. Five commits had therefore left five copies of each, and the repo sat at 6.5 GB of its 10 GB budget with a 3.8 GB entry still there from before the keys went per-target. That ceiling is not harmless. Past it GitHub evicts least-recently-used, which would drop the dependency-tree cache added in the previous commit and quietly undo the 24s per job it saves. Key on the contract sources and the pinned compiler version instead, so a docs-only or test-only commit reuses the existing entry rather than duplicating it. `restore-keys` stays: a source change should start from the previous build, and turbo's store is content-addressed, so every task whose own inputs did not move is still a hit. The stale 3.8 GB entry and the 30 commit-keyed ones are deleted; the repo is at 0.4 GB.
Restoring `node_modules` took the install from 24s to 9s, but run 32818574868 still reported every native module as needing a build on a full cache hit: Cache restored from key: Linux-deps-cf82a765... msgpackr-extract must be built because it never has been before classic-level must be built ... cpu-features must be built ... ssh2 must be built ... Done in 9s 266ms The record of a completed build is not in `node_modules`. Yarn 4 keeps it in `.yarn/install-state.gz`, which the entry did not cover, so every job recompiled the same five packages from source with nothing to tell it they were already done. It is 534 KB, so adding it costs nothing measurable and removes the last of the install.
`compile:integration` depended on `compile`, so the job that builds five mock files first generated real ZK keys for all 68 sources under `src`. On run 32819706075 that made `compile-integration` the only compile job still running after seven minutes while its six siblings finished inside a minute, and it is the barrier the whole integration lane waits on. The dependency was never needed for the build. `compile:integration` is `compact-compiler --src test/integration/_mocks`, and the Compact compiler follows each mock's imports into `src/*.compact` from source, emitting one artifact per top-level file. It reads sources, not sibling artifacts. The integration specs then deploy only the `Composed*` mocks. It was needed for cache invalidation, which is what the dependency was added for: the mocks import `src/token`, `src/crypto` and `src/security`, so a change there must rebuild them, and the task's `inputs` covered only `test/integration/**`. Naming those sources as inputs achieves the same invalidation directly — a src `.compact` edit changes the task hash — without dragging six categories of key generation into the job. Conservative on purpose: any src change invalidates the mocks, including one they do not import. That is the safe direction, and it costs a five-file rebuild.
A live test costs ~55s (deploy-per-test against a real node), so one
big spec file is one long serial job: ShieldedAccessControl.test.ts
(89 tests) ran 85+ minutes and set the whole run's wall clock, and
Signer.test.ts (57 tests) ran 48 minutes. Splitting per file cannot
help those; splitting within a file can.
New rule, driven only by MAX_TESTS_PER_LEG (scripts/ci/split.ts): a
file over 30 statically counted tests is parsed into its describe
tree and bin-packed by describe blocks into legs of at most 30, each
leg running the same file under a vitest -t (testNamePattern) regex.
The plan smoke now yields 63 legs instead of 46 (ShieldedAccessControl
4, Signer 2, nonFungibleToken 6, ...), the largest at 30 tests.
The filter format is pinned against the installed vitest 4.1.10: -t
becomes new RegExp(pattern) matched against the SPACE-joined describe
path plus test name (getTaskFullName in @vitest/runner), not the
" > " the reporters print. Each leg is an alternation of ^-anchored,
regex-escaped describe paths with a trailing space, so `grantRole `
matches neither _grantRole nor grantRoleExtra. Tests sitting beside
child describes get a remainder alternative: the parent path with a
lookahead excluding every literal child.
A wrong filter is worse than a long leg, so anything inexact falls
back to not splitting the file at all: a corrupted scan (unbalanced
brackets), a dynamic describe name (template `${}`, variable,
describe.each) whose remainder exceeds the limit, colliding sibling
prefixes, or unit counts that do not add back up to the file total.
The pattern threads as MIDNIGHT_LIVE_TEST_PATTERN: live-suite.yml
passes the leg's testFilter into the run step env, test-live.ts
validates it (new RegExp, else exit 2) and hands it to VitestRunner,
which appends -t to every spawn, so the round-2 flake re-run inherits
the same slice automatically. Vitest is silently green when -t
matches nothing (file "passed", all tests skipped), so the
orchestrator gains a guard: on a clean run the pattern must match at
least one reported test NAME. Names are reported for skipped tests
too, which is what tells a stale pattern (abort, exit 2) apart from a
slice that is legitimately all runtime-skipped on live (.skipIf,
itDryOnly) and still passes.
Known caveat, accepted: packing counts statically, so an it.each or
describe.each table counts once and property-based files under-split
relative to their runtime test count. Packing is a budget, not an
invariant; every test still runs exactly once.
actions/cache never overwrites an existing key, so a change to the cached path list keeps restoring the old, incomplete entry until the lockfile happens to move. A manual v1 token in the key is what gets bumped with any path-list change. A follow-up step turns a miss into a ::warning:: annotation: expected only after a lockfile change, anything else means eviction or key drift, and the job silently pays the full install otherwise.
With one suite job per spec file (and per split slice), the bare label fans out into 60+ checks on the PR — never what casually applying a label means. The plan job now rejects it with the scoped live-tests:<target> form and the valid target names; the full fan-out stays reachable via a deliberate dispatch with target 'all'. The workflow gate keeps accepting the bare label on purpose, so the author sees the red plan job with the message instead of silence.
GitHub evicts least-recently-used past the 10 GB repo budget, which is the worst policy for this workflow: the evicted entry is whichever one the next run needed. The nightly report job now prunes deliberately instead (scripts/ci/caches.ts, pure decision + gh api adapter): * an entry superseded within its group — same hash-stripped key prefix AND same ref, since restores are branch-scoped — is dead weight the moment a newer sibling exists (actions/cache restores the exact key or the newest prefix match, never an older one); * any entry not restored in 7 days goes whatever its group looks like. The step runs after the tracking-issue report on purpose, so a gh hiccup can never block the report, and the job gains actions: write.
Run 32831811290's 11 failed legs were all known-deterministic rejections — the block-limit deploy ceiling (node error 1010) and the unclaimed-shielded-output rule (ledger Custom error: 186, audit H-03). Both are properties of the transaction, not of node state, so the round-2 re-run on a fresh node returns the same verdict and only doubles the loss (a second stack reset plus the full re-run). A file now skips round 2 when EVERY failed test in its round-1 report matches one of the patterns in scripts/live/deterministic.ts, and reports REAL immediately with the cause named in the verdict block (exit code unchanged). Any unmatched or message-less failure — hook crashes report no failed assertions at all — keeps the file on today's round-2 path, so an unknown failure never loses its flake check.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
The split rule budgeted legs by test count, but a live test's cost is its transaction count times ~18s of indexer finality, which varies 3x across files: run 32831811290's MultiToken legs of <=30 tests ran 70-91 minutes (~3 min/test) while most files run ~1 min/test. Count budgeting under-split exactly the heavy files. The plan job now downloads the newest completed run's live-reports-* artifacts (best-effort: gh CLI only, continue-on-error, and the fetch command itself never fails — no history means count packing) and the packer weighs each test by its measured duration: * scripts/ci/weights.ts — reports → per-file (fullName → max ms), suffix-matched to the plan's contracts-relative paths so same-named describes in different files cannot inflate each other; * scripts/ci/history.ts — the gh run list/download adapter; * scripts/ci/split.ts — the budget becomes MAX_LEG_MS, derived as MAX_TESTS_PER_LEG x DEFAULT_TEST_MS so that a file with no matching history packs EXACTLY as the old count rule did (uniform weights scale every comparison; pinned by tests and by a byte-identical legs-output diff against the old code). An indivisible unit whose MEASURED weight exceeds the budget becomes its own oversized leg — refusing, as the count rule does for assumed weights, would collapse the whole file into one longer leg (MultiToken: 12 legs vs 331 min); * the plan log prints each leg's estimated minutes; the estimate is stripped before setOutput, so the published matrix shape and the suite workflow inputs are unchanged. With the fabricated 3 min/test history, MultiToken fans out into 12 legs (largest ~42m, an oversized measured leaf) instead of 5 count legs of up to ~90 min.
The label path could only name one target, so asking a PR for the full fan-out meant one label per target. The concurrency group is keyed by PR and event rather than by label, so those seven runs cancelled each other down to whichever label was applied last. `all` was already the dispatch input's word for every target; it just never reached the label path, which handed its suffix straight to the target lookup and rejected `all` as an unknown one. * map the label suffix `all` to "every target", as the dispatch input does * point the bare-label rejection at `live-tests:all` rather than at a dispatch, since the label form now covers it
CodeQL flags a job that inherits the repository's default GITHUB_TOKEN scope, and the 14 target pipelines are `uses:` declarations with no `permissions:` block of their own — one alert each on #681. Set the floor once at the workflow level rather than repeating an identical block across 14 jobs whose whole point is to stay one-liners. `plan` and `report-nightly` keep the wider scopes they declare, since a job-level block replaces the workflow one rather than adding to it, and both reusable workflows already pin `contents: read` on their own jobs.
The org shares 60 concurrent jobs on the Team plan and run 32941225488 peaked at 58 of them for about an hour, so an uncapped live run locks every other repo's CI out. Actions has no global cap, only per-matrix, so the bound is the sum of the seven per-target caps: ~28 concurrent on the nightly, ~14 on a PR label or dispatch. Held-back legs queue without holding a slot. Wall clock stretches to roughly 3-4h from 1.7h; the 350-minute budget is per leg, so it is unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Types of changes
What types of changes does your code introduce to OpenZeppelin Midnight Contracts?
Split out of #673 per review: the live CI workflow is large and complex, so it
lands on its own instead of alongside the test harness.
Stacked PR. Based on
test/multisig-live-harness(#673), so the diff hereis just
live.ymland its "Live tests in CI" CONTRIBUTING subsection. Once#673 merges to
main, this retargets tomainautomatically.Adds
.github/workflows/live.yml, which runs the live suite separately fromthe required PR checks:
main(regression net; a failure opens/comments alive-nightlytracking issue that auto-closes on the next green run).workflow_dispatch, optionally scoped withcategory/filterinputs.live-testslabel.PR Checklist