diff --git a/CHANGELOG.md b/CHANGELOG.md index 42b59a9bc9..cc35dc131a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,7 @@ All historical references to "CFWheels" in this changelog have been preserved fo - The Debian/Ubuntu `apt` install instructions now pipe the distribution key through `sudo gpg --dearmor` before writing `/usr/share/keyrings/wheels.gpg` instead of `tee`-ing it verbatim. The key published at `apt.wheels.dev/wheels.gpg` is ASCII-armored, and modern `apt` rejects an armored key in a `signed-by=` keyring with an "unsupported filetype" warning followed by `NO_PUBKEY` — so `apt update` failed signature verification and the install never worked. Corrected across the install guide, the CLI installation reference, the release-channels guide, the `apt.wheels.dev` landing page, and the `tools/distribution-drafts/` repo templates (#2838) - The `apt.wheels.dev` publishing template (`tools/distribution-drafts/apt-repo/`) no longer wipes the `stable` package index when a `bleeding-edge` snapshot publishes. `regenerate-apt-metadata.sh` rebuilt *both* channels on every run while the workflow synced only the dispatched channel's pool into the runner, so a frequent bleeding-edge publish scanned an empty local `pool/stable/`, produced an empty `Packages`, and the unscoped upload overwrote the good stable index on R2 — leaving `apt install wheels` with "Unable to locate package wheels" even though the `.deb` was present in the pool. The regen now honors a `CHANNELS` env (the workflow passes only the dispatched channel) and the upload is scoped to that channel's `dists/` subtree, so the two channels can no longer clobber each other (#2838) - The Wheels CLI test suite (`cli/lucli/tests/specs`, served at `/wheels/cli/tests`) is green again after the BDDRunner error-count fix unmasked 13 pre-existing failures the old `-1` bundle-error sentinel had been arithmetically cancelling (a negative error total netted real failures down to `<= 0`, so the CI gate read the suite as passing). The eight `*CommandSpec` bundles that instantiate `new cli.lucli.Module()` no longer fail to load with `can't find component [modules.BaseModule]`: a lightweight `BaseModule` test double under `cli/lucli/tests/_modules/` plus a `/modules` mapping (added alongside the existing `/modules/wheels`, which longest-prefix resolution keeps authoritative for the wheels module) lets `Module.cfc` instantiate under TestBox — resurrecting the Db/Info command specs as real behavioral coverage. The stale `AdminSpec` route assertion now expects `.namespace("admin")` (the service's current named-route-prefixed output) instead of the legacy `.scope(path="admin")`. Command specs that need the LuCLI runtime, a running Wheels server, CodeGen harness fixtures, or the CLI bash wrapper (Deploy/Destroy/Generate/Packages, plus the server-dependent Migrate/Test cases) and the unbuilt-feature specs (Doctor #2260 mixin-detail, Scaffold route-model-binding) are `xdescribe`/`xit`-skipped with documented reasons, pending a command-by-command CLI test audit. Finally, `tools/ci/run-tests.sh` now clamps a negative error count for its pass/fail decision and fails explicitly when it sees one, so this masking class of bug can never silently turn a red suite green again (#2829) +- WheelsTest BDD runner now captures spec-load and bundle-execution errors against the offending bundle instead of bubbling out as an anonymous `BundleRunnerMajorException`, and reports the resulting error count as a positive number (was the `-1` sentinel) so summaries read "1 error(s)" with the bundle path and `globalException` populated — covers both `it()` called outside a `describe()` body and a `beforeAll()` that throws during spec load (#2829) ---- diff --git a/vendor/wheels/tests/resources/wheelstest/BeforeAllErrorFixture.cfc b/vendor/wheels/tests/resources/wheelstest/BeforeAllErrorFixture.cfc new file mode 100644 index 0000000000..8b5e349270 --- /dev/null +++ b/vendor/wheels/tests/resources/wheelstest/BeforeAllErrorFixture.cfc @@ -0,0 +1,18 @@ +component extends="wheels.WheelsTest" { + + function beforeAll() { + throw( + type = "Test.Setup.MissingDep", + message = "application.wo.functionDoesNotExist is undefined" + ); + } + + function run() { + describe("foo", function() { + it("bar", function() { + expect(true).toBeTrue(); + }); + }); + } + +} diff --git a/vendor/wheels/tests/resources/wheelstest/OrphanItFixture.cfc b/vendor/wheels/tests/resources/wheelstest/OrphanItFixture.cfc new file mode 100644 index 0000000000..2e51082251 --- /dev/null +++ b/vendor/wheels/tests/resources/wheelstest/OrphanItFixture.cfc @@ -0,0 +1,9 @@ +component extends="wheels.WheelsTest" { + + function run() { + it("orphan", function() { + expect(true).toBeTrue(); + }); + } + +} diff --git a/vendor/wheels/tests/specs/wheelstest/BDDRunnerErrorReportingSpec.cfc b/vendor/wheels/tests/specs/wheelstest/BDDRunnerErrorReportingSpec.cfc new file mode 100644 index 0000000000..bf2970e0a5 --- /dev/null +++ b/vendor/wheels/tests/specs/wheelstest/BDDRunnerErrorReportingSpec.cfc @@ -0,0 +1,51 @@ +component extends="wheels.WheelsTest" { + + function run() { + + describe("BDDRunner load/run-time error reporting", function() { + + it("captures an it() call outside describe() against the bundle instead of bubbling out", function() { + // Reproduces issue #2829, case 1: a spec whose run() calls + // it() at the top level (no enclosing describe) currently + // bubbles up as a BundleRunnerMajorException, leaving the + // CLI summary at "0 passed" with no filename or message. + var testBox = new wheels.wheelstest.system.TestBox( + bundles = ["wheels.tests.resources.wheelstest.OrphanItFixture"] + ); + var state = {threw: false, results: ""}; + try { + state.results = testBox.runRaw(); + } catch (any e) { + state.threw = true; + } + expect(state.threw).toBeFalse(); + expect(isObject(state.results)).toBeTrue(); + expect(state.results.getTotalError()).toBe(1); + var bs = state.results.getBundleStats(); + expect(arrayLen(bs)).toBeGT(0); + expect(bs[1].totalError).toBe(1); + expect(bs[1].path).toInclude("OrphanItFixture"); + expect(isStruct(bs[1].globalException) || isObject(bs[1].globalException)).toBeTrue(); + }); + + it("uses a positive error count when beforeAll() throws during spec load", function() { + // Reproduces issue #2829, case 2: a spec whose beforeAll() + // throws records totalError = -1, which sums into the global + // count as "-1 error(s)" with no file context. + var testBox = new wheels.wheelstest.system.TestBox( + bundles = ["wheels.tests.resources.wheelstest.BeforeAllErrorFixture"] + ); + var results = testBox.runRaw(); + expect(results.getTotalError()).toBe(1); + var bs = results.getBundleStats(); + expect(arrayLen(bs)).toBeGT(0); + expect(bs[1].totalError).toBe(1); + expect(bs[1].path).toInclude("BeforeAllErrorFixture"); + expect(isStruct(bs[1].globalException) || isObject(bs[1].globalException)).toBeTrue(); + }); + + }); + + } + +} diff --git a/vendor/wheels/wheelstest/system/runners/BDDRunner.cfc b/vendor/wheels/wheelstest/system/runners/BDDRunner.cfc index d98c9218ec..70d80270ea 100755 --- a/vendor/wheels/wheelstest/system/runners/BDDRunner.cfc +++ b/vendor/wheels/wheelstest/system/runners/BDDRunner.cfc @@ -47,25 +47,33 @@ component structKeyExists( tartetAnnotations, "displayName" ) ? tartetAnnotations.displayname : targetMD.name ); - // Execute the suite descriptors - arguments.target.run( testResults = arguments.testResults, testbox = variables.testbox ); - - // Discover the test suite data to use for testing - var testSuites = getTestSuites( arguments.target, targetMD ); - var testSuitesCount = arrayLen( testSuites ); - - // Start recording stats for this bundle + // Start recording stats for this bundle BEFORE invoking the target, + // so a throw from `target.run()` (e.g. `it()` called outside a + // `describe()` body — see issue #2829) can be recorded against this + // bundle instead of bubbling out anonymously as a + // `BundleRunnerMajorException` with no file context. var bundleStats = arguments.testResults.startBundleStats( bundlePath = targetMD.name, name = bundleName ); - // Verify we can run this bundle - if ( - canRunBundle( - bundlePath = targetMD.name, - testResults = arguments.testResults, - targetMD = targetMD - ) - ) { - try { + // Wrap the suite-descriptor pass and bundle execution in a single + // try so both load-time errors (target.run, getTestSuites) and + // run-time errors (beforeAll/afterAll, suite iteration) land in the + // same catch and report the offending bundle. + try { + // Execute the suite descriptors + arguments.target.run( testResults = arguments.testResults, testbox = variables.testbox ); + + // Discover the test suite data to use for testing + var testSuites = getTestSuites( arguments.target, targetMD ); + var testSuitesCount = arrayLen( testSuites ); + + // Verify we can run this bundle + if ( + canRunBundle( + bundlePath = targetMD.name, + testResults = arguments.testResults, + targetMD = targetMD + ) + ) { // execute beforeAll() for this bundle, no matter how many suites they have. if ( structKeyExists( arguments.target, "beforeAll" ) ) { arguments.target.beforeAll(); @@ -142,26 +150,28 @@ component for ( var afterAllMethod in afterAllAnnotationMethods ) { invoke( arguments.target, "#afterAllMethod.name#" ); } - } catch ( Any e ) { - bundleStats.globalException = e; - // For a righteous man falls seven times, and rises (tests) again :) - // The amount doesn't matter, nothing can run at this point, failure with before/after aspects that need fixing - bundleStats.totalError = -1; - arguments.testResults.incrementStat( type = "error", count = bundleStats.totalError ); - - // Module call backs - variables.testbox.announceToModules( - "onSuiteError", - [ - e, - arguments.target, - arguments.testResults, - isNull( thisSuite ) ? {} : thisSuite - ] - ); } + // end if we can run bundle + } catch ( Any e ) { + bundleStats.globalException = e; + // Use a positive count so the summary reports "1 error(s)" rather + // than the legacy "-1 error(s)" sentinel — issue #2829. The bundle's + // globalException still carries the underlying throw for tooling + // that wants the full detail. + bundleStats.totalError = 1; + arguments.testResults.incrementStat( type = "error", count = 1 ); + + // Module call backs + variables.testbox.announceToModules( + "onSuiteError", + [ + e, + arguments.target, + arguments.testResults, + isNull( thisSuite ) ? {} : thisSuite + ] + ); } - // end if we can run bundle // finalize the bundle stats arguments.testResults.endStats( bundleStats ); diff --git a/web/sites/guides/src/content/docs/v4-0-0/testing/running-tests-locally.mdx b/web/sites/guides/src/content/docs/v4-0-0/testing/running-tests-locally.mdx index ab6ce0f783..7397994cc3 100644 --- a/web/sites/guides/src/content/docs/v4-0-0/testing/running-tests-locally.mdx +++ b/web/sites/guides/src/content/docs/v4-0-0/testing/running-tests-locally.mdx @@ -202,6 +202,7 @@ These are the errors you'll actually hit. Fix-ups first, diagnosis second. - **Populate SQL errors** — read the error. "Table already exists" means a stale table from a previous engine-specific run; add a `DROP TABLE IF EXISTS` first. "Syntax error near CURRENT_TIMESTAMP" means cross-engine SQL drift; use `NOW()` — it works on every supported engine. - **Flaky `wheels new` in parallel harness runs** — a known LuCLI race (framework gap tracker #11). Retry once, or serialize the runs until the fix lands. - **Server not responding after `compose up`** — engines need 30–90 seconds to cold-start. `curl -I http://localhost:60007/` returns connection-refused until the engine is ready. If it stays down after two minutes, check `docker compose logs `. +- **"1 error(s)" with a bundle path but no failing assertion** — the BDD runner caught a load-time or setup error for that spec file. Two common causes: (1) `it()` called directly inside `run()` without an enclosing `describe()` block, or (2) `beforeAll()` throwing an exception. The runner captures both against the bundle and populates `globalException` — check the spec file at the path shown in the summary for the underlying throw. ## Cleanup