Repository navigation
[DO NOT MERGE until v65 is released] Metabase CLI transform tests - #59
Merged
alxnddr merged 18 commits intoOct 2, 2026
Merged
Conversation
|
Published Install: |
alxnddr
force-pushed
the
fix/transform-test-ee-path
branch
from
September 28, 2026 18:24
a3ca6dd to
041c793
Compare
|
Published Install: |
alxnddr
force-pushed
the
fix/transform-test-ee-path
branch
from
September 28, 2026 19:40
041c793 to
8ac2a7a
Compare
|
Published Install: |
ranquild
approved these changes
Sep 28, 2026
alxnddr
changed the title
alxnddr
force-pushed
the
fix/transform-test-ee-path
branch
from
October 2, 2026 19:34
8ac2a7a to
b41b46c
Compare
|
Published Install: |
alxnddr
marked this pull request as ready for review
October 2, 2026 20:47
A transform test pins a transform's behaviour without touching real data: every table it reads is replaced by an input fixture, the transform runs into a temp table, and each expectation checks that output. `transform-test list/get/create/update/delete/run` over `/api/transform-test`, with the schemas the endpoint speaks — inputs as SQL or inline rows, expectations as `empty` or `equals`, and a run report naming what each expectation found. `run` exits non-zero when the test does not pass, so it drops into CI as it stands. Gated at v64, the first version serving the endpoint. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The server closes `::transform-test.create` and `::transform-test.update`, so a body cloned from `get` earned an opaque 400. Closing them here names the offending key instead; a stored test needs `id`, `entity_id`, `creator_id`, `created_at` and `updated_at` stripped before it can be sent back, which the command help, the README and the transform skill now say. `get` projects through the compact view, so `--full` is what shows `inputs` and `expectations` — the examples and docs advertise it. An `equals` expectation needs `format: "sql"` or `format: "rows"`, which the docs omitted, leaving a body written from them to fail on a three-branch union error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`equals` pins the exact rows; `empty` asserts an invariant the row data alone would not show. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…to test philosophy and design
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Declare each command's client method through requires in place of a minimum version, gate the transform-test skill passages and the transform-test-plan skill on transformTests, and name the enterprise endpoint in the README.
Input columns cast through `cast_type`; expectation SQL naming a table outside the output and declared inputs is refused with `transform-test.unremapped-reference` rather than reading the real table; MySQL reports `SIGNED` casts as `BIGINT`; the endpoint needs the `transforms-testing` premium feature.
Source: enterprise/backend/src/metabase_enterprise/transform_testing/{schema,validator,api}.clj and api_routes/routes.clj on origin/master (e28f66a5ac2).
…, and match the prose to the server A failed transform-test run throws FailedOutcomeError (category outcome, exit 1) instead of a bare Error the CLI reported as an unknown crash. transform-test run takes --timeout, passed through the resource method's request options, since the run endpoint is synchronous. The transform skill, core skill, README and command details describe equals as rows-only, say that only sql inputs touch the source database, name the testable transform kinds and drivers, warn about table-qualified columns, quote the server's duplicate-name refusal, and state that bodies are closed at every level; the check catalog notes the MySQL spelling of IS DISTINCT FROM.
…for every negative result FailedResultError (category failed, exit 1) replaces FailedOutcomeError, is exported from the client root and listed in its error taxonomy. transform run under --wait/--sync and every git-sync wait throw it for a run or task that ended anything but successful, so the CLI reports the result rather than an unknown crash. The transform skill says run exits 1, that an equals without a format is refused, and names the testable drivers as the whole list.
… over --max-bytes A failing run's --json report could pass the default cap, so the run that already happened ended as a config error with exit 2 and nothing printed. renderSummaryWithinMaxBytes hands the over-cap refusal back instead of throwing it; transform-test run then exits 1 with the expectations that did not pass and the cap message. The run-result view moves to output/views.
Matches transform runs --transform-id, and a malformed value names the flag.
…re core's dropped lines The plan skill says how to load its references and no longer claims every empty expectation passes on an empty output; its prose is unwrapped. The transform skill says equals compares only declared columns and that format sql can never pass. Core is main's text plus the transform-test entries.
Restoring main's core text left no room for the transform-test quirk, so core opened `skills get --all --json` on an empty window. The quirk goes (the transform skill carries it) and the transform-test-plan entry shrinks to one phrase.
It never reached a server: the v58 cases ran the binary against a seeded probe, re-checking the filtering the unit tier already covers, and it hard-coded the shipped skill list, so every new skill broke it. core/skills.test.ts loads and pins the shipped skills, and get.test.ts holds core under the default cap. SkillPathListEnvelope had no other importer and is no longer exported.
alxnddr
force-pushed
the
fix/transform-test-ee-path
branch
from
October 2, 2026 22:31
70b3702 to
efb54aa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds transform tests to the CLI