docs(testing): hurl is scaffolding; target-native suites are the goal#52
Merged
Conversation
Captures the strategic note: hurl was the right v0.1 conformance gate (cross-target proof with no per-language re-authoring), but it isn't the long-term contract a team adopting candy will run in their day-to-day loop. Candy should generate the test suite in the target's idiomatic framework alongside the feature implementation. Establishes two test tiers per target: - Tier 1 (focus, integration): HTTP-level, language-native — Go testing + httptest, Rust #[tokio::test] + axum-test, vitest + hono.fetch, pytest + httpx. Each hurl scenario transliterates mechanically to one tier-1 test. - Tier 2 (supporting, unit): per-policy, per-flow, per-actor. Already proven on the Go and Rust auth targets — every spec policy example becomes a unit test. Hurl is demoted to its proper role: the cross-target reference contract. Teams running on a single target use only the language-native suite. Names spec-side evolutions that would enrich tier-1 generation beyond simple transliteration: `controller examples`, `flow properties`, `mock external X`, declarative `seed:` blocks. Marks all of these as not in immediate scope. The "diff between hurl and `_test.go`" framing is the adoption metric — a tool teams already-read becomes the default; a tool they have to learn stays niche. Same logic as the green/brown distinction in cli-modes.md, applied to the test surface. No code changes — strategy doc only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Captures the strategic note that hurl is not a long-term
requirement for candy. Hurl was the right v0.1 conformance gate
(cross-target proof, no per-language re-authoring), but it isn't
the test surface a team adopting candy will actually run in their
day-to-day loop.
Candy should generate the test suite in the target's idiomatic
test framework alongside the feature implementation.
Two test tiers per target
Each hurl scenario transliterates mechanically to one tier-1 test.
The codegen has the spec; it knows the routes, status mappings,
and expected body shapes — it can emit the target-native suite
from the same inputs.
Hurl's new role
Not deleted. Demoted to the cross-target reference contract.
A single hurl run still verifies a generated backend matches the
conformance contract. Teams running on a single target use the
language-native suite for daily work; hurl is the shared fence-line
for multi-target conformance.
Spec evolutions that would enrich tier-1 generation
Out of scope for this PR; flagged in the doc:
on controllers (today examples live on `policy` blocks).
stubbing for tier-1 tests.
data.
Out of scope
No code changes. No commitment to specific test framework choices
beyond the conventional defaults. The doc is a strategic framing;
concrete codegen prompts for tier-1 emission land in a follow-up.
Why this matters
The diff between "candy emits hurl I have to learn" and "candy
emits `_test.go` I already read every day" is the difference
between a niche tool and a default.
Combined with brownfield mode (PR #51), this gives candy a
realistic adoption story: applies to existing code, ships in your
team's existing test idiom.
Refs
brown adoption palatable).
targets yet"; tier-1 native suites are the natural CI surface.