feat(sdk): adapter conformance testkit (@ratel-ai/sdk/testkit)#116
Merged
Conversation
rstagi
force-pushed
the
feat/adapter-testkit
branch
2 times, most recently
from
July 20, 2026 16:46
316cb7e to
3750d7b
Compare
exports map (. byte-identical to main/types, ./testkit, ./package.json), typedoc second entry point, CHANGELOG + README testkit section.
rstagi
force-pushed
the
feat/adapter-testkit
branch
from
July 20, 2026 18:13
3750d7b to
ec6f7e6
Compare
rstagi
added a commit
that referenced
this pull request
Jul 20, 2026
* refactor(sdk): extract reference adapter to testkit/reference-adapter * feat(sdk): testkit types, case builder skeleton, runner helper * feat(sdk): conformance battery cases, discrimination tests, green path * feat(sdk): package testkit as @ratel-ai/sdk/testkit subpath exports map (. byte-identical to main/types, ./testkit, ./package.json), typedoc second entry point, CHANGELOG + README testkit section.
rstagi
added a commit
that referenced
this pull request
Jul 20, 2026
* refactor(sdk): extract reference adapter to testkit/reference-adapter * feat(sdk): testkit types, case builder skeleton, runner helper * feat(sdk): conformance battery cases, discrimination tests, green path * feat(sdk): package testkit as @ratel-ai/sdk/testkit subpath exports map (. byte-identical to main/types, ./testkit, ./package.json), typedoc second entry point, CHANGELOG + README testkit section.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #111 (
feat/sdk-adapter-spi, base of this PR). GitHub auto-retargets this tomainwhen #111 merges.Ships the adapter conformance testkit (OSI-14) — a runner-agnostic battery every framework adapter must pass, so an AI SDK / Mastra / community
RatelAdaptercan prove it satisfies the ADR-0013 SPI contract before it ships.Surface (
@ratel-ai/sdk/testkit)adapterConformanceCases(options)→ named cases{ name, area, skipped?, run() }; assertions vianode:assert(no test-runner leaks into shipped types), each case builds its own freshratel()core + adapted view, BM25 only (deterministic, model-free).describeAdapterConformance(options, { describe, it })→ registers the battery as first-class tests; structurally typed runner (Vitest/Jest/node:testall fit),it.skipwith a name-suffix fallback.AdapterConformanceOptionshooks: requiredadapter(),makeExecutableTool,callExposedTool,validateRecallPair; optionalmakePassthroughTool(absent → the 4 passthrough cases emit as skipped) andvalidateExposedTool.referenceAdapter/referenceConformanceOptions— the worked example a real adapter's options copy;ratel.test.tsnow imports it instead of an inline fake.Battery (21 cases across 6 areas)
ingest-expose · reserved-ids · recall-topk · passthrough · recall-pair · guards. Expected recall + call ids are computed by the testkit from the pure
core.recall(query)and the deterministic counter (recall_0,recall_1).Discrimination
9 build-excluded broken-adapter variants (lossyIngest, inertIngest, passthroughAlways/Never, rawExpose, fixedCallId, droppedResult, shadowingExtend, namelessAdapter) each fail exactly their targeted cases while unrelated cases stay green — proving the battery actually catches defects, not just passes the reference.
Packaging
@ratel-ai/sdk/testkitsubpath — anexportsmap whose.is byte-identical to today'smain/types(default, not import-only, so the CJSrequirein verify-install keeps working), plus./testkitand./package.json. Zero new deps, no new release unit.conformance.test.ts+test-support/stay build-excluded; typedoc gains the testkit entry point.Verification
src/sdk/ts:build·typecheck·lint(biome + typedocdocs:check) ·test— 211 tests green.pnpm -r build/typecheck/lint/test— green.importand CJSrequire, running a real case through the native binding;.entry intact.