feat(eval): fail A/B test runs fast when Transaction Search is disabled

[jariy17]

Why

Config-based and target-based A/B tests score traffic through their online evaluations, which read agent spans from the aws/spans log group. Without CloudWatch Transaction Search enabled:

• config-based fails server-side with an opaque error, and
• target-based silently produces no results.

Neither surfaces the real cause (Transaction Search not enabled), which sent a bug-bash tester down a long wrong path.

What

• Add EvalClient.transactionSearchEnabled(options) — probes the aws/spans log group via DescribeLogGroups (reuses the existing CloudWatch Logs client; no new SDK dependency).
• Gate the shared private createABTest path on it, so both config-based and target-based runs fail fast before provisioning a role or calling the service.
• New TransactionSearchNotEnabledError (USER source) whose message names the setup step and links the enablement docs.

Scope

• online-eval create is intentionally not gated: it is a persistent, forward-looking config legitimately created before any traffic/spans exist.
• batch-evaluation is not gated here.

Tests

• ab-test fixture suite: recorded DescribeLogGroupsCommand fixture (live RECORD=1) so both run variants exercise the enabled path.
• ab-test unit suite: disabled-path tests assert createConfigBasedABTest throws TransactionSearchNotEnabledError before provisioning/calling the service, and that the message links the docs.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

AgentCore Harness Review

Verdict: Changes requested

The change itself is small, well-scoped, and correctly placed inside the shared private createABTest path so both config- and target-based variants get the fast-fail. Tests mock at the AWS SDK boundary, which is fine.

The one thing worth fixing before merge is that the doc comments and error text describe a scope this PR explicitly does not implement, which will mislead the next reader.

Doc/scope mismatch between comments and actual gating

The PR body says batch-evaluation is intentionally not gated, and the gate is only wired into createABTest. But the surrounding prose talks about batch-evaluation as if that's where the check runs:

• src/errors/errors.tsx:209-215 — docstring on TransactionSearchNotEnabledError says "Raised when a trace-consuming eval operation (batch-evaluation) targets an account/region…". It should describe the A/B test path (or use neutral wording like "an A/B test run"), since that's the only site that throws it today.
• src/errors/errors.tsx:219-222 — error message reads "…the 'aws/spans' log group that evaluation reads." Since this is only surfaced from A/B test creation, "…that A/B test evaluation reads" (or similar) would give the user a more actionable pointer to what they were doing when it fired.
• src/core/eval.tsx:672-679 — docstring on transactionSearchEnabled says "Batch evaluation scores sessions by reading their spans from that group, so its absence means the run has nothing to read." The method is only called from createABTest; the rationale here should reference A/B test online evaluation, matching the comment already in createABTest (lines 516-519).

Either update the comments/message to match the current scope (A/B test only), or, if the intent is to extend to startBatchEvaluation in this PR, add the guard call there too. Otherwise looks good.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.26%. Comparing base (a23e472) to head (79b4d34).

Additional details and impacted files

@@            Coverage Diff            @@
##           refactor    #2349   +/-   ##
=========================================
  Coverage     97.26%   97.26%           
=========================================
  Files           610      610           
  Lines         40590    40606   +16     
=========================================
+ Hits          39480    39496   +16     
  Misses         1110     1110           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jariy17]

Folded into #2356.