fix(offload): tracing/token hardening — astral token fix, L1.5 null fallback, createRequire

[rainforest888]

fix(offload): Tracing & Token Hardening

拆分自 #391，per YOMXXX review。

修复范围

聚焦 src/offload/ 层：token estimate、local LLM、Opik tracing。不涉及 store、runtime、gateway 层。

具体修复

src/offload/fast-token-estimate.ts — Astral-平面 Token 修正

问题	修复
astral-plane 字符（emoji / CJK extension B，U+10000+）编码为 surrogate pair，原实现把 high surrogate 和 low surrogate 各计为一个 token	high surrogate 检测后跳过低代理（i += 2），修正 ~2× token 高估

src/offload/local-llm/index.ts — L1.5 Fallback 修正

问题	修复
解析失败返回 {taskCompleted: false, ...} — normalizeJudgment 把 false 当有效判断，永远不触发 "LLM 不可用" 重试	改为 {taskCompleted: null, ...}，== null 检查触发重试路径

src/offload/opik-tracer.ts — ESM 兼容 + Trace 截断

问题	修复
ESM 下 require("opik") 为 undefined，tracer 永久静默关闭	改为 createRequire(import.meta.url)
未脱敏 tool 输出（文件内容、环境变量）完整写入 trace，泄露到 Opik backend	统一截断到 2000 字符

验证

• 3 files, +41/-15
• 无 API 变更，无新增依赖
• 与 fix(gateway): harden CORS / Host header / startup host binding #39 / fix(tcvdb): escape session filter literals #232 / fix(offload): keep backend TLS verification enabled #242 / fix(offload): sanitize session storage paths #287 / fix(offload): keep MMD files inside storage dir #288 / fix(offload): sanitize ref and skill paths #289 / fix(offload): invalidate token cache after tool use stripping #347 均不重叠

[mayankpandey0]

Thanks for splitting this out from #391! I have manually reviewed the changes, and everything looks exceptionally well done.

A few key highlights from the review:

• The astral-plane token fix: Catching the surrogate pair double-counting issue is an excellent catch. Fixing that ~2x token overestimation will significantly improve tracking accuracy for emojis and extended CJK characters.

• L1.5 Fallback & Opik Tracer: Great attention to detail on correcting the normalize Judgment falsy check, resolving the ESM compatibility issue, and proactively adding the 2000-character trace truncation to prevent data leaks.

The code is exceptionally clean, focused, and has zero overlap with the other hardening PRs.

Everything is in perfect shape and completely ready to merge.

LGTM! 🚀

[YOMXXX]

Approved.

I verified this PR locally by applying the patch to its base commit and running:

• pnpm test — passed: 4 files, 67 tests
• pnpm build — passed
• git diff --check --cached — passed
• targeted token-estimator comparison for astral-plane samples — passed: single emoji / CJK extension-B samples drop from 5 to 3 estimated tokens, mixed samples also stop double-counting the low surrogate

The offload changes are focused and independent: the token estimator now skips the low surrogate in its existing charCodeAt-based loop, failed L1.5 judgment parsing now reaches the null/unavailable path, and Opik trace payloads are bounded while using createRequire for the CommonJS package load.