diff --git a/.changeset/skill-draft-output-budgets.md b/.changeset/skill-draft-output-budgets.md new file mode 100644 index 000000000..50a9742e1 --- /dev/null +++ b/.changeset/skill-draft-output-budgets.md @@ -0,0 +1,7 @@ +--- +"dsh-mnemon": patch +--- + +Give skill drafts configurable 32k initial and 64k retry output budgets. Regenerate once under a fresh task after truncation only when the explicit retry budget is larger, discard partial output, and report attempted budgets and source file size with provider-limit guidance. Keep other skill operations and adoption unchanged. + +技能草案新增可配置的首次 32k 与重试 64k 输出预算。仅在截断且显式重试预算更大时使用新任务重新生成一次,丢弃不完整输出,并报告尝试预算、原文件大小和模型提供方上限建议。其他技能操作与采纳流程保持不变。 diff --git a/docs/en/guides/skills.md b/docs/en/guides/skills.md index b2aab3f27..42087903e 100644 --- a/docs/en/guides/skills.md +++ b/docs/en/guides/skills.md @@ -146,10 +146,13 @@ Skills has its own page under **Plugins → dsh-mnemon**, like every component, | **Suggest skill improvements** | On | Allow reviews to improve existing skills; reviews pause when both suggestion types are off | | **New skill destination** | User | Choose User, Project or Follow storage; saved with **Apply**, reset with **Discard**. Existing skill improvements keep their original file | | **Backtest drafts** | On | Compare each new draft with the current version on its test cases | +| **Draft generation** | 32,768 / 65,536 tokens | First attempt / one larger retry after truncation; saved with **Apply** and available with self-evolution off | | **Review parameters** | — | The thresholds above, folded, applied together with **Apply** | | **In the conversation** | On | Show the **Turn skills** line under each turn | | **Sidebar entry** | Off | Show Skills as its own sidebar entry instead of in the Memory System | +Under **Draft generation**, set **Draft output token limit** and **Retry output token limit** to budgets your model provider supports, including reasoning and the complete `SKILL.md`. They are separate so retries never silently double your chosen limit. A retry limit equal to or below the first disables the truncation retry. There are at most two attempts, shared with the existing file-validation repair. Truncated output is discarded; only a complete, validated draft is saved, and the installed skill changes only after adoption. If generation fails, the error gives the attempted budgets and source file size. Lower a budget rejected by your provider; for a file that still cannot fit, shorten the skill or choose a model with a larger output limit. See [Skills settings](../reference/configuration.md#skills-settings) for the exact keys and ranges. + `skills.targetScope` defaults to `user` to preserve existing installations: upgrading or changing storage scope does not move new skill writes to another root. Choose `storage` explicitly to follow storage scope. New suggestions record their resolved destination; drafts and redrafts keep that destination, and adoption uses the draft unless you explicitly choose another root. Suggestions from older versions without a destination use the current setting when drafted; existing drafts retain their recorded root. Improvements always write back to their original `baseKey`, regardless of this setting or a destination override. Project roots come from the Skills Source catalog for the selected workspace, following DSH discovery (nearest `.git` ancestor, or the workspace itself); the model task's working directory is not used. A missing workspace/project or an unwritable destination produces a visible review/draft error. Mnemon does not silently fall back to user skills. Adoption checks the final selected project again: a project removed after drafting is not recreated, and you can explicitly select a user root instead. Later filesystem failures are also reported. No skills are migrated, discovery precedence is unchanged, and adoption remains an explicit user action. diff --git a/docs/en/reference/configuration.md b/docs/en/reference/configuration.md index ec4a8c76a..0bbb8cdf5 100644 --- a/docs/en/reference/configuration.md +++ b/docs/en/reference/configuration.md @@ -359,7 +359,7 @@ When the active Strategy does not drive idle review or runtime capacity maintena ## Skills settings -Skills keeps its settings under `skills` in the `mnemon` entry, edited on its own page under **Plugins → dsh-mnemon** through the `mnemon-skills` settings scope. Settings apply live once saved, without rebuilding the memory runtime; the destination selector and review parameters wait for **Apply** or **Discard**. The [Skills guide](../guides/skills.md#self-evolution) explains what each does. +Skills keeps its settings under `skills` in the `mnemon` entry, edited on its own page under **Plugins → dsh-mnemon** through the `mnemon-skills` settings scope. Settings apply live once saved, without rebuilding the memory runtime; the destination selector, draft budgets and review parameters wait for **Apply** or **Discard**. Draft budgets are read at the start of each draft. The [Skills guide](../guides/skills.md#self-evolution) explains what each does. | Key | Default | Range | Meaning | |---|---:|---|---| @@ -368,6 +368,8 @@ Skills keeps its settings under `skills` in the `mnemon` entry, edited on its ow | `skills.allowUpdate` | `true` | boolean | **Suggest skill improvements**: allow reviews to improve existing skills | | `skills.targetScope` | `user` | `storage` / `user` / `project` | **New skill destination**: `user` → `user-dsh`; `project` → `project-dsh`; `storage` follows `storageScope` (`global` / `custom` → user, `workspace` / `workspaces` → project). Apply or Discard on the settings page | | `skills.autoBacktest` | `true` | boolean | **Backtest drafts**: compare each new draft with the current version on its test cases | +| `skills.draftMaxTokens` | `32768` | integer, 128–131072 | **Draft output token limit**: budget for the first complete draft, including reasoning | +| `skills.draftRetryMaxTokens` | `65536` | integer, 128–131072 | **Retry output token limit**: one fresh attempt after `max-tokens`, only when this is greater than `draftMaxTokens`; an equal or lower value disables the truncation retry | | `skills.hints` | `true` | boolean | **In the conversation**: the **Turn skills** line under each turn | | `skills.sidebar` | `false` | boolean | **Sidebar entry**: Skills as its own sidebar entry, out of the Memory System | | `skills.reviewThreshold` | `5` | 1–50 | **Review threshold**: the activity score a conversation's window must reach before a review | @@ -378,6 +380,10 @@ Skills keeps its settings under `skills` in the `mnemon` entry, edited on its ow | `skills.maxPendingPerSession` | `5` | 1–20 | **Pending per conversation**: past this, the least confident pending suggestions of a conversation expire | | `skills.casesPerRun` | `8` | 1–20 | **Cases per backtest**: the most cases one automatic backtest runs | +**Draft generation** is available even with self-evolution off. The two budgets explicitly bound each attempt instead of silently doubling a user's limit. A draft gets at most two attempts total, shared with the existing retry for a rejected skill file. Only a `max-tokens` stop can use the larger budget; cancellation and other model failures do not retry. A truncated result is discarded before parsing or saving, and a retry starts a fresh task and child. Successful drafts still require the existing schema and content checks and explicit adoption; generating a draft does not change the installed skill. Reviews, case generation, executors and judges retain their existing budgets. + +Keep both values within the selected provider/model's output limit. If generation still truncates, the error reports the attempted budgets and source `SKILL.md` size in UTF-8 bytes (zero for a new skill), and points to these settings. Raise the budgets only within the model's supported limit; if the provider rejects a requested budget, lower it. If the complete file cannot fit that limit, shorten the skill or select a model with a larger output limit. + `skills.targetScope` defaults to `user` to preserve existing installations: upgrading or changing storage scope does not move new skill writes to another root. Choose `storage` explicitly to follow storage scope. New suggestions record their resolved destination; drafts and redrafts keep that destination, and adoption uses the draft unless you explicitly choose another root. Suggestions from older versions without a destination use the current setting when drafted; existing drafts retain their recorded root. Improvements always write back to their original `baseKey`, regardless of this setting or a destination override. Project roots come from the Skills Source catalog for the selected workspace, following DSH discovery (nearest `.git` ancestor, or the workspace itself); the model task's working directory is not used. A missing workspace/project or an unwritable destination produces a visible review/draft error. Mnemon does not silently fall back to user skills. Adoption checks the final selected project again: a project removed after drafting is not recreated, and you can explicitly select a user root instead. Later filesystem failures are also reported. No skills are migrated, discovery precedence is unchanged, and adoption remains an explicit user action. diff --git a/docs/pr-assets/issue-570-skill-draft-budgets/README.md b/docs/pr-assets/issue-570-skill-draft-budgets/README.md new file mode 100644 index 000000000..bf65451af --- /dev/null +++ b/docs/pr-assets/issue-570-skill-draft-budgets/README.md @@ -0,0 +1,17 @@ +# Skill draft output budgets + +[简体中文](./README.zh-CN.md) + +The real DSH 0.2.2-rc.1 WebUI runs the code from `3475288406b68e005f6e69afc67922542cf8bc12` on Node 24.22.2. The disposable `scripts/serve-e2e.mjs --without-mnemon-cli` fixture isolates its profile, workspace, memory, and skill roots and uses only a loopback model stub. No real gateway or paid model call was used. + +1. Open **Plugins → dsh-mnemon → Skills → Draft generation** with self-evolution off: the budgets are 32,768 and 65,536. +2. Set them to 49,152 and 98,304 and select **Apply**: the real settings page reports that they are saved and live. +3. Reload the page and reopen the settings: both saved values remain. [Validation](./validation.json) records the readback and screenshot hashes. + +![Default budgets with self-evolution off](./settings-defaults.png) + +![Explicit budgets saved through the real settings page](./settings-applied.png) + +The screenshots capture only the settings dialog and contain no credentials, private data, or local paths. The fixture and its test browsers were stopped after capture. These images verify settings behavior, not live-model generation quality; truncation recovery, cancellation, content guards and unchanged ledger behavior are covered by `tests/skill-evolution.spec.ts` and `tests/subagent.spec.ts`. + +Full verification on the production revision passed: 2,512 root tests with 7 skips, 753 workspace tests with 3 skips, and documentation, types, deterministic builds, Headless and package checks. The official DSH 0.2.2-rc.1 root run also passed 2,512 tests with 7 skips. Two review followup regressions change only tests; the resulting skill-evolution suite passes all 53 tests. diff --git a/docs/pr-assets/issue-570-skill-draft-budgets/README.zh-CN.md b/docs/pr-assets/issue-570-skill-draft-budgets/README.zh-CN.md new file mode 100644 index 000000000..9eac8a138 --- /dev/null +++ b/docs/pr-assets/issue-570-skill-draft-budgets/README.zh-CN.md @@ -0,0 +1,17 @@ +# 技能草案输出预算 + +[English](./README.md) + +真实 DSH 0.2.2-rc.1 WebUI 在 Node 24.22.2 上运行 `3475288406b68e005f6e69afc67922542cf8bc12` 的代码。一次性 `scripts/serve-e2e.mjs --without-mnemon-cli` 夹具隔离 profile、工作区、记忆和技能根目录,只配置回环模型桩,未调用真实网关或付费模型。 + +1. 关闭自进化时打开**插件 → 可组合记忆 → 技能管理 → 草案生成**,默认预算为 32,768 和 65,536。 +2. 改为 49,152 和 98,304 后点击**应用**,真实设置页面显示“已保存并实时生效”。 +3. 刷新页面并重新打开设置,两项保存值保持一致。[验证记录](./validation.json)保存了回读结果和截图摘要。 + +![关闭自进化时显示的默认预算](./settings-defaults.png) + +![通过真实设置页面保存显式预算](./settings-applied.png) + +截图只包含设置对话框,没有凭据、私有数据或本地路径。截图完成后已停止夹具及测试浏览器。图片验证设置行为,不代表真实模型生成质量;截断恢复、取消、内容校验和 ledger 保持不变由 `tests/skill-evolution.spec.ts` 与 `tests/subagent.spec.ts` 覆盖。 + +生产代码提交的完整验证通过:根测试 2,512 项通过、7 项跳过,工作区测试 753 项通过、3 项跳过,文档、类型、确定性构建、Headless 和包检查均通过。官方 DSH 0.2.2-rc.1 根测试同样 2,512 项通过、7 项跳过。审查补充的两项回归只修改测试,补充后技能进化测试共 53 项全部通过。 diff --git a/docs/pr-assets/issue-570-skill-draft-budgets/settings-applied.png b/docs/pr-assets/issue-570-skill-draft-budgets/settings-applied.png new file mode 100644 index 000000000..f568cd62a Binary files /dev/null and b/docs/pr-assets/issue-570-skill-draft-budgets/settings-applied.png differ diff --git a/docs/pr-assets/issue-570-skill-draft-budgets/settings-defaults.png b/docs/pr-assets/issue-570-skill-draft-budgets/settings-defaults.png new file mode 100644 index 000000000..61cdbee8c Binary files /dev/null and b/docs/pr-assets/issue-570-skill-draft-budgets/settings-defaults.png differ diff --git a/docs/pr-assets/issue-570-skill-draft-budgets/validation.json b/docs/pr-assets/issue-570-skill-draft-budgets/validation.json new file mode 100644 index 000000000..e7ac7626a --- /dev/null +++ b/docs/pr-assets/issue-570-skill-draft-budgets/validation.json @@ -0,0 +1,34 @@ +{ + "codeRevision": "3475288406b68e005f6e69afc67922542cf8bc12", + "host": "@deepseek-ai/dsh@0.2.2-rc.1", + "node": "24.22.2", + "locale": "zh-CN", + "fixture": "scripts/serve-e2e.mjs --without-mnemon-cli", + "defaultBudgets": { + "draftMaxTokens": 32768, + "draftRetryMaxTokens": 65536 + }, + "savedBudgets": { + "draftMaxTokens": 49152, + "draftRetryMaxTokens": 98304 + }, + "afterPageReload": { + "draftMaxTokens": 49152, + "draftRetryMaxTokens": 98304 + }, + "selfEvolutionEnabled": false, + "paidProviderCalls": 0, + "screenshots": { + "settings-applied.png": { + "sha256": "dcfc861b3925d72f523bba4db0c175c1d62fd27361730bf4b392c6f661dc7356", + "bytes": 82099 + }, + "settings-defaults.png": { + "sha256": "abfb78e2eb8570c188f5b7372e6592003b0d48b60b8f4e6d1054b20db1955f66", + "bytes": 79561 + } + }, + "limits": [ + "Settings UI and profile persistence only; no real gateway generation was run." + ] +} diff --git a/docs/zh-CN/guides/skills.md b/docs/zh-CN/guides/skills.md index 09971c738..91a4466f4 100644 --- a/docs/zh-CN/guides/skills.md +++ b/docs/zh-CN/guides/skills.md @@ -146,10 +146,13 @@ Agent 会按技能内容执行,技能也可能带有脚本。只安装你信 | **改进已有技能** | 开启 | 允许复盘提出已有技能的改进;两类建议均关闭时暂停复盘 | | **新技能写入位置** | 用户 | 选择用户、项目或跟随存储范围;点击**应用**保存或**放弃**还原。改进已有技能仍写回原文件 | | **草案自动回测** | 开启 | 草案写好后,用测试用例与当前版本对比 | +| **草案生成** | 32,768 / 65,536 token | 首次尝试 / 截断后一次扩容重试;点击**应用**保存,关闭自进化时也可设置 | | **复盘参数** | — | 上文提到的各项阈值,默认折叠,修改后一起**应用** | | **对话提示** | 开启 | 在每个回合下方显示**本回合技能** | | **侧边栏入口** | 关闭 | 技能管理单独显示在侧边栏,不再放在记忆系统里 | +在**草案生成**里,将**草案输出 token 上限**和**重试输出 token 上限**设为模型提供方支持的预算,预算包含推理和完整 `SKILL.md`。两项独立配置,重试不会隐性翻倍你选择的上限。重试上限等于或小于首次值时关闭截断重试。每份草案最多尝试两次,与原有文件校验修正共用重试机会。截断输出会丢弃,只保存完整且校验通过的草案,采纳后才改动已安装技能。失败信息包含实际尝试预算与原文件大小:提供方拒绝预算时应调低;完整文件仍无法容纳时,可缩短技能或选择输出上限更高的模型。准确配置键与范围见[技能管理设置](../reference/configuration.md#技能管理设置)。 + `skills.targetScope` 默认保留 `user`,兼容现有安装:升级或修改存储范围不会把新技能写到另一个根目录。需要跟随存储范围时显式选择 `storage`。新建议记录解析后的目标;草案及重新起草保留该目标,采纳沿用草案的目标,除非用户手动选择其他位置。旧版本中没有目标字段的建议在起草时使用当前设置;已有草案保留记录的目标。改进已有技能始终写回原 `baseKey`,不受设置或目标覆盖影响。 项目根来自所选工作区的 Skills Source 目录,遵循 DSH 的发现规则(最近的 `.git` 祖先目录,否则为工作区本身),不使用模型任务的工作目录。缺少工作区/项目或目标不可写时,复盘/起草会显示明确错误,不静默降级到用户技能。采纳时重新检查最终选择的项目:不会重建起草后被删除的项目,用户可以明确改选用户根目录;后续文件系统错误也会显示。不会迁移技能或改变发现优先顺序,采纳仍由用户明确操作。 diff --git a/docs/zh-CN/reference/configuration.md b/docs/zh-CN/reference/configuration.md index fe1e65858..8ecfa2d13 100644 --- a/docs/zh-CN/reference/configuration.md +++ b/docs/zh-CN/reference/configuration.md @@ -359,7 +359,7 @@ DSH 的实时模型目录会报告各模型的输入模态,选择器为支持 ## 技能管理设置 -技能管理的设置保存在 `mnemon` 条目的 `skills` 下,通过 `mnemon-skills` 设置范围在**插件 → 可组合记忆**里它自己的页面上编辑。设置保存后立即作用于运行中的自进化和客户端,不重建记忆运行图;目标选择与复盘参数需要点击**应用**或**放弃**。各项含义见[技能管理指南](../guides/skills.md#自进化)。 +技能管理的设置保存在 `mnemon` 条目的 `skills` 下,通过 `mnemon-skills` 设置范围在**插件 → 可组合记忆**里它自己的页面上编辑。设置保存后实时生效,不重建记忆运行图;目标选择、草案预算与复盘参数需要点击**应用**或**放弃**。草案预算在每次起草开始时读取。各项含义见[技能管理指南](../guides/skills.md#自进化)。 | 键 | 默认值 | 范围 | 含义 | |---|---:|---|---| @@ -368,6 +368,8 @@ DSH 的实时模型目录会报告各模型的输入模态,选择器为支持 | `skills.allowUpdate` | `true` | boolean | **改进已有技能**:允许复盘提出已有技能的改进 | | `skills.targetScope` | `user` | `storage` / `user` / `project` | **新技能写入位置**:`user` → `user-dsh`;`project` → `project-dsh`;`storage` 跟随 `storageScope`(`global` / `custom` → 用户,`workspace` / `workspaces` → 项目)。设置页面支持应用或放弃 | | `skills.autoBacktest` | `true` | boolean | **草案自动回测**:草案写好后用测试用例与当前版本对比 | +| `skills.draftMaxTokens` | `32768` | 整数,128–131072 | **草案输出 token 上限**:首次生成完整草案的预算,包含推理 token | +| `skills.draftRetryMaxTokens` | `65536` | 整数,128–131072 | **重试输出 token 上限**:仅在 `max-tokens` 且此值大于 `draftMaxTokens` 时重新生成一次;相同或更小值关闭截断重试 | | `skills.hints` | `true` | boolean | **对话提示**:在每个回合下方显示**本回合技能** | | `skills.sidebar` | `false` | boolean | **侧边栏入口**:技能管理单独显示在侧边栏,不再放在记忆系统里 | | `skills.reviewThreshold` | `5` | 1–50 | **复盘阈值**:对话窗口的活动积分达到它才会复盘 | @@ -378,6 +380,10 @@ DSH 的实时模型目录会报告各模型的输入模态,选择器为支持 | `skills.maxPendingPerSession` | `5` | 1–20 | **单个对话待处理上限**:超出后,这段对话里置信度最低的待处理建议过期 | | `skills.casesPerRun` | `8` | 1–20 | **每次回测用例数**:一次自动回测最多运行的用例数 | +关闭自进化时也可以配置**草案生成**。两项预算分别明确限制每次尝试,避免隐性翻倍用户设置的上限。每份草案最多尝试两次,与原有的文件校验失败修正共用一次重试机会。只有 `max-tokens` 才可使用较大的预算;取消和其他模型错误不重试。截断结果在解析和保存前丢弃,重试使用新的任务与子代理。成功草案仍通过原有 schema、内容校验和用户明确采纳;生成草案不会改动已安装技能。复盘、用例生成、执行和评分仍使用原有预算。 + +两项均应符合所选提供方/模型的输出上限。若仍被截断,错误会列出实际尝试的预算、原 `SKILL.md` 的 UTF-8 字节数(新技能为零)以及设置位置。只在模型支持的范围内提高预算;提供方拒绝请求预算时应调低。如果完整文件无法容纳于该上限,需要缩短技能或选择输出上限更高的模型。 + `skills.targetScope` 默认保留 `user`,兼容现有安装:升级或修改存储范围不会把新技能写到另一个根目录。需要跟随存储范围时显式选择 `storage`。新建议记录解析后的目标;草案及重新起草保留该目标,采纳沿用草案的目标,除非用户手动选择其他位置。旧版本中没有目标字段的建议在起草时使用当前设置;已有草案保留记录的目标。改进已有技能始终写回原 `baseKey`,不受设置或目标覆盖影响。 项目根来自所选工作区的 Skills Source 目录,遵循 DSH 的发现规则(最近的 `.git` 祖先目录,否则为工作区本身),不使用模型任务的工作目录。缺少工作区/项目或目标不可写时,复盘/起草会显示明确错误,不静默降级到用户技能。采纳时重新检查最终选择的项目:不会重建起草后被删除的项目,用户可以明确改选用户根目录;后续文件系统错误也会显示。不会迁移技能或改变发现优先顺序,采纳仍由用户明确操作。 diff --git a/src/client/locales.ts b/src/client/locales.ts index 158fdfcff..d9a643f96 100644 --- a/src/client/locales.ts +++ b/src/client/locales.ts @@ -800,6 +800,10 @@ export const zh = { 'skills.autoBacktest': '草案自动回测', 'skills.autoBacktestHint': '草案写好后,用测试用例与当前版本对比', 'skills.parameters': '复盘参数', + 'skills.draftGeneration': '草案生成', + 'skills.draftMaxTokens': '草案输出 token 上限', + 'skills.draftRetryMaxTokens': '重试输出 token 上限', + 'skills.draftBudgetHint': '预算包含推理与完整 SKILL.md。截断时仅在重试上限更大时重新生成一次;设为相同或更小值可关闭扩容重试。两项均需符合模型提供方的输出限制,保存后用于下一次草案。', 'skills.reviewThreshold': '复盘阈值', 'skills.idleSeconds': '空闲等待(秒)', 'skills.cooldownSeconds': '复盘间隔(秒)', @@ -1622,6 +1626,10 @@ export const en: Record = { 'skills.autoBacktest': 'Backtest drafts', 'skills.autoBacktestHint': 'Compare a new draft with the current version on its test cases', 'skills.parameters': 'Review parameters', + 'skills.draftGeneration': 'Draft generation', + 'skills.draftMaxTokens': 'Draft output token limit', + 'skills.draftRetryMaxTokens': 'Retry output token limit', + 'skills.draftBudgetHint': 'Budgets include reasoning and the complete SKILL.md. Truncation triggers one fresh attempt only when the retry limit is higher; an equal or lower value disables it. Keep both within the model provider’s output limit. Saved changes apply to the next draft.', 'skills.reviewThreshold': 'Review threshold', 'skills.idleSeconds': 'Idle wait (s)', 'skills.cooldownSeconds': 'Review interval (s)', diff --git a/src/client/skills-settings.tsx b/src/client/skills-settings.tsx index ecf0db6bc..83dbf792e 100644 --- a/src/client/skills-settings.tsx +++ b/src/client/skills-settings.tsx @@ -28,8 +28,13 @@ const PARAMETERS: ReadonlyArray<{ key: Parameter; min: number; max: number; inte { key: 'casesPerRun', min: 1, max: 20, integer: true }, ] +const DRAFT_PARAMETERS = [ + { key: 'draftMaxTokens', min: 128, max: 131_072, integer: true }, + { key: 'draftRetryMaxTokens', min: 128, max: 131_072, integer: true }, +] as const + /** The value a parameter stores, or undefined when the Host would refuse it. */ -function parameterValue(field: typeof PARAMETERS[number], text: string): number | undefined { +function parameterValue(field: { min: number; max: number; integer: boolean }, text: string): number | undefined { if (text.trim() === '') return undefined const value = Number(text) if (!Number.isFinite(value) || value < field.min || value > field.max) return undefined @@ -66,8 +71,13 @@ export function SkillsSettings(props: { scope: ClientSettingsScope const changed = PARAMETERS.filter(field => parameterValue(field, draft[field.key]) !== saved[field.key]) if (changed.length > 0) await scope.mutate(changed.map(field => set(field.key, parameterValue(field, draft[field.key])!))) }) + const draftBudgets = useStaged({ draftMaxTokens: String(saved.draftMaxTokens), draftRetryMaxTokens: String(saved.draftRetryMaxTokens) }, async draft => { + const changed = DRAFT_PARAMETERS.filter(field => parameterValue(field, draft[field.key]) !== saved[field.key]) + if (changed.length > 0) await scope.mutate(changed.map(field => set(field.key, parameterValue(field, draft[field.key])!))) + }) const disabled = !page.writable || snapshot.status === 'loading' || !snapshot.writable const invalid = PARAMETERS.find(field => parameterValue(field, parameters.draft[field.key]) === undefined) + const invalidDraft = DRAFT_PARAMETERS.find(field => parameterValue(field, draftBudgets.draft[field.key]) === undefined) return
@@ -78,6 +88,20 @@ export function SkillsSettings(props: { scope: ClientSettingsScope disabled={disabled || destination.saving} onChange={targetScope => destination.edit({ targetScope })} /> { void destination.apply() }} /> +
+ {t('skills.draftGeneration')} +

{t('skills.draftBudgetHint')}

+
+ {DRAFT_PARAMETERS.map(field => )} +
+ { void draftBudgets.apply() }} /> +
{evolution.value && } {evolution.value &&
{t('skills.parameters')} diff --git a/src/host/config.ts b/src/host/config.ts index 1535375fa..f4b9b7f8a 100644 --- a/src/host/config.ts +++ b/src/host/config.ts @@ -82,6 +82,8 @@ export const SkillsConfig: z, ResolvedSkillsConfig allowUpdate: z.boolean().default(DEFAULT_SKILLS_CONFIG.allowUpdate), targetScope: z.union(['storage', 'user', 'project'] as const).default(DEFAULT_SKILLS_CONFIG.targetScope), autoBacktest: z.boolean().default(DEFAULT_SKILLS_CONFIG.autoBacktest), + draftMaxTokens: z.number().step(1).min(128).max(131_072).default(DEFAULT_SKILLS_CONFIG.draftMaxTokens), + draftRetryMaxTokens: z.number().step(1).min(128).max(131_072).default(DEFAULT_SKILLS_CONFIG.draftRetryMaxTokens), hints: z.boolean().default(DEFAULT_SKILLS_CONFIG.hints), sidebar: z.boolean().default(DEFAULT_SKILLS_CONFIG.sidebar), reviewThreshold: z.number().step(1).min(1).max(50).default(DEFAULT_SKILLS_CONFIG.reviewThreshold), diff --git a/src/host/plugin.ts b/src/host/plugin.ts index 2ce8b9010..2491385e9 100644 --- a/src/host/plugin.ts +++ b/src/host/plugin.ts @@ -164,7 +164,7 @@ export function apply(rawContext: unknown, rawConfig: MnemonConfig | LiveHostCon settings: () => skillSettings.get(), source: agent => skillsSourceFor(runtime, agent), runTask: (workspaceRoot, signal, operation, route) => lifecycle.runSkillTask(workspaceRoot, signal, operation, route), - skillTask: (parent, label, prompt, schema, persona, signal) => coordinator.skillTask(parent, label, prompt, schema, persona, signal, taskRoute(parent)), + skillTask: (parent, label, prompt, schema, persona, signal, _route, maxTokens) => coordinator.skillTask(parent, label, prompt, schema, persona, signal, taskRoute(parent), maxTokens), }) lifecycle.useSkillEvolution(skills) // Registered before the lifecycle root so teardown runs the other way round: diff --git a/src/host/protocol.ts b/src/host/protocol.ts index d902cc4ed..b290c5e46 100644 --- a/src/host/protocol.ts +++ b/src/host/protocol.ts @@ -415,6 +415,10 @@ export interface ResolvedSkillsConfig { targetScope: 'storage' | 'user' | 'project' /** Backtest a draft against the current version as soon as it is written. */ autoBacktest: boolean + /** Output budget for a complete skill draft, including model reasoning. */ + draftMaxTokens: number + /** One fresh attempt after truncation, only when larger than draftMaxTokens. */ + draftRetryMaxTokens: number /** The one-line skills bar under a finished turn. */ hints: boolean /** 技能管理 as a sidebar entry of its own instead of a Memory System page. */ @@ -441,6 +445,8 @@ export const DEFAULT_SKILLS_CONFIG: Readonly = Object.free allowUpdate: true, targetScope: 'user', autoBacktest: true, + draftMaxTokens: 32_768, + draftRetryMaxTokens: 65_536, hints: true, sidebar: false, reviewThreshold: 5, diff --git a/src/host/skill-evolution.ts b/src/host/skill-evolution.ts index 89ed2a1c6..c39f7b0af 100644 --- a/src/host/skill-evolution.ts +++ b/src/host/skill-evolution.ts @@ -12,6 +12,7 @@ import { skillCasesPrompt, skillDraftPrompt, skillExecutorPrompt, skillJudgePrompt, skillReviewPrompt, } from './skill-review.ts' import { captureTurn, redact, sessionTitle, TrajectoryWindow, turnSkillUses } from './skill-trajectory.ts' +import { SubagentStoppedError } from './subagent-errors.ts' /** The Skills Source's management protocol, as the Host calls it. */ export interface SkillsSourcePort { @@ -28,7 +29,7 @@ export interface SkillEvolutionHost { source(agent: HostAgent): SkillsSourcePort | undefined /** Run model work under a fresh task Agent in a workspace, outside any conversation. */ runTask(workspaceRoot: string | undefined, signal: AbortSignal, operation: (agent: HostAgent) => Promise, route?: ModelRoute): Promise - skillTask(parent: HostAgent, label: string, prompt: string, schema: Record, persona: string, signal: AbortSignal, route?: ModelRoute): Promise<{ structured: unknown }> + skillTask(parent: HostAgent, label: string, prompt: string, schema: Record, persona: string, signal: AbortSignal, route?: ModelRoute, maxTokens?: number): Promise<{ structured: unknown }> now?(): Date warn?(message: string, error?: unknown): void } @@ -385,17 +386,41 @@ export class SkillEvolution { const settings = this.host.settings() const { pack, targetRoot, guidance } = prepared let prompt = skillDraftPrompt(pack, guidance, targetRoot) + let maxTokens = settings.draftMaxTokens + const attemptedBudgets: number[] = [] let draft: SkillDraft | undefined for (let attempt = 0; draft === undefined; attempt++) { - const answer = await this.host.runTask(workspace, signal, task => this.host.skillTask(task, 'Mnemon skill draft', prompt, SKILL_DRAFT_SCHEMA as unknown as Record, SKILL_DRAFT_PERSONA, signal, input.route), input.route) + signal.throwIfAborted() + attemptedBudgets.push(maxTokens) + let answer: { structured: unknown } + try { + answer = await this.host.runTask(workspace, signal, task => this.host.skillTask(task, 'Mnemon skill draft', prompt, SKILL_DRAFT_SCHEMA as unknown as Record, SKILL_DRAFT_PERSONA, signal, input.route, maxTokens), input.route) + signal.throwIfAborted() + } catch (error) { + signal.throwIfAborted() + if (error instanceof SubagentStoppedError && error.maxTokens !== undefined) attemptedBudgets[attemptedBudgets.length - 1] = error.maxTokens + // Share the existing one-retry limit with file-validation repair. A + // fresh task discards truncated JSON and gets a new result capability. + if (error instanceof SubagentStoppedError && error.stopReason === 'max-tokens' && attempt === 0 && settings.draftRetryMaxTokens > maxTokens) { + maxTokens = settings.draftRetryMaxTokens + prompt += '\n\nThe previous attempt exhausted its output budget and was discarded. Return a complete SKILL.md from the beginning; keep notes and changes concise.' + continue + } + const bytes = Buffer.byteLength(pack.skill?.content ?? '', 'utf8') + // The Source retains 500 error characters. Put actionable context first + // and bound provider detail so the page keeps the complete guidance. + throw new Error(`Skill draft failed. Attempted budgets: ${attemptedBudgets.join(', ')} tokens; source SKILL.md: ${bytes} bytes (UTF-8). Adjust skills.draftMaxTokens / skills.draftRetryMaxTokens in Plugins → dsh-mnemon → Skills → Draft generation. Stay within provider/model caps: lower rejected budgets, shorten the skill or use a model with a higher output cap. Reason: ${message(error).slice(0, 100)}`, { cause: error }) + } const parsed = parseSkillDraft(answer.structured) try { + signal.throwIfAborted() draft = await source.mutate('draft-record', { ...(input.suggestionId === undefined ? {} : { suggestionId: input.suggestionId }), skillName: pack.skillName, content: parsed.content, notes: parsed.notes, changes: parsed.changes, guidance, ...(pack.action === 'create' ? { targetRoot } : {}), }, signal) } catch (error) { + signal.throwIfAborted() // One retry, told exactly why DSH would refuse the file. if (attempt > 0) throw error prompt += `\n\nYour previous SKILL.md was rejected: ${message(error)}. Return a corrected complete file.` diff --git a/src/host/subagent-errors.ts b/src/host/subagent-errors.ts new file mode 100644 index 000000000..0d546b9b5 --- /dev/null +++ b/src/host/subagent-errors.ts @@ -0,0 +1,7 @@ +/** A provider stop, kept separate from schema, startup and cancellation errors. */ +export class SubagentStoppedError extends Error { + constructor(readonly stopReason: string, readonly maxTokens: number | undefined, detail?: string) { + super(`memory subagent stopped with ${stopReason}${detail === undefined ? '' : `: ${detail}`}`) + this.name = 'SubagentStoppedError' + } +} diff --git a/src/host/subagent.ts b/src/host/subagent.ts index e19b7c374..13b94f084 100644 --- a/src/host/subagent.ts +++ b/src/host/subagent.ts @@ -28,6 +28,7 @@ import { archivableElsewhere } from './archive-destinations.ts' import { runtimeArchiveSource, runtimeLocked, type RuntimeModelResult, type RuntimeWriteContext } from './runtime-capacity.ts' import { archiveDocumentLocked, documentLocked, type DocumentWorkflowHost } from './document-archive.ts' import { AutomaticReviewAdmission } from './review-admission.ts' +import { SubagentStoppedError } from './subagent-errors.ts' import { FINISHED_TASK_REFUSAL, isLegacyMemoryTask, TaskContinuation } from './task-continuation.ts' export type { CoordinatedDocumentResult, CoordinatedRuntimeMemoryResult, DelegatedAnswerResult, DelegatedWriteResult, SubagentCounters } from "./protocol.ts" @@ -787,8 +788,8 @@ export class MnemonSubagentCoordinator { * One tool-less structured run for skill evolution: a review, a draft, a * backtest step or a case derivation. The child sees only the prompt. */ - async skillTask(parent: HostAgent, label: string, prompt: string, schema: Record, persona: string, signal: AbortSignal, route?: ModelRoute): Promise<{ provider: string; runId: string; structured: unknown }> { - const { provider, runId, result } = await this.delegate(parent, 'skill', label, prompt, [], schema, signal, 'spawn', persona, undefined, undefined, route) + async skillTask(parent: HostAgent, label: string, prompt: string, schema: Record, persona: string, signal: AbortSignal, route?: ModelRoute, maxTokens?: number): Promise<{ provider: string; runId: string; structured: unknown }> { + const { provider, runId, result } = await this.delegate(parent, 'skill', label, prompt, [], schema, signal, 'spawn', persona, undefined, undefined, route, undefined, maxTokens) return { provider, runId, structured: result.structured } } @@ -960,6 +961,7 @@ ${naturalRequest(request, target === undefined ? REQUEST_LABELS : TARGETED_WRITE guard?: (execution: ToolExecution) => string | undefined, route?: ModelRoute, admission?: () => void, + skillMaxTokens?: number, ): Promise<{ provider: string; runId: string; result: HostSubagentResult; receipts: CapturedToolReceipt[]; writes: CapturedToolReceipt[] }> { if (this.disposed) throw new Error('dsh-mnemon subagent coordinator is disposed') const provider = this.provider(preferredProvider, this.runtimeSource.forAgent(parent).config.subagentProviderNames) @@ -1067,7 +1069,7 @@ This is the only completion channel for this run. Do not finish with a plain-tex : operation === 'document-archive' ? 8_192 : operation === 'metadata-maintenance' ? 4_096 : operation === 'review' ? this.runtimeSource.config.idleReview.maxTokens - : operation === 'skill' ? 16_384 + : operation === 'skill' ? skillMaxTokens ?? 16_384 : undefined // A route the caller picked is the model the user asked this run to use, // so it wins over the resolver rather than being added to it. @@ -1105,11 +1107,11 @@ This is the only completion channel for this run. Do not finish with a plain-tex // tool receipt is already the host's authoritative commit evidence. structured = recoverStructuredResult(recovery, committedReceipts.filter(receipt => receipt.agentId === activeRun.id)) } - if (structured !== undefined) assertDshOutputValue(outputSchema, structured) if (result.stopReason !== 'completed') { const detail = subagentFailureDetail(activeRun, result) - throw new Error(`memory subagent stopped with ${result.stopReason}${detail === undefined ? '' : `: ${detail}`}`) + throw new SubagentStoppedError(result.stopReason, perOpMaxTokens, detail) } + if (structured !== undefined) assertDshOutputValue(outputSchema, structured) if (structured === undefined) throw new Error('memory subagent completed without recording its result') if (operation === 'review' && ['failed', 'partial', 'unknown'].includes(String(object(structured).action))) throw new Error(`memory review reported ${object(structured).action}`) completed = { diff --git a/tests/client-skills.spec.tsx b/tests/client-skills.spec.tsx index 05e3953bb..02642145e 100644 --- a/tests/client-skills.spec.tsx +++ b/tests/client-skills.spec.tsx @@ -22,6 +22,27 @@ const locale = { active: 'en' as const, locales: [], revision: 0 } const localeRuntime = { getSnapshot: () => locale, subscribe: () => () => {} } describe('技能管理 settings under Plugins', () => { + it('stages both draft budgets while evolution is off and rejects invalid values', async () => { + const scope = liveSettingsScope({ status: 'ready', value: {}, revision: 1, writable: true, mode: 'host' }) + const view = render() + fireEvent.click(screen.getByText('Draft generation')) + const first = screen.getByLabelText('Draft output token limit') + const retry = screen.getByLabelText('Retry output token limit') + expect(first).toHaveProperty('value', '32768') + expect(retry).toHaveProperty('value', '65536') + fireEvent.change(first, { target: { value: '49152' } }) + fireEvent.change(retry, { target: { value: '131073' } }) + expect(screen.getByRole('button', { name: t('options.apply') })).toHaveProperty('disabled', true) + expect(scope.getSnapshot().value?.draftMaxTokens).toBeUndefined() + fireEvent.change(retry, { target: { value: '49152' } }) + fireEvent.click(screen.getByRole('button', { name: t('options.apply') })) + await waitFor(() => expect(scope.getSnapshot().value).toMatchObject({ draftMaxTokens: 49152, draftRetryMaxTokens: 49152 })) + view.unmount() + render() + expect(screen.getByLabelText('Draft output token limit')).toHaveProperty('value', '49152') + expect(screen.getByLabelText('Retry output token limit')).toHaveProperty('value', '49152') + }) + it('stages the new-skill destination until Apply and restores it on Discard', async () => { const writes = vi.fn(async () => {}) const scope = liveSettingsScope({ status: 'ready', value: { evolution: true }, revision: 1, writable: true, mode: 'host' }, writes) @@ -44,6 +65,8 @@ describe('技能管理 settings under Plugins', () => { const scope = liveSettingsScope({ status: 'ready', value: { evolution: true }, revision: 1, writable: blocked !== 'scope', mode: 'host' }, writes) render() expect(screen.getByRole('button', { name: `${t('skills.targetScope')} ${t('skills.targetScope.user')}` })).toHaveProperty('disabled', true) + expect(screen.getByLabelText('Draft output token limit')).toHaveProperty('disabled', true) + expect(screen.getByLabelText('Retry output token limit')).toHaveProperty('disabled', true) expect(writes).not.toHaveBeenCalled() }) diff --git a/tests/config.spec.ts b/tests/config.spec.ts index f79c58324..13bd9c7e3 100644 --- a/tests/config.spec.ts +++ b/tests/config.spec.ts @@ -9,6 +9,17 @@ import { Config, resolveConfig, resolveSkillsConfig } from "../src/host/config.t afterEach(() => vi.unstubAllEnvs()) describe('Mnemon config and resolution', () => { + it('defaults and validates explicit draft budgets without forcing a larger retry budget', () => { + expect(resolveSkillsConfig()).toMatchObject({ draftMaxTokens: 32_768, draftRetryMaxTokens: 65_536 }) + expect(Config({ skills: { draftMaxTokens: 98_304, draftRetryMaxTokens: 32_768 } }).skills) + .toMatchObject({ draftMaxTokens: 98_304, draftRetryMaxTokens: 32_768 }) + for (const key of ['draftMaxTokens', 'draftRetryMaxTokens'] as const) { + for (const value of [128, 131_072]) expect(resolveSkillsConfig({ [key]: value })[key]).toBe(value) + for (const value of [0, 127, 131_073, 1.5, NaN, Infinity, '32768']) { + expect(() => resolveSkillsConfig({ [key]: value } as never)).toThrow() + } + } + }) it('defaults explicit subagent providers off and validates their configured names', () => { expect(resolveConfig({}).subagentProviderNames).toEqual([]) expect(Config({ subagentProviderNames: ['custom'] }).subagentProviderNames).toEqual(['custom']) diff --git a/tests/fixtures/presentation-baseline.json b/tests/fixtures/presentation-baseline.json index 2b0eb6888..9f12e3de8 100644 --- a/tests/fixtures/presentation-baseline.json +++ b/tests/fixtures/presentation-baseline.json @@ -13,12 +13,12 @@ "classesSha256": "1023db19fca62acf746f435b5185a5247017e30b0c2f5cf1c9be98e3415eb793" }, "zh": { - "keys": 1409, - "sha256": "844ea5efa91dea78a8f71a84b86cba03d34b7bef5667094ac60883b09ff3849b" + "keys": 1413, + "sha256": "04ef97676f715767d9dc6ba432f0b302174ff725df97c5229d5608cf38b6e321" }, "en": { - "keys": 1409, - "sha256": "cccb97f68b3768dcbb8c933a759416faa43048b96c3d681330dcc2ae959ca732" + "keys": 1413, + "sha256": "d7679ffcbd9a9f1f2fbeea7dad8e22c2bd7a7f1e2f5f14e47dc0330f97f6c4f4" } } } diff --git a/tests/profile-settings-revisions.spec.ts b/tests/profile-settings-revisions.spec.ts index b21c5fe90..29f6a4c4a 100644 --- a/tests/profile-settings-revisions.spec.ts +++ b/tests/profile-settings-revisions.spec.ts @@ -1,10 +1,11 @@ import { Context, type Fiber } from '@deepseek-ai/cordis' import { afterEach, describe, expect, it, vi } from 'vitest' -import { Config, InteractionConfig } from '../src/host/config.ts' +import { Config, InteractionConfig, SkillsConfig } from '../src/host/config.ts' import type { HostContextShape, HostSettingsService } from '../src/host/dsh.ts' import type { SettingsOperation } from '../src/host/protocol.ts' import { ProfileMnemonSettings } from '../src/host/settings-service.ts' import { schema as ViewSchema } from '../src/host/view-preferences.ts' +import { createSettingsHandler } from '../src/host/settings.ts' type Data = Record type Layer = 'value' | 'base' | 'user' @@ -135,6 +136,27 @@ const uiWrite: SettingsOperation[] = [{ op: 'set', path: ['turnBar'], value: fal const viewWrite: SettingsOperation[] = [{ op: 'set', path: ['strategyTypeId'], value: 'scoped' }] describe('profile settings virtual namespace revision fences', () => { + it('persists both draft budgets through the Skills bridge and rejects invalid writes before saving', async () => { + const f = await fixture() + f.settings.register('mnemon-skills', SkillsConfig, { applies: 'live' }) + const handler = createSettingsHandler(f.settings) + const ops: SettingsOperation[] = [ + { op: 'set', path: ['draftMaxTokens'], value: 49_152 }, + { op: 'set', path: ['draftRetryMaxTokens'], value: 98_304 }, + ] + expect(await handler('mutate', { namespace: 'mnemon-skills', expectedRevision: 0, ops })).toMatchObject({ ok: true }) + expect(f.state.user).toMatchObject({ skills: { draftMaxTokens: 49_152, draftRetryMaxTokens: 98_304 } }) + expect(Config(JSON.parse(JSON.stringify(f.state.user))).skills).toMatchObject({ draftMaxTokens: 49_152, draftRetryMaxTokens: 98_304 }) + expect(f.read('mnemon-skills').value).toMatchObject({ draftMaxTokens: 49_152, draftRetryMaxTokens: 98_304 }) + const before = structuredClone(f.state.user) + expect(await handler('mutate', { + namespace: 'mnemon-skills', expectedRevision: 1, + ops: [{ op: 'set', path: ['draftRetryMaxTokens'], value: 131_073 }], + })).toMatchObject({ ok: false }) + expect(f.state.user).toEqual(before) + expect(f.state.revision).toBe(1) + }) + it.each(['core-first', 'ui-first'] as const)('saves concurrent Core and UI changes with one shared observed revision (%s)', async order => { const f = await fixture() const revision = f.read().revision diff --git a/tests/skill-evolution.spec.ts b/tests/skill-evolution.spec.ts index eee6f09d3..bad93c8bf 100644 --- a/tests/skill-evolution.spec.ts +++ b/tests/skill-evolution.spec.ts @@ -8,7 +8,8 @@ import * as generalStrategy from 'dsh-mnemon-strategy-general' import type { BacktestRun, EvalCase, EvidencePack, SkillCatalog, SkillDraft, SkillSuggestion, Trajectory } from 'dsh-mnemon-source-skills/contracts' import type { HostAgent, HostAgentContext, HostSessionEvent } from '../src/host/dsh.ts' import { DEFAULT_SKILLS_CONFIG, type ResolvedSkillsConfig } from '../src/host/protocol.ts' -import { SkillEvolution, type SkillsSourcePort } from '../src/host/skill-evolution.ts' +import { SkillEvolution, type SkillEvolutionHost, type SkillsSourcePort } from '../src/host/skill-evolution.ts' +import { SubagentStoppedError } from '../src/host/subagent-errors.ts' import { gradeCase, parseSkillDraft, parseSkillReview, renderSteps, skillDraftPrompt, skillReviewPrompt, SKILL_REVIEW_PERSONA } from '../src/host/skill-review.ts' import { captureTurn, redact, sessionTitle, TrajectoryWindow, turnSkillUses } from '../src/host/skill-trajectory.ts' import { sessionLog } from './fixtures/session-log.ts' @@ -149,7 +150,7 @@ describe('skill evolution in the Host', () => { }) function evolution(answers: Array<(label: string, prompt: string) => unknown>) { - const skillTask = vi.fn(async (_parent: HostAgent, label: string, prompt: string) => { + const skillTask = vi.fn(async (_parent, label, prompt) => { const next = answers.shift() if (next === undefined) throw new Error('unexpected model call: ' + label) const value = next(label, prompt) @@ -503,6 +504,132 @@ describe('skill evolution in the Host', () => { expect(readFileSync(join(directory, 'dsh', 'skills', 'release-notes', 'SKILL.md'), 'utf8')).toBe(next) }) + it('regenerates a truncated draft under a fresh task with the configured larger budget', async () => { + config = { ...config, autoBacktest: false, draftMaxTokens: 32_768, draftRetryMaxTokens: 65_536 } + const file = join(directory, 'dsh', 'skills', 'release-notes', 'SKILL.md') + const original = readFileSync(file, 'utf8') + '发布说明:保留完整工作流。\n'.repeat(1_000) + writeFileSync(file, original) + const { skills, host, runTask } = evolution([]) + const tasks: string[] = [] + runTask.mockImplementation(async (_root, _signal, operation) => operation({ ...agent, id: `task-${runTask.mock.calls.length}` })) + const route = { provider: 'chosen-provider', model: 'chosen-model' } + host.skillTask.mockImplementation(async (task, _label, prompt, _schema, _persona, _signal, selected, maxTokens) => { + tasks.push(task.id) + expect(selected).toEqual(route) + expect(prompt.match(/发布说明:保留完整工作流。/gu)).toHaveLength(1_000) + expect(await source.read('drafts', { skill: 'release-notes' })).toEqual([]) + if (maxTokens === 32_768) throw new SubagentStoppedError('max-tokens', maxTokens) + expect(maxTokens).toBe(65_536) + return { structured: { content: original + '2. Group by component\n', notes: '', changes: ['Group by component'] } } + }) + await skills.draft(source, undefined, { skill: 'release-notes', route }) + await vi.waitFor(async () => expect(await source.read('drafts', { skill: 'release-notes' })).toHaveLength(1)) + expect(tasks).toEqual(['task-1', 'task-2']) + expect(readFileSync(file, 'utf8')).toBe(original) + expect(host.warn).not.toHaveBeenCalled() + }) + + it.each(['truncation', 'invalid-file'] as const)('preserves the saved draft and ledger when the final attempt ends in %s', async final => { + config = { ...config, autoBacktest: false, draftMaxTokens: 24_576, draftRetryMaxTokens: 49_152 } + const file = join(directory, 'dsh', 'skills', 'release-notes', 'SKILL.md') + const original = readFileSync(file, 'utf8') + '\n按组件分组。\n' + writeFileSync(file, original) + await source.mutate('draft-record', { skillName: 'release-notes', content: original, notes: 'Saved draft', changes: [], guidance: [] }) + const ledgerPath = join(directory, 'data', 'skills', 'ledger.json') + const ledger = readFileSync(ledgerPath, 'utf8') + const { skills, host, runTask } = evolution([]) + host.skillTask.mockImplementation(async (_task, _label, _prompt, _schema, _persona, _signal, _route, maxTokens) => { + if (maxTokens === 24_576 || final === 'truncation') throw new SubagentStoppedError('max-tokens', maxTokens) + return { structured: { content: '# incomplete skill file', notes: '', changes: [] } } + }) + await skills.draft(source, undefined, { skill: 'release-notes' }) + await vi.waitFor(() => expect(host.warn).toHaveBeenCalledOnce()) + expect(runTask).toHaveBeenCalledTimes(2) + expect(host.skillTask.mock.calls.map(call => call[7])).toEqual([24_576, 49_152]) + expect(readFileSync(ledgerPath, 'utf8')).toBe(ledger) + expect(readFileSync(file, 'utf8')).toBe(original) + if (final === 'truncation') { + const error = host.warn.mock.calls[0]![1] as Error + expect(error.message).toContain('24576, 49152') + expect(error.message).toContain(`${Buffer.byteLength(original, 'utf8')} bytes`) + expect(error.message).toContain('skills.draftMaxTokens') + expect(error.message).toContain('skills.draftRetryMaxTokens') + expect(error.message).toContain('provider') + } + }) + + it.each([16_384, 32_768])('leaves a suggestion pending without retry when the retry budget is %s', async retryBudget => { + config = { ...config, autoBacktest: false, draftRetryMaxTokens: retryBudget } + const { skills, host, runTask } = evolution([mixedReview]) + turn(events, 1, 'Group release notes by component.') + await skills.reviewNow(agent) + const suggestion = (await source.read('suggestions')).find(entry => entry.action === 'update')! + runTask.mockClear() + host.skillTask.mockImplementation(async () => { throw new SubagentStoppedError('max-tokens', 32_768, 'upstream detail '.repeat(40)) }) + await skills.draft(source, undefined, { suggestionId: suggestion.id }) + await vi.waitFor(() => expect(host.warn).toHaveBeenCalledOnce()) + expect(runTask).toHaveBeenCalledOnce() + const saved = (await source.read('suggestions')).find(entry => entry.id === suggestion.id)! + expect(saved).toMatchObject({ status: 'pending', lastError: expect.stringContaining('32768') }) + expect(saved.lastError).toContain('skills.draftRetryMaxTokens') + expect(saved.lastError).toContain('lower') + expect(saved.lastError).toContain('shorten the skill') + expect(saved.lastError).toMatch(/model with a (?:larger|higher)/u) + expect(saved.draftId).toBeUndefined() + expect(await source.read('drafts', { skill: 'release-notes' })).toEqual([]) + }) + + it('does not retry a truncated draft after user cancellation', async () => { + const { skills, host, runTask } = evolution([]) + host.skillTask.mockImplementation(async () => { + skills.dispose() + throw new SubagentStoppedError('max-tokens', 32_768) + }) + await skills.draft(source, undefined, { skill: 'release-notes' }) + await vi.waitFor(() => expect(host.warn).toHaveBeenCalledOnce()) + expect(runTask).toHaveBeenCalledOnce() + expect(await source.read('drafts', { skill: 'release-notes' })).toEqual([]) + }) + + it('does not start a third attempt when file-validation repair then hits max-tokens', async () => { + const { skills, host, runTask } = evolution([ + () => ({ content: '# missing frontmatter', notes: '', changes: [] }), + () => new SubagentStoppedError('max-tokens', 32_768), + ]) + await skills.draft(source, undefined, { skill: 'release-notes' }) + await vi.waitFor(() => expect(host.warn).toHaveBeenCalledOnce()) + expect(runTask).toHaveBeenCalledTimes(2) + expect(host.skillTask.mock.calls.map(call => call[7])).toEqual([32_768, 32_768]) + expect(await source.read('drafts', { skill: 'release-notes' })).toEqual([]) + }) + + it('does not save a completed model result after user cancellation', async () => { + const { skills, host, runTask } = evolution([]) + const file = join(directory, 'dsh', 'skills', 'release-notes', 'SKILL.md') + const original = readFileSync(file, 'utf8') + host.skillTask.mockImplementation(async () => { + skills.dispose() + return { structured: { content: original + '2. Group by component\n', notes: '', changes: [] } } + }) + await skills.draft(source, undefined, { skill: 'release-notes' }) + await vi.waitFor(() => expect(host.warn).toHaveBeenCalledOnce()) + expect(runTask).toHaveBeenCalledOnce() + expect(await source.read('drafts', { skill: 'release-notes' })).toEqual([]) + expect(readFileSync(file, 'utf8')).toBe(original) + }) + + it.each([ + new SubagentStoppedError('cancelled', 32_768), + new Error('provider rejected max_tokens'), + new Error('memory subagent stopped with max-tokens'), + ])('does not retry a draft error without a real max-tokens stop: %s', async error => { + const { skills, host, runTask } = evolution([() => error]) + await skills.draft(source, undefined, { skill: 'release-notes' }) + await vi.waitFor(() => expect(host.warn).toHaveBeenCalledOnce()) + expect(runTask).toHaveBeenCalledOnce() + expect(await source.read('drafts', { skill: 'release-notes' })).toEqual([]) + }) + it.each([false, true])('uses a fresh comparison baseline when a draft picks a model (picked=%s)', async picked => { await source.mutate('case-save', { case: { skillName: 'release-notes', title: 'Grouped notes', prompt: 'Write v2 notes', checks: [{ text: 'Grouped by component', must: true }] } }) const { skills, host } = evolution([]) diff --git a/tests/subagent.spec.ts b/tests/subagent.spec.ts index d8419b1d0..de2ed848e 100644 --- a/tests/subagent.spec.ts +++ b/tests/subagent.spec.ts @@ -3036,6 +3036,29 @@ describe('Mnemon memory subagent coordinator', () => { })) }) + it('passes a draft-specific output budget without changing other skill work', async () => { + const host = subagents({ answer: 'done' }) + const resultTools = toolRegistry() + const coordinator = new MnemonSubagentCoordinator(host.value, runtimeSource(), resultTools.value) + const route = { provider: 'chosen-provider', model: 'chosen-model' } + const schema = { type: 'object', properties: { answer: { type: 'string' } }, required: ['answer'], additionalProperties: false } + await coordinator.skillTask(parent(), 'Skill draft', 'Write a draft', schema, 'Write skills.', new AbortController().signal, route, 49_152) + await coordinator.skillTask(parent(), 'Skill judge', 'Judge the result', schema, 'Judge skills.', new AbortController().signal) + expect(host.start.mock.calls.map(([, request]) => request?.agentOptions)).toEqual([ + { ...route, maxTokens: 49_152 }, { maxTokens: 16_384 }, + ]) + }) + + it('reports truncation with its actual budget before parsing an incomplete skill result', async () => { + const host = subagents({ incomplete: true }, 'max-tokens') + const resultTools = toolRegistry() + const coordinator = new MnemonSubagentCoordinator(host.value, runtimeSource(), resultTools.value) + const schema = { type: 'object', properties: { answer: { type: 'string' } }, required: ['answer'], additionalProperties: false } + await expect(coordinator.skillTask(parent(), 'Skill draft', 'Write a draft', schema, 'Write skills.', new AbortController().signal, undefined, 32_768)) + .rejects.toMatchObject({ stopReason: 'max-tokens', maxTokens: 32_768 }) + expect(host.dispose).toHaveBeenCalledOnce() + }) + it('pins a fixed task Agent model onto the bounded idle review delegation', async () => { const host = subagents({ summary: 'No mutation needed.', action: 'skipped', memoryBodyIds: [] }, 'completed', ['spawn', 'fork']) const resultTools = toolRegistry()