Skip to content

feat(cosh): add clipboard image support and attachment ui to cli#584

Open
kongld-mq wants to merge 5 commits into
alibaba:mainfrom
kongld-mq:feat/cosh/support/pasting/image
Open

feat(cosh): add clipboard image support and attachment ui to cli#584
kongld-mq wants to merge 5 commits into
alibaba:mainfrom
kongld-mq:feat/cosh/support/pasting/image

Conversation

@kongld-mq
Copy link
Copy Markdown
Collaborator

@kongld-mq kongld-mq commented May 21, 2026

Description

Add clipboard image paste support for the CLI prompt, enabling users to paste images directly from clipboard into the input field. The pasted images are saved to a global temporary directory and displayed as attachments above the input prompt, which are automatically converted to file references when submitting.

Key features:

  • Cross-platform clipboard image detection using @teddyzhu/clipboard library
  • Image storage in workspace/.copilot-shell/tmp/clipboard/ (within workspace for @ file processor access)
  • Attachment UI with keyboard navigation (↑/↓ to manage, ←/→ to select, Delete to remove)
  • LRU cleanup for old clipboard images (max 100 files, 7 days retention)
  • Platform-specific keyboard shortcut hints (macOS: Cmd+V, Windows/Linux: Ctrl+V)
  • Internationalization support (English/Chinese)
  • Supports submission with attachments only (no text required)

Related Issue

closes #

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional change)
  • Performance improvement
  • CI/CD or build changes

Scope

  • cosh (copilot-shell)
  • sec-core (agent-sec-core)
  • skill (os-skills)
  • sight (agentsight)
  • tokenless (tokenless)
  • Multiple / Project-wide

Checklist

  • I have read the Contributing Guide
  • My code follows the project's code style
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation accordingly
  • For cosh: Lint passes, type check passes, and tests pass
  • For sec-core (Rust): cargo clippy -- -D warnings and cargo fmt --check pass
  • For sec-core (Python): Ruff format and pytest pass
  • For skill: Skill directory structure is valid and shell scripts pass syntax check
  • For sight: cargo clippy -- -D warnings and cargo fmt --check pass
  • For tokenless: cargo clippy -- -D warnings and cargo fmt --check pass
  • Lock files are up to date (package-lock.json / Cargo.lock)

Testing

Unit Tests:

  • clipboardUtils.test.ts: 9 tests for clipboard image detection and saving
  • InputPrompt.test.tsx: Tests for paste handling, error handling, and attachment UI
  • keyMatchers.test.tsx: Tests for platform-specific keybinding matching
    All tests pass: 3422 passed | 7 skipped

Manual Testing Steps:

  1. Build the CLI: npm run build
  2. Start the CLI: npm run start
  3. Copy an image to clipboard (on macOS: screenshot or copy from browser)
  4. Press Ctrl+V (or Cmd+V on macOS - note: terminal may intercept Cmd+V)
  5. Verify attachment appears above input prompt
  6. Use ↑ to navigate to attachments, ←/→ to select, Delete to remove
  7. Press Enter to submit (works with empty text + attachment)
  8. Check workspace/.copilot-shell/tmp/clipboard/ for saved images
  9. Verify the image path is correctly passed to model context via @ file processor

Additional Notes

New Dependency:

@teddyzhu/clipboard: Cross-platform clipboard access library with native bindings for macOS, Linux, and Windows

macOS Terminal Behavior Notes:

  • The UI displays Cmd+V as the keyboard shortcut hint (matching qwen-code's approach for user familiarity)
  • However, macOS terminals typically intercept Cmd+V as system paste, sending the pasted text directly to the terminal input buffer
  • For clipboard image paste detection, users should use Ctrl+V instead, which the terminal passes through as a key event that the CLI can handle
  • This is a known limitation of terminal applications on macOS; the displayed shortcut hint may differ from the actual working key combination

@github-actions github-actions Bot added the component:cosh src/copilot-shell/ label May 21, 2026
Copy link
Copy Markdown
Collaborator

@kongche-jbw kongche-jbw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Blocker: 图片附件提交后可能不会被实际读取

当前实现把图片保存到全局临时目录 ~/.copilot-shell/tmp/clipboard,但提交时又转成相对 workspace 的 @... 引用。现有 @ 文件处理器会校验路径是否在 workspace 内,workspace 外路径会被跳过。结果是 UI 显示附件已添加,但提交后图片可能不会进入模型上下文。

建议修复方式之一:

  • 将图片保存到 workspace/project temp 中,并确保该路径可被 @ 文件读取链路访问;
  • 或者不要通过 @path 文本链路传附件,而是在提交时直接把附件转换成模型可消费的 image/file part;
  • 补一个端到端或集成测试,覆盖“粘贴图片 -> 提交 -> 图片被实际读入请求”。
  1. Windows 快捷键与 PR 描述不一致

PR 描述说 Windows/Linux 使用 Ctrl+V 粘贴图片,但 win32 分支当前只绑定了 command/meta + v,没有绑定 ctrl + v。这会导致 Windows 用户按说明操作时无法触发图片粘贴。

建议:

  • Windows 分支增加 { key: 'v', ctrl: true }
  • 补充/更新 key matcher 测试覆盖 Windows Ctrl+V
  1. 只贴图片、不输入文字时无法提交

当前提交逻辑只有在 buffer.text.trim() 非空时才会触发。用户只粘贴一张图片然后直接回车时,输入为空,因此不会提交,表现为静默无响应。

建议:

  • 提交条件改为类似 buffer.text.trim() || attachments.length > 0
  • 当只有附件无文本时,也应生成有效提交内容;
  • 补测试覆盖“空文本 + 附件 + 回车提交”。

@kongld-mq
Copy link
Copy Markdown
Collaborator Author

  1. Blocker: 图片附件提交后可能不会被实际读取

当前实现把图片保存到全局临时目录 ~/.copilot-shell/tmp/clipboard,但提交时又转成相对 workspace 的 @... 引用。现有 @ 文件处理器会校验路径是否在 workspace 内,workspace 外路径会被跳过。结果是 UI 显示附件已添加,但提交后图片可能不会进入模型上下文。

建议修复方式之一:

  • 将图片保存到 workspace/project temp 中,并确保该路径可被 @ 文件读取链路访问;
  • 或者不要通过 @path 文本链路传附件,而是在提交时直接把附件转换成模型可消费的 image/file part;
  • 补一个端到端或集成测试,覆盖“粘贴图片 -> 提交 -> 图片被实际读入请求”。
  1. Windows 快捷键与 PR 描述不一致

PR 描述说 Windows/Linux 使用 Ctrl+V 粘贴图片,但 win32 分支当前只绑定了 command/meta + v,没有绑定 ctrl + v。这会导致 Windows 用户按说明操作时无法触发图片粘贴。

建议:

  • Windows 分支增加 { key: 'v', ctrl: true }
  • 补充/更新 key matcher 测试覆盖 Windows Ctrl+V
  1. 只贴图片、不输入文字时无法提交

当前提交逻辑只有在 buffer.text.trim() 非空时才会触发。用户只粘贴一张图片然后直接回车时,输入为空,因此不会提交,表现为静默无响应。

建议:

  • 提交条件改为类似 buffer.text.trim() || attachments.length > 0
  • 当只有附件无文本时,也应生成有效提交内容;
  • 补测试覆盖“空文本 + 附件 + 回车提交”。

感谢详细的 review feedback,三个问题均已修复:

问题 1:图片保存路径
修复方案:采用建议方案 1,将图片保存路径从 ~/.copilot-shell/tmp/clipboard/ 改为 workspace/.copilot-shell/tmp/clipboard/,确保路径在 workspace 内,@ 文件处理器可正常读取。
问题 2:Windows Ctrl+V 快捷键
修复方案:Windows 分支增加 { key: 'v', ctrl: true } 绑定,并更新 key matcher 测试覆盖 Windows Ctrl+V。
问题 3:空文本 + 附件提交
修复方案:提交条件改为 buffer.text.trim() || attachments.length > 0,允许空文本 + 附件提交,并补充测试覆盖该场景。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:cosh src/copilot-shell/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants