Skip to content

Improve truncated transcribe.cpp results#1627

Closed
vladkalinichencko wants to merge 1 commit into
cjpais:mainfrom
vladkalinichencko:fix/transcribe-partial-results
Closed

Improve truncated transcribe.cpp results#1627
vladkalinichencko wants to merge 1 commit into
cjpais:mainfrom
vladkalinichencko:fix/transcribe-partial-results

Conversation

@vladkalinichencko

@vladkalinichencko vladkalinichencko commented Jul 7, 2026

Copy link
Copy Markdown

Summary

This PR improves long transcriptions for transcribe.cpp models that can produce useful text but stop before emitting an end-of-stream token.

The main change is that Handy now preserves partial transcripts from truncated transcribe.cpp runs instead of treating them as a total failure. The PR also exposes a generation-token limit through the local transcribe.cpp binding and raises the Qwen3-ASR cap used by Handy, so longer Qwen outputs are not forced to stop at the previous short default.

History re-transcribe failures now show the underlying error details in the toast as well.

Why

Some models can hit their decoder generation limit even when the audio itself is within the model's supported input length. In that case the app previously discarded the text that had already been produced. This made a recoverable result look like a failed transcription.

For Qwen3-ASR, the previous generation cap was also too small for longer dictation-style recordings. Raising that cap allows the larger Qwen model to finish recordings that were previously cut off early.

Validation

  • cargo check --manifest-path src-tauri/Cargo.toml
  • bun run build
  • Tested local Qwen3-ASR 1.7B transcription on a multi-minute recording; it completed without truncating at the previous 256-token limit.
  • Tested local Qwen3-ASR 0.6B truncation; Handy now returns the partial transcript instead of failing the transcription.

@vladkalinichencko
vladkalinichencko force-pushed the fix/transcribe-partial-results branch from 6a297b1 to aabe333 Compare July 7, 2026 19:08
@vladkalinichencko vladkalinichencko changed the title Return partial transcripts on truncation Improve truncated transcribe.cpp results Jul 7, 2026
@vladkalinichencko
vladkalinichencko marked this pull request as ready for review July 7, 2026 19:11
@cjpais

cjpais commented Jul 7, 2026

Copy link
Copy Markdown
Owner

500k lines changed...

@cjpais cjpais closed this Jul 7, 2026
@vladkalinichencko

Copy link
Copy Markdown
Author

I accidentally vendored transcribe.cpp when upstreaming API change, sorry. My fix was for ~30 lines, but i patched transcribe.cpp, because for some reason they just didn't include the corresponding parameter for Qwen/Canary generation length through the current C/Rust API, which makes audio generation for more than ~2.5 mins impossible for these models, while, unrestricted, they can handle it pretty well

@cjpais

cjpais commented Jul 8, 2026

Copy link
Copy Markdown
Owner

please feel free to resubmit, or submit respective PR's to each repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants