Skip to content

refactor: replace ffmpeg shell-out with pure Rust audio decoding - #15

Merged
robdefeo merged 2 commits into
mainfrom
claude/strange-euclid-ef07fc
Apr 23, 2026
Merged

refactor: replace ffmpeg shell-out with pure Rust audio decoding#15
robdefeo merged 2 commits into
mainfrom
claude/strange-euclid-ef07fc

Conversation

@robdefeo

Copy link
Copy Markdown
Owner

Summary

  • Removes the hard runtime dependency on the system ffmpeg binary — voxscribe now ships as a single self-contained binary on all platforms
  • symphonia decodes mp3, aac, alac, ogg/vorbis, flac, wav, and isomp4 (m4a/mp4) natively; rubato resamples any source rate to 16 kHz
  • Downmix uses arithmetic mean for stereo and ITU-R BS.775 coefficients for surround content, preserving the centre channel (dialogue) for Whisper accuracy
  • AppError variants FfmpegNotFound/FfmpegFailed replaced by DecodeFailed/ResampleFailed
  • hound and tempfile moved to [dev-dependencies]

Test plan

  • just lint — clippy + fmt clean
  • just test — 18 tests pass, including 5 new audio tests: identity decode, 44.1 kHz resample, stereo downmix, chunk-aligned tail (empty-tail rubato edge case), and sub-chunk-length input
  • Manual smoke test against a real .m4a Voice Memo — transcription successful
  • Single-binary check: works without ffmpeg in PATH

Closes #5

Removes the hard runtime dependency on the system ffmpeg binary.
Decode, downmix, and resample are now done in-process via symphonia
and rubato, enabling single-binary distribution on all platforms.

- symphonia decodes mp3, aac, alac, ogg/vorbis, flac, wav, isomp4 (m4a/mp4)
- rubato FftFixedIn resamples any source rate to 16 kHz
- Downmix uses arithmetic mean for stereo and ITU-R BS.775 coefficients
  for surround content, preserving centre-channel dialogue for Whisper
- AppError variants FfmpegNotFound/FfmpegFailed replaced by
  DecodeFailed/ResampleFailed
- hound and tempfile moved to dev-dependencies (used only in tests)
- Five new unit tests cover identity decode, 44.1 kHz resample,
  stereo downmix, chunk-aligned tail (the empty-tail rubato edge case),
  and sub-chunk-length input

Closes #5
@greptile-apps

greptile-apps Bot commented Apr 23, 2026

Copy link
Copy Markdown

Greptile Summary

This PR replaces the hard runtime dependency on the system ffmpeg binary with a pure-Rust pipeline using symphonia for demuxing/decoding and rubato (FftFixedIn) for resampling, making voxscribe a self-contained single binary. All concerns flagged in prior review threads (stale spec on ResetRequired, SampleBuffer capacity growth for variable-frame-size codecs, debug_assert for interleaved alignment) have been addressed in the current HEAD.

Confidence Score: 5/5

Safe to merge — no correctness or security issues remain; the one open comment is a minor efficiency P2.

All prior P1 concerns are resolved. The only remaining finding is a redundant channel_count == 1 guard that causes an extra to_vec() clone for multi-channel 16 kHz sources — a P2 style issue that does not affect correctness. Eight new unit tests cover identity decode, resample, stereo downmix, chunk-aligned tail, sub-chunk input, 5.1 surround downmix, and corrupt-file rejection.

No files require special attention.

Important Files Changed

Filename Overview
src/audio.rs Core audio pipeline replaced with symphonia + rubato; decode loop handles ResetRequired correctly (spec/sample_buf both cleared), SampleBuffer is grown on capacity increase, downmix covers stereo (arithmetic mean) and surround (ITU-R BS.775 with fallback), resample tail flush handles both chunk-aligned and sub-chunk inputs; 8 new unit tests covering all new code paths.
src/error.rs FfmpegNotFound/FfmpegFailed replaced by DecodeFailed/ResampleFailed; straightforward and correct.
Cargo.toml symphonia (0.5) and rubato (0.16) added as production deps; hound and tempfile correctly moved to dev-dependencies, eliminating them from the release binary.
Cargo.lock Lock file updated to reflect new transitive deps (rubato, rustfft, realfft, symphonia-, arrayvec, bytemuck, num-, etc.); all checksums are from the crates.io registry.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[load input path] --> B[validate extension & existence]
    B --> C[decode_to_mono_16k]
    C --> D[Open file + MediaSourceStream]
    D --> E[symphonia probe format]
    E --> F[Find first decodable track]
    F --> G[Create decoder]
    G --> H{next_packet}
    H -->|UnexpectedEof| I[break]
    H -->|ResetRequired| J[Recreate decoder\nspec=None\nsample_buf=None]
    J --> H
    H -->|other track| H
    H -->|Ok packet| K{decode packet}
    K -->|DecodeError| H
    K -->|UnexpectedEof| I
    K -->|Ok decoded| L[copy_interleaved_ref\nextend interleaved]
    L --> H
    I --> M{interleaved empty?}
    M -->|yes| N[DecodeFailed error]
    M -->|no| O[downmix_to_mono]
    O --> P{Already 16kHz?}
    P -->|yes| Q[Return mono samples]
    P -->|no| R[resample_to_target FftFixedIn rubato]
    R --> S[Chunked resample loop]
    S --> T[process_partial_into_buffer tail flush]
    T --> U[Return resampled Vec f32]
Loading
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/audio.rs
Line: 135-137

Comment:
**Redundant `channel_count == 1` guard causes an extra clone for multi-channel 16 kHz input**

After `downmix_to_mono`, `mono` is always single-channel regardless of the source layout. The `channel_count == 1` check therefore only fast-paths when the source was *already* mono; for stereo (or surround) input that is already at 16 kHz the code falls through into `resample_to_target`, which immediately hits its own `src_rate == TARGET_RATE` guard and returns `Ok(mono.to_vec())` — an unnecessary allocation and copy.

The condition can simply be `spec.rate == TARGET_RATE`:

```suggestion
    if spec.rate == TARGET_RATE {
        return Ok(mono);
    }
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (2): Last reviewed commit: "fix(audio): address review feedback on d..." | Re-trigger Greptile

Comment thread src/audio.rs
Comment thread src/audio.rs
Comment thread src/audio.rs
- Reset `spec` alongside `sample_buf` on ResetRequired so stale
  channel layout and sample rate are not used after a codec reset
- Regrow SampleBuffer when a packet's capacity exceeds the current
  buffer size, preventing panic on variable-block codecs (FLAC)
- Add debug_assert for interleaved sample count divisibility
- Guard weight_sum with f32::EPSILON instead of == 0.0
- Add 5.1 surround downmix unit test via direct downmix_to_mono call
@robdefeo
robdefeo merged commit 599e2b2 into main Apr 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: replace ffmpeg shell-out with pure Rust audio decoding

1 participant