Skip to content

feat(audio): add load_pcm — play samples the caller already decoded - #1481

Merged
paul-hammant merged 2 commits into
mainfrom
feat/audio-load-pcm
Aug 9, 2026
Merged

feat(audio): add load_pcm — play samples the caller already decoded#1481
paul-hammant merged 2 commits into
mainfrom
feat/audio-load-pcm

Conversation

@paul-hammant

Copy link
Copy Markdown
Collaborator

Implements asks/pcm-please.md.

The gap

std.audio could only take an encoded container. load_wav is better than its name — it is ma_decoder_init_memory, so it sniffs the format and mp3/flac already work — but every entry point wanted bytes miniaudio could demux itself. That left no way in for samples a different decoder produced, which is exactly the case once contrib.avcodec has demuxed an MP4 and is holding the audio packets.

The workaround was pre-extracting a sidecar WAV:

  • roughly doubles on-disk cost — a 20 MB sidecar for a 21 MB clip, gigabytes for a feature film
  • a manual step before playback, so an app cannot just open a file the user picked
  • nothing at all for a live source with no file to extract from

That is the same intermediate-file problem contrib/avcodec was written to remove, reappearing on the audio side.

What this adds

audio.load_pcm(data, length, sample_rate, channels, format) -> ptr!

The ask's shape 1 (whole-buffer PCM), which it states is enough on its own to remove the sidecar.

The implementation is small because the existing code was well shaped: every downstream function reads s->sound, never s->decoder. So feeding an ma_audio_buffer into the same ma_sound_init_from_data_source the encoded path uses gives the whole transport surface unchanged. Verified live:

duration_ms   = 1000      ← 1s of 44.1kHz stereo s16; wrong frame maths shows as 500 or 2000
channels      = 2
position_ms   = 0
seek_ms(500) → position_ms = 500      ← the A/V-sync master clock works

Sample formats are exposed as audio.FORMAT_U8 / _S16 / _S24 / _S32 / _F32 so callers never hardcode miniaudio's numbering. The values were checked against the ma_format enum in miniaudio.h (u8=1, s16=2, s24=3, s32=4, f32=5) rather than assumed — a wrong constant there would be a silent correctness bug, not a build error.

length must be a whole number of frames; a partial trailing frame is refused rather than played as noise off the end of the last whole frame.

Deliberately not done

Shape 2 — the streaming push/ring-buffer API for live sources. The ask itself raises the open question it depends on: whether position_ms should then report the device's play position rather than a decoder offset. For A/V sync it must, and that is a design decision worth its own change rather than being guessed at alongside this one.

Verification

  • make ci — C suite 230/230; .ae 978/979 with [PASS] regression_test_audio_load_pcm, count up 978 → 979. The one failure is integration_http_server_h2, pre-existing and diagnosed in pesky_bug.md.
  • Test verified to have teeth — removing the partial-frame guard makes it fail, so it cannot silently rot into accepting bad geometry.
  • Valgrind clean — worth stating, since unload now has an ma_audio_buffer to uninit alongside the decoder.
  • gcc -Werror checked locally before pushing.
  • The test SKIPs cleanly with no audio device, which is the normal CI-runner case.

CHANGELOG

There was no [current] section, so this adds one. I checked whether the VERSION 0.511.0 / newest-heading [0.510.0] mismatch was another gap: 0.511.0's window contains no feature merges — it was the no-op bump from the docs-only PR #1478 — so the absent heading is accurate, and inventing an empty [0.511.0] section would have been worse than leaving it. See #1477 for the pipeline behaviour behind that.

🤖 Generated with Claude Code

paul-hammant and others added 2 commits August 9, 2026 21:04
Implements asks/pcm-please.md.

std.audio could only take an ENCODED container. load_wav is better than its
name — it is ma_decoder_init_memory, so it sniffs the format and mp3/flac
already work — but every entry point wanted bytes miniaudio could demux
itself. That left no way in for samples a DIFFERENT decoder produced, which is
exactly the case once contrib.avcodec has demuxed an MP4 and is holding the
audio packets.

The workaround was pre-extracting a sidecar WAV: roughly double the on-disk
cost (a 20 MB sidecar for a 21 MB clip, gigabytes for a feature film), a manual
step before playback so an app cannot just open a file the user picked, and
nothing at all for a live source with no file to extract from. That is the same
intermediate-file problem contrib/avcodec was written to remove, reappearing on
the audio side.

audio.load_pcm(data, length, sample_rate, channels, format) is the ask's shape
1 — whole-buffer PCM, which the ask says is enough on its own to remove the
sidecar. Shape 2 (a streaming push/ring-buffer API for live sources) is
deliberately NOT attempted here; it needs a decision about whether position_ms
should then report the device's play position rather than a decoder offset,
and that is a design question worth its own change.

Implementation is small because the existing code was well shaped: every
downstream function reads s->sound, never s->decoder, so feeding an
ma_audio_buffer into the SAME ma_sound_init_from_data_source gives play,
pause, position_ms, duration_ms, seek_ms and volume for free. position_ms in
particular keeps working as the A/V-sync master clock, which is the reason the
ask matters.

Sample formats are exposed as audio.FORMAT_U8/_S16/_S24/_S32/_F32 rather than
leaving callers to hardcode miniaudio's numbering. The values were verified
against the ma_format enum in miniaudio.h (u8=1, s16=2, s24=3, s32=4, f32=5)
rather than assumed.

length must be a whole number of frames; a partial trailing frame is refused
rather than played as noise off the end of the last whole frame.

Tests: tests/regression/test_audio_load_pcm.ae — a caller-supplied buffer
loads, duration_ms is right (so the frame maths is right), seek_ms/position_ms
work, and bad geometry/format is refused. Verified the test fails when the
partial-frame guard is removed, so it cannot silently rot. Valgrind-clean,
which matters because unload now has an ma_audio_buffer to uninit as well.
SKIPs cleanly with no audio device, the normal CI-runner case.

Verified under gcc -Werror as well as the normal build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Windows CI failed this test on PR #1481 (a change that touches only std.audio,
so the test was collateral, not caused by it):

    FAIL read errored: fd_read_into failed
    5 FAILURE(S)

os_run_pipe_raw lives inside `#ifndef _WIN32`; the Windows build returns a stub
whose read fd is -1. Wiring a real one up needs coordinated _open_osfhandle on
both sides, which aether_os.c:1369 records as deliberately not done.

My original guard only checked `err != ""`, and the stub returns fd=-1 with an
EMPTY error, so the guard never fired and the test proceeded to read from -1.
Two guards now: an explicit fd < 0 check, and treating a failure on the very
FIRST read as "this fd is not readable here" rather than a regression. A
failure on any LATER read is still a real failure, so the test keeps its teeth
on platforms where the fd works.

Verified on winbaz (real MSYS2 MINGW64) rather than by another CI round-trip:

    sh: line 1: 3: Bad file descriptor      <- the child's >&3 also fails
    SKIP: run_pipe gave no readable fd on this platform (fd=-1)

Linux still runs the real assertions — all 7 pass.

Note for anyone diagnosing this on winbaz: build/libaether.a there was stale
and missing io_fd_read_into_raw, which produced a misleading "undefined
reference" at link time and sent me looking for a missing #ifdef branch. CI
builds fresh and had no link error at all; `make stdlib` on the box clears it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@paul-hammant

Copy link
Copy Markdown
Collaborator Author

The Windows failure was not from this change — std.audio is untouched by it and regression_test_audio_load_pcm passed on that runner. The failing test was regression_test_fd_read_into, which I added in #1476.

Cause: os_run_pipe_raw lives inside #ifndef _WIN32. The Windows build returns a stub whose read fd is -1 — wiring up a real one needs coordinated _open_osfhandle on both sides, which aether_os.c:1369 records as deliberately not done.

My guard only checked err != "", and the stub returns fd=-1 with an empty error, so it never fired and the test read from -1.

Fixed in 3500b71e with two guards: an explicit fd < 0 check, and treating a failure on the first read as "not readable here" rather than a regression. A failure on any later read is still real, so the test keeps its teeth where the fd works.

Verified on winbaz (real MSYS2 MINGW64) rather than another CI round-trip:

sh: line 1: 3: Bad file descriptor      <- the child's >&3 fails too
SKIP: run_pipe gave no readable fd on this platform (fd=-1)

Linux still runs the real assertions — all 7 pass.

@paul-hammant
paul-hammant merged commit 9c9ae60 into main Aug 9, 2026
23 checks passed
@paul-hammant
paul-hammant deleted the feat/audio-load-pcm branch August 9, 2026 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant