contrib/avcodec: whole-track audio decode to PCM — load_pcm's producer

.github/scripts/contrib_check.sh

@@ -54,7 +54,7 @@ I18N="contrib/i18n"
 TESTS=(
   # avcodec: needs FFmpeg's dev libraries to LINK and the ffmpeg BINARY to
   # generate its clip; the test itself SKIPs cleanly without the latter.
-  "avcodec/decode|$AVC/test_avcodec.ae|$AVC/aether_avcodec.c|run|libavcodec libavformat libavutil libswscale"
+  "avcodec/decode|$AVC/test_avcodec.ae|$AVC/aether_avcodec.c|run|libavcodec libavformat libavutil libswscale libswresample"
   "tinyweb/spec|$TW/test_spec.ae||run|"
   "tinyweb/inventory|$TW/test_inventory.ae|$TW/ws_handshake.c|run|"
   "tinyweb/integration|$TW/test_integration.ae|$TW/ws_handshake.c|run|"

CHANGELOG.md

@@ -9,6 +9,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 `main`, the release pipeline automatically replaces `[current]` with the next
 version number before tagging the release.
 
+## [current]
+
+### Added
+
+- **`avcodec.audio_pcm(url)`** — decode a file's whole audio track to PCM, the
+  producer side of the `load_pcm` consumer that landed in 0.512.0. Together
+  they close the loop `asks/pcm-please.md` described: an MP4's audio can now
+  reach the speakers without a hand-extracted sidecar WAV.
+
+  Returns `(pcm, n, rate, channels, err)` — interleaved **s16 stereo at the
+  source rate**, in one shot. libswresample does the conversion in the same
+  pass, so a 5.1 float-planar AAC track (Big Buck Bunny's, for instance) comes
+  back as plain stereo s16 without the caller arranging anything. The
+  whole-buffer shape deliberately matches `audio.load_pcm`'s, so the two
+  compose directly:
+
+  ```aether
+  pcm, n, rate, ch, err = avcodec.audio_pcm(path)
+  src, e = audio.load_pcm(pcm, n, rate, ch, audio.FORMAT_S16)
+  ```
+
+  Verified end to end: a 117.3s clip decodes to 22,523,904 bytes at 48 kHz
+  stereo — exactly 117.3s — and `audio.load_pcm` then reports
+  `duration_ms=117312` with `position_ms` advancing in real time.
+
+  `contrib/avcodec` now requires **libswresample** alongside the other four
+  FFmpeg libraries. All five are required together; a partial install stays a
+  clean SKIP rather than a build failure.
+
 ## [0.512.0]
 
 ### Added

asks/pcm-please.md

@@ -0,0 +1,120 @@
+# std.audio: a PCM source, so a decoder can supply samples
+
+**From:** the aether-ui video line (2026-08-09) · **Where it bit:**
+`apps/video_frame`, which plays an MP4 in a UI frame with A/V sync working —
+but takes its audio from a hand-extracted sidecar WAV.
+
+## The ask, in one line
+
+A way to play PCM samples that the caller already has, rather than only
+container bytes that miniaudio can parse itself:
+
+```
+audio.load_pcm(data, length, sample_rate, channels, format) -> ptr!
+```
+
+Everything downstream — `play`, `pause`, `position_ms`, `duration_ms`,
+`seek_ms`, `volume` — should work on the returned source exactly as it does
+for `load_wav` today. `position_ms` in particular is the A/V-sync master
+clock, and it is the reason this matters.
+
+## Why the current surface cannot do it
+
+`load_wav` is better than its name: it is `ma_decoder_init_memory`, which
+sniffs the format, so it already accepts more than WAV. Measured on
+0.510.0:
+
+| Input | Result |
+| --- | --- |
+| WAV | accepted |
+| **MP3** | **accepted** — `duration_ms=6013` |
+| MP4 (whole file) | rejected: "unsupported or malformed audio data" |
+| raw AAC (`-c:a copy` out of the MP4) | rejected: same |
+
+So the gap is not "only WAV". It is that **every** entry point takes an
+encoded container miniaudio can demux, and there is no way in for samples a
+*different* decoder produced. (The `load_wav` name understates what it does
+and is worth revisiting separately — a caller reading the API would not
+guess MP3 works.)
+
+## What we are doing instead
+
+`contrib/avcodec` (in-process video decode, landed 0.510.0) removed the
+intermediate file for video: frames come straight from FFmpeg into a vg
+raster region. Audio still cannot make the same trip, so `video_frame`
+requires a manual pre-step:
+
+```
+ffmpeg -i clip.mp4 -vn -ar 44100 -ac 2 clip.wav     # 20 MB sidecar for a 21 MB source
+```
+
+That is the exact intermediate-file problem `contrib/avcodec` was written to
+eliminate, reappearing on the audio side — and it is worse than it looks:
+
+- it roughly **doubles on-disk cost** (20 MB sidecar for a 21 MB clip; a
+  feature film would be gigabytes of PCM);
+- it is a **manual step before playback**, so an app cannot just open a file
+  the user picked;
+- for a **live source** — a camera, a network stream, a generator — there is
+  no file to extract from and no workaround at all. Same shape as the
+  `fd_read_into` ask (#1471), which fixed the equivalent hole on the read
+  side.
+
+## Why this is the natural seam
+
+FFmpeg is already demuxing the container. `contrib/avcodec` opens the file,
+finds the video stream, and reads *past* the audio packets. Teaching it to
+decode those packets to PCM is a small, contained addition on our side — it
+already has the format context, and the frame-handoff pattern
+(`try_/get_/release_` plus a zero-allocation `_into` variant) is written and
+tested.
+
+What is missing is somewhere to put the samples. `std.audio` already owns
+the device, the mixer and the clock; it needs a source constructed from
+memory rather than from a decoded blob. In miniaudio terms that is
+`ma_audio_buffer` (or a custom `ma_data_source`) instead of `ma_decoder`,
+fed into the same `ma_sound_init_from_data_source` the shim already calls.
+
+## Two shapes, either would work
+
+**1. Whole-buffer PCM** (simpler; matches today's ownership model)
+
+```
+audio.load_pcm(data, length, sample_rate, channels, format) -> ptr!
+```
+
+The caller decodes fully, hands over the samples, and `std.audio` copies
+them the way `load_wav` copies its encoded input. Good enough for a clip
+that fits in memory, which covers the current demo and most app audio.
+
+**2. Streaming push** (the one that unblocks live sources)
+
+```
+audio.open_stream(sample_rate, channels, format) -> ptr!
+audio.push_pcm(src, data, length) -> int!     # bytes accepted; 0 = buffer full
+audio.stream_end(src)
+```
+
+A ring buffer the caller tops up from a decode loop, so nothing needs to fit
+in memory. This is what a camera, a network stream, or a two-hour film
+actually wants. It also raises a question worth answering deliberately:
+whether `position_ms` should then report the DEVICE's play position rather
+than a decoder offset — for A/V sync it must, since that is the clock video
+chases.
+
+Shape 1 alone would remove the sidecar for `video_frame`. Shape 2 is the
+one that makes `std.audio` usable for anything live.
+
+## Not urgent, and not blocking
+
+A/V sync is proven and correct today — video chases `audio.position_ms` to
+within 3 ms on a real 720p/5.1 clip. Nothing about the clock relationship
+changes with where the samples come from; this is packaging, not
+architecture. Filing it because the sidecar is the last hand-cranked step in
+an otherwise in-process pipeline.
+
+## Environment
+
+aether 0.510.0 (`92619ba1`), Linux/CachyOS. `std.audio` is miniaudio-backed
+with `MA_NO_ENCODING`; decoders are compiled in, which is why MP3 already
+works.

contrib/avcodec/module.ae

@@ -38,7 +38,10 @@ exports(
     avc_try_next_frame, avc_get_frame_bytes, avc_get_frame_length,
     avc_release_frame, avc_copy_frame_into_raw, avc_error_raw,
     open, close, width, height, frame_bytes, fps, pts_ms,
-    next_frame, next_frame_into, errmsg
+    next_frame, next_frame_into, errmsg,
+    avc_audio_decode_raw, avc_audio_get_pcm, avc_audio_get_pcm_length,
+    avc_audio_sample_rate, avc_audio_channels, avc_audio_release,
+    audio_pcm
 )
 
 extern avc_open_raw(url: string, want_w: int, want_h: int) -> ptr
@@ -55,6 +58,12 @@ extern avc_get_frame_length() -> int
 extern avc_release_frame()
 extern avc_copy_frame_into_raw(dec: ptr, buf: ptr, cap: int) -> int
 extern avc_error_raw(dec: ptr) -> string
+extern avc_audio_decode_raw(url: string) -> int
+extern avc_audio_get_pcm() -> string
+extern avc_audio_get_pcm_length() -> int
+extern avc_audio_sample_rate() -> int
+extern avc_audio_channels() -> int
+extern avc_audio_release()
 extern string_new_with_length(data: string, length: int) -> ptr
 
 // Open a media source. `want_w`/`want_h` <= 0 means "source size"; anything
@@ -102,3 +111,25 @@ next_frame_into(dec: ptr, buf: ptr, cap: int) -> {
     if n == 0 { return 0, "eof" }
     return n, ""
 }
+
+// Decode the file's ENTIRE audio stream to interleaved s16 stereo PCM at the
+// source sample rate -- the whole-buffer counterpart to std.audio's
+// load_pcm, which is where the result goes:
+//
+//     pcm, n, rate, ch, err = avcodec.audio_pcm(url)
+//     src, aerr = audio.load_pcm(pcm, n, rate, ch, audio.FORMAT_S16)
+//
+// Whatever the source is (5.1 float-planar AAC included), libswresample
+// downmixes and converts in one pass. ("", 0, 0, 0, err) when the file has
+// no audio stream -- callers treat that as "no audio", not a failure.
+audio_pcm(url: string) -> {
+    ok = avc_audio_decode_raw(url)
+    if ok == 0 { return "", 0, 0, 0, "no decodable audio stream in ${url}" }
+    raw = avc_audio_get_pcm()
+    n = avc_audio_get_pcm_length()
+    rate = avc_audio_sample_rate()
+    ch = avc_audio_channels()
+    owned = string_new_with_length(raw, n)
+    avc_audio_release()
+    return owned, n, rate, ch, ""
+}