Phonon: run the Mimi decoder's transposed convolutions as convolutions on XNNPACK (#25004) by psiddh · Pull Request #25004 · pytorch/executorch

psiddh · 2026-11-12T23:58:50Z

Summary:

XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames.

Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis.

This diff was written with the help of Claude Code (Claude Opus 5.5).

Reviewed By: shoumikhin, JakeStevens

Differential Revision: D131605676

pytorch-bot · 2026-11-12T23:58:54Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25004

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 1 Pending

As of commit a41a877 with merge base 3e85cae ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

meta-codesync · 2026-11-12T23:59:27Z

@psiddh has exported this pull request. If you are a Meta employee, you can view the originating Diff in D131605676.

github-actions · 2026-11-13T00:00:11Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…s on XNNPACK (pytorch#25004) Summary: Pull Request resolved: pytorch#25004 XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Differential Revision: D131605676

JakeStevens

Review automatically exported from Phabricator review in Meta.

…s on XNNPACK (pytorch#25004) Summary: Pull Request resolved: pytorch#25004 XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: JakeStevens Differential Revision: D131605676

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

…s on XNNPACK (pytorch#25004) Summary: Pull Request resolved: pytorch#25004 XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: JakeStevens Differential Revision: D131605676

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

…s on XNNPACK (pytorch#25004) Summary: Pull Request resolved: pytorch#25004 XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: JakeStevens Differential Revision: D131605676

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

…s on XNNPACK (pytorch#25004) Summary: XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: shoumikhin, JakeStevens Differential Revision: D131605676

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

…s on XNNPACK (pytorch#25004) Summary: XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: shoumikhin, JakeStevens Differential Revision: D131605676

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

…s on XNNPACK (pytorch#25004) Summary: XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames. Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: shoumikhin, JakeStevens Differential Revision: D131605676

psiddh requested review from SS-JIA, digantdesai and larryliu0820 as code owners November 12, 2026 23:58

Copilot AI balanced review requested due to automatic review settings November 12, 2026 23:58

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 12, 2026

Copilot AI reviewed Nov 12, 2026

meta-codesync Bot added the meta-exported label Nov 12, 2026

JakeStevens approved these changes Nov 13, 2026

View reviewed changes

This was referenced Nov 13, 2026

[XNNPACK] Lower dense transposed convs as one polyphase conv + depth to space #25037

Draft

[XNNPACK] One polyphase pass for dense and depthwise transposed convs #25041

Draft

meta-codesync Bot changed the title ~~Phonon: run the Mimi decoder's transposed convolutions as convolutions on XNNPACK~~ Phonon: run the Mimi decoder's transposed convolutions as convolutions on XNNPACK (#25004) Nov 13, 2026

psiddh force-pushed the export-D131605676 branch from 3495700 to 6d81c32 Compare November 13, 2026 17:24

Copilot AI balanced review requested due to automatic review settings November 13, 2026 17:24

Copilot AI reviewed Nov 13, 2026

psiddh force-pushed the export-D131605676 branch from 6d81c32 to 0d18bb7 Compare November 13, 2026 17:30

Copilot AI balanced review requested due to automatic review settings November 13, 2026 17:30

Copilot AI reviewed Nov 13, 2026

psiddh force-pushed the export-D131605676 branch from 0d18bb7 to 49a9227 Compare November 13, 2026 19:08

Copilot AI balanced review requested due to automatic review settings November 13, 2026 19:08

Copilot AI reviewed Nov 13, 2026

psiddh force-pushed the export-D131605676 branch from 49a9227 to 44ff6b4 Compare November 14, 2026 03:18

Copilot AI balanced review requested due to automatic review settings November 14, 2026 03:18

Copilot AI reviewed Nov 14, 2026

psiddh force-pushed the export-D131605676 branch from 44ff6b4 to a41a877 Compare November 14, 2026 07:50

Copilot AI balanced review requested due to automatic review settings November 14, 2026 07:50

Copilot AI reviewed Nov 14, 2026

Navigation Menu

Repository navigation

Phonon: run the Mimi decoder's transposed convolutions as convolutions on XNNPACK (#25004) - #25004

Phonon: run the Mimi decoder's transposed convolutions as convolutions on XNNPACK (#25004)#25004
psiddh wants to merge 1 commit into
pytorch:mainfrom
psiddh:export-D131605676

psiddh commented Nov 12, 2026 •

edited by meta-codesync Bot

Loading

Uh oh!

pytorch-bot Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

meta-codesync Bot commented Nov 12, 2026

Uh oh!

github-actions Bot commented Nov 13, 2026

Uh oh!

JakeStevens left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

psiddh commented Nov 12, 2026 • edited by meta-codesync Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25004

⏳ No Failures, 1 Pending

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

meta-codesync Bot commented Nov 12, 2026

Uh oh!

github-actions Bot commented Nov 13, 2026

This PR needs a release notes: label

Uh oh!

JakeStevens left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

psiddh commented Nov 12, 2026 •

edited by meta-codesync Bot

Loading

pytorch-bot Bot commented Nov 12, 2026 •

edited

Loading

This PR needs a `release notes:` label