Skip to content

Keep only the output dtype config on VideoStreamOptions - #1698

Closed
NicolasHug wants to merge 1 commit into
packet-seam-const-avpacketfrom
output-dtype-config-only
Closed

Keep only the output dtype config on VideoStreamOptions#1698
NicolasHug wants to merge 1 commit into
packet-seam-const-avpacketfrom
output-dtype-config-only

Conversation

@NicolasHug

Copy link
Copy Markdown
Contributor

OutputDtype is resolve_output_dtype(config, source pixel format) - derived,
not configured - and VideoStreamOptions held it next to the config it is
derived from. That made the struct half-valid until someone resolved, and
the two users resolved it in ways the shared field couldn't express:
SingleStreamDecoder wrote the answer back into its own options once per
stream, while ColorConverter, which has to resolve per frame because AUTO
"can differ from one frame to the next", fabricated a whole
VideoStreamOptions for the sole purpose of carrying the answer across
initialize_color_conversion().

So make the resolved dtype an argument of initialize_color_conversion(),
the only stage that needs it, and leave VideoStreamOptions with the config
alone - one field, meaningful from the moment it is constructed.
SingleStreamDecoder's once-per-stream answer moves to StreamInfo, where
decoder state belongs.

Worth knowing while reading this: the two resolutions take their pixel
format from different places, the stream header in SingleStreamDecoder and
the frame itself in ColorConverter, so they can disagree. Passing the
resolved value explicitly doesn't fix that, but it does stop a single
struct field from implying they are one thing.

initialize_color_conversion() and initialize_video() both gain a parameter,
which out-of-tree device interfaces will need to follow.


Stack created with GitHub Stacks CLIGive Feedback 💬

OutputDtype is resolve_output_dtype(config, source pixel format) - derived,
not configured - and VideoStreamOptions held it next to the config it is
derived from. That made the struct half-valid until someone resolved, and
the two users resolved it in ways the shared field couldn't express:
SingleStreamDecoder wrote the answer back into its own options once per
stream, while ColorConverter, which has to resolve per frame because AUTO
"can differ from one frame to the next", fabricated a whole
VideoStreamOptions for the sole purpose of carrying the answer across
initialize_color_conversion().

So make the resolved dtype an argument of initialize_color_conversion(),
the only stage that needs it, and leave VideoStreamOptions with the config
alone - one field, meaningful from the moment it is constructed.
SingleStreamDecoder's once-per-stream answer moves to StreamInfo, where
decoder state belongs.

Worth knowing while reading this: the two resolutions take their pixel
format from different places, the stream header in SingleStreamDecoder and
the frame itself in ColorConverter, so they can disagree. Passing the
resolved value explicitly doesn't fix that, but it does stop a single
struct field from implying they are one thing.

initialize_color_conversion() and initialize_video() both gain a parameter,
which out-of-tree device interfaces will need to follow.
@pytorch-bot

pytorch-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1698

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 17 Pending

As of commit 02a6273 with merge base 2312413 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 4, 2026
@NicolasHug

Copy link
Copy Markdown
Contributor Author

Folded into #1696 — the two changes are one story (the output dtype config becoming the only dtype on VideoStreamOptions), so they're now a single commit there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant