Keep only the output dtype config on VideoStreamOptions - #1698
Closed
NicolasHug wants to merge 1 commit into
Closed
Conversation
OutputDtype is resolve_output_dtype(config, source pixel format) - derived, not configured - and VideoStreamOptions held it next to the config it is derived from. That made the struct half-valid until someone resolved, and the two users resolved it in ways the shared field couldn't express: SingleStreamDecoder wrote the answer back into its own options once per stream, while ColorConverter, which has to resolve per frame because AUTO "can differ from one frame to the next", fabricated a whole VideoStreamOptions for the sole purpose of carrying the answer across initialize_color_conversion(). So make the resolved dtype an argument of initialize_color_conversion(), the only stage that needs it, and leave VideoStreamOptions with the config alone - one field, meaningful from the moment it is constructed. SingleStreamDecoder's once-per-stream answer moves to StreamInfo, where decoder state belongs. Worth knowing while reading this: the two resolutions take their pixel format from different places, the stream header in SingleStreamDecoder and the frame itself in ColorConverter, so they can disagree. Passing the resolved value explicitly doesn't fix that, but it does stop a single struct field from implying they are one thing. initialize_color_conversion() and initialize_video() both gain a parameter, which out-of-tree device interfaces will need to follow.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1698
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 17 PendingAs of commit 02a6273 with merge base 2312413 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Contributor
Author
|
Folded into #1696 — the two changes are one story (the output dtype config becoming the only dtype on VideoStreamOptions), so they're now a single commit there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OutputDtype is resolve_output_dtype(config, source pixel format) - derived,
not configured - and VideoStreamOptions held it next to the config it is
derived from. That made the struct half-valid until someone resolved, and
the two users resolved it in ways the shared field couldn't express:
SingleStreamDecoder wrote the answer back into its own options once per
stream, while ColorConverter, which has to resolve per frame because AUTO
"can differ from one frame to the next", fabricated a whole
VideoStreamOptions for the sole purpose of carrying the answer across
initialize_color_conversion().
So make the resolved dtype an argument of initialize_color_conversion(),
the only stage that needs it, and leave VideoStreamOptions with the config
alone - one field, meaningful from the moment it is constructed.
SingleStreamDecoder's once-per-stream answer moves to StreamInfo, where
decoder state belongs.
Worth knowing while reading this: the two resolutions take their pixel
format from different places, the stream header in SingleStreamDecoder and
the frame itself in ColorConverter, so they can disagree. Passing the
resolved value explicitly doesn't fix that, but it does stop a single
struct field from implying they are one thing.
initialize_color_conversion() and initialize_video() both gain a parameter,
which out-of-tree device interfaces will need to follow.
Stack created with GitHub Stacks CLI • Give Feedback 💬