From b5281a0565f230a7e30ffb68eb23321282f5c97b Mon Sep 17 00:00:00 2001 From: James Sandford Date: Tue, 27 Jan 2026 15:58:03 +0000 Subject: [PATCH 01/16] Add ADR0045 on Flow init segment support --- docs/README.md | 14 +-- docs/adr/0045-flow-init-segments.md | 135 ++++++++++++++++++++++++++++ 2 files changed, 143 insertions(+), 6 deletions(-) create mode 100644 docs/adr/0045-flow-init-segments.md diff --git a/docs/README.md b/docs/README.md index 739a9b5d..f98a1782 100644 --- a/docs/README.md +++ b/docs/README.md @@ -77,12 +77,14 @@ For more information on how we use ADRs, see [here](./adr/README.md). | [0037](./adr/0037-improve-webhooks.md) | Proposal for improvements to the Webhooks endpoints | | [0038](./adr/0038-improved-storage-management.md) | Improved Storage Management | | [0039](./adr/0039-remove-pre-actions.md) | Proposal to remove pre-actions from storage allocation response | -| [0040](./adr/0040-tag-usability-enhancements.md) | Tag Usability Enhancements | -| [0041](./adr/0041-require-explicit-framerate.md) | Requiring explicit frame rates | -| [0042](./adr/0042-uncontrolled-object-instance-labels.md) | Make `label` Mandatory for Uncontrolled Object Instances | -| [0043](./adr/0043-signalling-retention-time.md) | Signalling retention time | -| [0044](./adr/0044-signalling-timeouts.md) | Signalling timeout periods | -| [0046](./adr/0046-governance.md) | Governance | +| [0040](./adr/0040-tag-usability-enhancements.md) | Tag Usability Enhancements | +| [0041](./adr/0041-require-explicit-framerate.md) | Requiring explicit frame rates | +| [0042](./adr/0042-uncontrolled-object-instance-labels.md) | Make `label` Mandatory for Uncontrolled Object Instances | +| [0043](./adr/0043-signalling-retention-time.md) | Signalling retention time | +| [0044](./adr/0044-signalling-timeouts.md) | Signalling timeout periods | +| [0045](./adr/0045-flow-init-segments.md) | Support for init Segments in Flows | +| [0046](./adr/0046-governance.md) | Governance | | [0048](./adr/0048-media-integrity.md) | Integrity model for media in TAMS, and when interacting with other systems | + \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md new file mode 100644 index 00000000..bed71d7d --- /dev/null +++ b/docs/adr/0045-flow-init-segments.md @@ -0,0 +1,135 @@ +--- +status: "proposed" +--- +# Support for init Segments in Flows + +## Context and Problem Statement + +Various media formats employ initialisation segments (aka init segments) for communicating parameters required to configure decoders. +Notably, CMAF-compatible formats such as MPEG DASH and Fragmented MP4 (fMP4). +Init segments are normally communicated via a manifest, alongside media segments. +The decoder will fetch and process the init segment, before identifying the required media segments (which may be mid-stream) and fetching and processing those. + +From a TAMS perspective, init segments are not normally associated with any particular point of time. +A client must be able to write/read, or construct, an init segment independently of the timeline. + +## Decision Drivers + +We would like to: + +* support TAMS features (e.g. edit-by-reference) as fully as possible +* be compatible with as many decoders/players as possible +* place as little burden on implementations as possible with TAMS-specific functionality +* have no/minimal impact on existing (non-init segment based) implementations +* re-use existing patterns/functionality where possible/practical +* allow the use of existing init segment extensions (e.g. C2PA) +* avoid specifying Flow parameters that are format-specific + +## Considered Options + +* {title of option 1} +* {title of option 2} +* {title of option 3} +* … + +## Decision Outcome + +Chosen option: "{title of option 1}", because +{Justification, e.g., only option which resolves requirements, or comes out best (see below)}. + + +### Implementation + +{Once the proposal has been implemented, add a link to the relevant PRs here} + + +## Pros and Cons of the Options + +### Option 1: Generate init segments from essence parameters + +This option would see reading clients (or the store) generate their own init segments based on the existing technical metadata in the Flow metadata. +This is likely not possible with TAMS currently. +The TAMS API likely does not capture all metadata media formats may need to include in init segments. + +* Good, because it requires no changes to the API specification +* Bad, because initial research indicates this will not be possible due to missing metadata +* Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise + +### Option 2: Significantly expand essence parameters to fully capture everything + +This option would see Option 1 extended to include the addition of required parameters to Flow metadata. + +* Bad, because it will likely require significant additions to Flow technical metadata +* Bad, because it will likely require the addition of format specific parameters to Flow technical metadata +* Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise + +### Option 3: Create a Flow Segment with a never timerange for the init segment + +The TAMS TimeRange format has a special "never" value - `()`. +This currently has no meaning for Flow Segments, as media segments always have a duration. +Init segments can be considered to have no duration. +This option would see the never TimeRange used to represent init segments. + +* Good, because it requires little/no spec changes +* Good, because it doesn't require format-specific Flow parameters +* Neutral, because it doesn't allow for different init segments for different parts of the Flow + * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS + * Initial testing indicates common fMP4/DASH players do not support multiple init segments +* Bad, because the approach is somewhat unintuitive +* Bad, because it may result in unexpected behaviour with existing TAMS implementations + +### Option 4: Make the init segment a Flow-level property for which a URL can be generated + +This option would see the init segment managed via the existing Object CRUD mechanisms. +But rather than being registered against a Flow Segment, it would be registered against a new Flow property. +Existing Object patterns would be used for upload, URL generation, re-use, delete, etc. +Like Flow Segments, different Flows could re-use the same init Object. +Implementations might use the init segment's Object ID to determine the compatibility of Flow Segments in edit-by-reference workflows. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Neutral, because it doesn't allow for different init segments for different parts of the Flow + * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS + * Initial testing indicates common fMP4/DASH players do not support multiple init segments +* Netural, because it requires a backwards-compatible addition to the spec + +### Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated + +As with Option 4, this option would see the init segment managed via the existing Object CRUD mechanisms. +Rather than associating the init Object with the Flow, this option would see it associated with each Media Object. +A change in the init segment's Object ID may be used to determine a need to re-provision the init segment in the consuming client. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Neutral, because it allows for different init segments for different parts of the Flow + * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS + * Initial testing indicates common fMP4/DASH players do not support multiple init segments +* Netural, because it requires a backwards-compatible addition to the spec +* Bad, because it will result in a significant increase in Object metadata which makes up a large proportion of data stored in TAMS + +### Option 6: Have a Flow that is the init segment Flow + +This option would see a new init segment Flow type. +An init Flow would be associated with the media Flow. +Init segments would be assigned the TimeRange over which they are valid in the associated media Flow. +This would allow for multiple init segments to be applied to different parts of the media Flow's timeline. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Neutral, because it allows for different init segments for different parts of the Flow + * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS + * Initial testing indicates common fMP4/DASH players do not support multiple init segments +* Netural, because it requires a backwards-compatible addition to the spec +* Neutral, because it may be unintuitive +* Neutral, because it would require open-ended Flow Segments to support live Flows + +### Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) + +As with Options 4, 5, or 6. +But the init segment would be stored as a base 64 encoded blob (or similar) that may be held in the backend database, instead of as file stored in an object store. + +In addition to the base options: + +* Bad, because it would represent a new pattern where one isn't strictly required +* Bad, because it would require a different GET path to media segments +* Bad, because it would likely require additional work on the part of clients to construct a virtual file for the init segment to pass to decoders/players From 5b610a3ed5e8f17f075e97c0a79f624835fd0d7e Mon Sep 17 00:00:00 2001 From: James Sandford Date: Tue, 7 Apr 2026 17:15:27 +0100 Subject: [PATCH 02/16] Updates following discussion at TAMSCon --- docs/adr/0045-flow-init-segments.md | 33 +++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md index bed71d7d..90477557 100644 --- a/docs/adr/0045-flow-init-segments.md +++ b/docs/adr/0045-flow-init-segments.md @@ -22,6 +22,7 @@ We would like to: * place as little burden on implementations as possible with TAMS-specific functionality * have no/minimal impact on existing (non-init segment based) implementations * re-use existing patterns/functionality where possible/practical +* maintain, as far as possible, the "independently decodable Objects" property of TAMS * allow the use of existing init segment extensions (e.g. C2PA) * avoid specifying Flow parameters that are format-specific @@ -54,6 +55,7 @@ The TAMS API likely does not capture all metadata media formats may need to incl * Good, because it requires no changes to the API specification * Bad, because initial research indicates this will not be possible due to missing metadata * Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise +* Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing ### Option 2: Significantly expand essence parameters to fully capture everything @@ -62,6 +64,7 @@ This option would see Option 1 extended to include the addition of required para * Bad, because it will likely require significant additions to Flow technical metadata * Bad, because it will likely require the addition of format specific parameters to Flow technical metadata * Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise +* Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing ### Option 3: Create a Flow Segment with a never timerange for the init segment @@ -75,6 +78,7 @@ This option would see the never TimeRange used to represent init segments. * Neutral, because it doesn't allow for different init segments for different parts of the Flow * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS * Initial testing indicates common fMP4/DASH players do not support multiple init segments +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed * Bad, because the approach is somewhat unintuitive * Bad, because it may result in unexpected behaviour with existing TAMS implementations @@ -92,6 +96,7 @@ Implementations might use the init segment's Object ID to determine the compatib * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS * Initial testing indicates common fMP4/DASH players do not support multiple init segments * Netural, because it requires a backwards-compatible addition to the spec +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed ### Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated @@ -105,6 +110,7 @@ A change in the init segment's Object ID may be used to determine a need to re-p * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS * Initial testing indicates common fMP4/DASH players do not support multiple init segments * Netural, because it requires a backwards-compatible addition to the spec +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed * Bad, because it will result in a significant increase in Object metadata which makes up a large proportion of data stored in TAMS ### Option 6: Have a Flow that is the init segment Flow @@ -122,6 +128,7 @@ This would allow for multiple init segments to be applied to different parts of * Netural, because it requires a backwards-compatible addition to the spec * Neutral, because it may be unintuitive * Neutral, because it would require open-ended Flow Segments to support live Flows +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed ### Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) @@ -133,3 +140,29 @@ In addition to the base options: * Bad, because it would represent a new pattern where one isn't strictly required * Bad, because it would require a different GET path to media segments * Bad, because it would likely require additional work on the part of clients to construct a virtual file for the init segment to pass to decoders/players + +### Option 8: Embed the init segment in every Object + +As with Option 5, but with the init segment embedded in the Object file itself, rather than as a property against the Object. +This would make each Object entirely independently decodable, maintaining this important property of TAMS. +It is unclear how well players/decoders would support this workflow which is more akin to a playlist of multiple fMP4 files, than a manifest. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Good, because it requires no changes to the API specification +* Neutral, because it allows for different init segments for different parts of the Flow + * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS + * It is unclear if such a workflow would be supported by players/decoders as standard +* Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing +* Bad, because it will result in a significant increase in the size of Objects, which makes up a large proportion of data stored in TAMS + +### Option 9: Do not add support for init segments + +Many/all of the above options require significant changes to how the TAMS API is used, or requires formats such as fMP4 to be used in non-standard ways. +This may lead to fragmentation of the TAMS ecosystem, or require fMP4 codecs to be modified in a manner that requires significant in-depth domain knowledge. +We have, however, seen multiple off-spec extensions to the TAMS specification to add init-segment support which itself exacerbates the issue of fragmentation of the TAMS ecosystem. + +* Good, because it encourages TAMS interoperability by maintaining a small number of workflow variations. +* Neutral, because we would explicitly not support a format commonly used in the media industry + * If the way the format would be supported would be non-standard from an fMP4 point of view, TAMS support may bring its own issues - hence "neutral" +* Bad, because this option may persist the existing situation with off-spec extensions \ No newline at end of file From b9d4fa98cedc76c90834356fe2aaea3acdd8164e Mon Sep 17 00:00:00 2001 From: James Sandford Date: Wed, 13 May 2026 14:37:33 +0100 Subject: [PATCH 03/16] Updates following feedback from CMAF specialists --- docs/adr/0045-flow-init-segments.md | 49 ++++++++++++++++------------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md index 90477557..ec8f0e38 100644 --- a/docs/adr/0045-flow-init-segments.md +++ b/docs/adr/0045-flow-init-segments.md @@ -75,10 +75,10 @@ This option would see the never TimeRange used to represent init segments. * Good, because it requires little/no spec changes * Good, because it doesn't require format-specific Flow parameters -* Neutral, because it doesn't allow for different init segments for different parts of the Flow - * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS - * Initial testing indicates common fMP4/DASH players do not support multiple init segments * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it doesn't allow for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters * Bad, because the approach is somewhat unintuitive * Bad, because it may result in unexpected behaviour with existing TAMS implementations @@ -92,11 +92,11 @@ Implementations might use the init segment's Object ID to determine the compatib * Good, because it doesn't require format-specific Flow parameters * Good, because it re-uses existing patterns/mechanisms -* Neutral, because it doesn't allow for different init segments for different parts of the Flow - * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS - * Initial testing indicates common fMP4/DASH players do not support multiple init segments -* Netural, because it requires a backwards-compatible addition to the spec +* Neutral, because it requires a backwards-compatible addition to the spec * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it doesn't allow for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters ### Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated @@ -106,10 +106,10 @@ A change in the init segment's Object ID may be used to determine a need to re-p * Good, because it doesn't require format-specific Flow parameters * Good, because it re-uses existing patterns/mechanisms -* Neutral, because it allows for different init segments for different parts of the Flow - * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS - * Initial testing indicates common fMP4/DASH players do not support multiple init segments -* Netural, because it requires a backwards-compatible addition to the spec +* Good, because it allows for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Neutral, because it requires a backwards-compatible addition to the spec * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed * Bad, because it will result in a significant increase in Object metadata which makes up a large proportion of data stored in TAMS @@ -122,13 +122,14 @@ This would allow for multiple init segments to be applied to different parts of * Good, because it doesn't require format-specific Flow parameters * Good, because it re-uses existing patterns/mechanisms -* Neutral, because it allows for different init segments for different parts of the Flow - * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS - * Initial testing indicates common fMP4/DASH players do not support multiple init segments -* Netural, because it requires a backwards-compatible addition to the spec +* Good, because it allows for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Neutral, because it requires a backwards-compatible addition to the spec * Neutral, because it may be unintuitive * Neutral, because it would require open-ended Flow Segments to support live Flows * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it requires Objects from multiple Flows (init segment Flow, and media segment Flow) to decode a give piece of media ### Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) @@ -145,16 +146,22 @@ In addition to the base options: As with Option 5, but with the init segment embedded in the Object file itself, rather than as a property against the Object. This would make each Object entirely independently decodable, maintaining this important property of TAMS. -It is unclear how well players/decoders would support this workflow which is more akin to a playlist of multiple fMP4 files, than a manifest. +Relevant formats have a concept of "self-initialising" media segments. +But these are, in at least some formats, explicitly not allowed for segmented media such as is used in TAMS. +And are only permitted for playlist-style use cases where a media segment is used to contain a full file. * Good, because it doesn't require format-specific Flow parameters * Good, because it re-uses existing patterns/mechanisms * Good, because it requires no changes to the API specification -* Neutral, because it allows for different init segments for different parts of the Flow - * Note: It is not clear we have a real use case for this as a new init segment likely means a change to the technical parameters, which requires a new Flow in TAMS - * It is unclear if such a workflow would be supported by players/decoders as standard -* Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing -* Bad, because it will result in a significant increase in the size of Objects, which makes up a large proportion of data stored in TAMS +* Good, because it allows for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Bad, because media using extensions such as C2PA metadata may need to be re-encoded/re-packaged on ingest due to the sequence of init and media segments changing +* Bad, because it will result in an increase in the size of Objects, which makes up a large proportion of data stored in TAMS + * Such an increase is likely to be minor for individual Objects, but significant in larger deployments with relatively small Segment sizes + * Note: This point is in comparison to the other Options in the ADR, rather than to the current situation in the TAMS ecosystem where equivalent metadata is already stored in Objects +* Bad, because this approach may result in un-needed overheads or even interrupted playback in some player implementations + * Some relevant implementations of player technologies, such as Media Source Extensions, are known to re-initialise decoders on (superfluous) new init segments which may result in disruption to playback ### Option 9: Do not add support for init segments From 8f6f351de82d6daf28eb78fae2e9d458db4e8a98 Mon Sep 17 00:00:00 2001 From: James Sandford Date: Mon, 18 May 2026 12:07:25 +0100 Subject: [PATCH 04/16] Select option 5 in ADR0045 --- docs/adr/0045-flow-init-segments.md | 41 ++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md index ec8f0e38..96d17cc3 100644 --- a/docs/adr/0045-flow-init-segments.md +++ b/docs/adr/0045-flow-init-segments.md @@ -28,22 +28,43 @@ We would like to: ## Considered Options -* {title of option 1} -* {title of option 2} -* {title of option 3} -* … +* Option 1: Generate init segments from essence parameters +* Option 2: Significantly expand essence parameters to fully capture everything +* Option 3: Create a Flow Segment with a never timerange for the init segment +* Option 4: Make the init segment a Flow-level property for which a URL can be generated +* Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated +* Option 6: Have a Flow that is the init segment Flow +* Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) +* Option 8: Embed the init segment in every Object ## Decision Outcome -Chosen option: "{title of option 1}", because -{Justification, e.g., only option which resolves requirements, or comes out best (see below)}. +Chosen option: Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated. + +Options 1, 2, 3, and 4 would not allow for Flows which have multiple init segments. +This would place additional restrictions on workflows that TAMS interfaces with, or would require transformation to/from a single-init segment rendition where multiple init segment renditions are required elsewhere. +Examples of such workflows are those which use multi-period manifests, and those which use certain profiles of C2PA. +Additionally, edit-by-reference workflows would be unduly restricted by the need for a single init segment per Flow. + +Option 1, 2, 3, and 7 were considered particularly undesirable in terms of how they interacted with the TAMS data model, clashed with existing patterns, or would have a poor developer experience. + +Option 6 would require implementations generic operations (e.g. store transfer) to have specific understanding/behaviour of media types to operate correctly. +It may also break existing implementations of such functionality silently in some cases (e.g. media segments copied without init segments), or result in unexpected behaviour. + +Option 8 would again place restrictions on workflows that TAMS interfaces with, or would require transformation to/from an embedded-segment rendition. +Such an approach was also identified as being expressly prohibited by some media formats. +Real-world issues were also identified regarding processing load, and smoothness of playback with some player/decoder implementations where superfluous init segments are provided. + +Option 5 provides little/no restrictions on broader workflows. +It provides good backwards compatibility. +It provides good feature compatibility with the likes of edit-by-reference. +It provides minimal burden on client implementations. +It re-uses existing patterns, keeping the burden on service implementations as low as possible. - ### Implementation -{Once the proposal has been implemented, add a link to the relevant PRs here} +Implemented by . - ## Pros and Cons of the Options ### Option 1: Generate init segments from essence parameters @@ -172,4 +193,4 @@ We have, however, seen multiple off-spec extensions to the TAMS specification to * Good, because it encourages TAMS interoperability by maintaining a small number of workflow variations. * Neutral, because we would explicitly not support a format commonly used in the media industry * If the way the format would be supported would be non-standard from an fMP4 point of view, TAMS support may bring its own issues - hence "neutral" -* Bad, because this option may persist the existing situation with off-spec extensions \ No newline at end of file +* Bad, because this option may persist the existing situation with off-spec extensions From 6bf4b3c86e31d23eafa74be6ffb984b26b21ae3b Mon Sep 17 00:00:00 2001 From: James Sandford Date: Mon, 18 May 2026 12:09:09 +0100 Subject: [PATCH 05/16] Re-categorize Option 6's potential unintuitiveness as Bad following feedback. --- docs/adr/0045-flow-init-segments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md index 96d17cc3..07760ba3 100644 --- a/docs/adr/0045-flow-init-segments.md +++ b/docs/adr/0045-flow-init-segments.md @@ -147,9 +147,9 @@ This would allow for multiple init segments to be applied to different parts of * Relevant formats support multiple init-segments via features such as "multi-Period" manifests * Some relevant formats may use init segments to signal minor changes to encoding parameters * Neutral, because it requires a backwards-compatible addition to the spec -* Neutral, because it may be unintuitive * Neutral, because it would require open-ended Flow Segments to support live Flows * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it may be unintuitive * Bad, because it requires Objects from multiple Flows (init segment Flow, and media segment Flow) to decode a give piece of media ### Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) From 753f20c12f98417ac5fd5516d5b1d7afa6b89b04 Mon Sep 17 00:00:00 2001 From: James Sandford Date: Mon, 18 May 2026 15:53:16 +0100 Subject: [PATCH 06/16] Add init segment support to specification sem-ver: feature --- api/TimeAddressableMediaStore.yaml | 156 +++++++++++------- .../flow-get-200-video-h264-fmp4.json | 54 ++++++ api/examples/flow-segments-get-200-init.json | 53 ++++++ api/examples/objects-get-200-init.json | 22 +++ api/schemas/flow-audio.json | 4 + api/schemas/flow-core.json | 2 +- api/schemas/flow-data.json | 4 + api/schemas/flow-multi.json | 12 ++ api/schemas/flow-segment-post.json | 6 +- api/schemas/flow-segment.json | 24 ++- api/schemas/flow-storage-post.json | 4 + api/schemas/flow-storage.json | 2 +- api/schemas/flow-video.json | 4 + api/schemas/object-core.json | 4 - api/schemas/object-media-core.json | 18 ++ api/schemas/object.json | 24 ++- 16 files changed, 327 insertions(+), 66 deletions(-) create mode 100644 api/examples/flow-get-200-video-h264-fmp4.json create mode 100644 api/examples/flow-segments-get-200-init.json create mode 100644 api/examples/objects-get-200-init.json create mode 100644 api/schemas/object-media-core.json diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index e5fd5788..6b69c4f1 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -865,6 +865,11 @@ paths: description: Filter on video Flows that have the given frame height. schema: type: integer + - name: init_segments + in: query + description: Filter Flows on the value of `init_segments`. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: @@ -952,6 +957,11 @@ paths: description: Filter on video Flows that have the given frame height. schema: type: integer + - name: init_segments + in: query + description: Filter Flows on the value of `init_segments`. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: @@ -1053,6 +1063,9 @@ paths: mxf_h264: summary: Video Flow - H.264 (MXF container) externalValue: examples/flow-get-200-video-h264-mxf.json + fmp4_h264: + summary: Video Flow - H.264 (fmp4 container) + externalValue: examples/flow-get-200-video-h264-fmp4.json jp2_jpeg2k_singleframe: summary: Video Flow - JPEG-2000 (JP2 container, single frame segments) externalValue: examples/flow-get-200-video-jpeg-jp2.json @@ -1754,39 +1767,39 @@ paths: in: query description: | Include storage metadata in `get_urls` in the response. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Flow Segment `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Flow Segment `get_urls` with no label will only be returned if `accept_get_urls` is omitted. + A comma separated list of labels of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Flow Segment `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. Without `get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Flow Segment `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Flow Segment `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Flow Segment `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/uuid-list.json' - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` in the response. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls` in the response. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: include_object_timerange @@ -1856,6 +1869,9 @@ paths: Note that for codecs with temporal re-ordering, the timerange representes the _presentation_ timeline, and clients may need to check the `key_frame_count` property and/or read backwards from the start of the requested timerange to retrieve enough reference material to start decoding. + Where Flow Segments reference initialisation segment Objects with the same ID, the initialisation segment is the same. + Consuming clients may choose to ignore initialisation segment Objects that haven't changed in subsequent Flow Segments. + When making requests to the provided `get_urls`, clients should include credentials if the provided URL is on the same origin as the API itself, akin to the `same-origin` mode in the [WhatWG Fetch Standard](https://fetch.spec.whatwg.org/#concept-request-credentials-mode). operationId: GET_flows-flowId-segments tags: @@ -1881,39 +1897,39 @@ paths: in: query description: | Include storage metadata in `get_urls` in the response. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Flow Segment `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Flow Segment `get_urls` with no label will only be returned if `accept_get_urls` is omitted. + A comma separated list of labels of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Flow Segment `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. Without `get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Flow Segment `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Flow Segment `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Flow Segment `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/uuid-list.json' - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls`. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: include_object_timerange @@ -1974,6 +1990,12 @@ paths: This example shows how the Media Object for a Flow Segment could be accessed using multiple URLs. Clients are free to choose any of the URLs given (which return identical Media Objects), however the service has expressed a preference for the "pipeline-a" URLs by putting them first in the list. externalValue: examples/flow-segments-get-200-multiple-urls.json + init_segments: + summary: Objects with initialisation segments + description: | + This example shows how Media Objects signal the use of initialisation segments. + Note the re-use of the same initialisation segment in some consecutive Flow Segments. + externalValue: examples/flow-segments-get-200-init.json "400": description: Bad request. Invalid query options. "404": @@ -2014,7 +2036,10 @@ paths: - The timerange of the Segment MUST NOT overlap any other Segment in the same Flow. - The Flow Segment's `timerange` start and end, once offset by `ts_offset`, MUST be contained entirely within the Media Object's `timerange` - When re-using Media Objects, requests which change object properties (e.g. `key_frame_count` or `object_timerange`) SHOULD be rejected. + When re-using Media Objects, requests which change object properties (e.g. `key_frame_count`, `object_timerange`, or `init_object_id`) SHOULD be rejected. + + If an Object has previously been registered as an initialisation segment (i.e. via `init_object_id`), Service implementations SHOULD reject its use as a media segment (i.e. via `object_id`). + If an Object has previously been registered as a media segment (i.e. via `object_id`), Service implementations SHOULD reject its use as an initialisation segment (i.e. via `init_object_id`). operationId: POST_flows-flowId-segments tags: - FlowSegments @@ -2113,6 +2138,12 @@ paths: Objects will likely go unused in cases such as shutdown of ingesting clients, the end of ingested live streams, and unexpected network congestion. Clients SHOULD, however, adapt the number of Objects they request such that they may reasonably expect to use them before the timeout advertised in [`min_object_timeout` at the `/service`](#/operations/GET_service) endpoint, which is subject to a specified minimum (see service endpoint schema). + Where media format make use of initialisation segments, the mime-type of those initialisation segments may differ from that of the media segments. + In such cases, the `container` on the Flow will be set to the mime-type of the media segments and is the assumed default. + A separate request to this endpoint should be made to allocate storage for the initialisation segment(s) with `content-type` set to the mime type of the initialisation segment(s). + `content-type` MUST NOT be set in the request body of this end point at any other time. + Initialisation segment Objects SHOULD be re-used wherever possible. + Service implementations need to handle situations where Objects are not used, and where content is were uploaded but no Flow Segment was registered successfully. In these circumstances, Services should garbage collect Objects after the timeout advertised in [`min_object_timeout` at the `/service`](#/operations/GET_service) endpoint. @@ -2172,42 +2203,42 @@ paths: in: query description: | Include storage metadata in `get_urls`. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Media Object `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Media Object `get_urls` with no label will only be returned if `accept_get_urls` is omitted. - Without `get_urls`, the response from the service could be substantially faster if it is not required to + A comma separated list of labels of Media Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Media Object `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. + Without `get_urls` or `init_object.get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Media Object `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Media Object `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Media Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Media Object `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the `service/storage-backends` endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: string pattern: ^([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})(,[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})*$ - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls`. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: flow_tag.{name} @@ -2269,42 +2300,42 @@ paths: in: query description: | Include storage metadata in `get_urls`. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Media Object `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Media Object `get_urls` with no label will only be returned if `accept_get_urls` is omitted. - Without `get_urls`, the response from the service could be substantially faster if it is not required to + A comma separated list of labels of Media Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Media Object `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. + Without `get_urls` and `init_object.get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Media Object `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Media Object `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Media Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Media Object `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the `service/storage-backends` endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: string pattern: ^([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})(,[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})*$ - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls`. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: flow_tag.{name} @@ -2339,8 +2370,13 @@ paths: type: string content: application/json: - example: - $ref: examples/objects-get-200.json + examples: + basic: + summary: Basic example + externalValue: examples/objects-get-200.json + init_segment: + summary: Object that makes use of an initialisation segment + externalValue: examples/objects-get-200-init.json schema: $ref: "schemas/object.json" "400": @@ -2361,6 +2397,9 @@ paths: The API instances SHOULD be capable of handling the case where the only existant instances are uncontrolled. + API instances SHALL NOT cascade the creation of new Object instances to initialisation Objects. + Clients MUST initiate the creation of new Object instances of initialisation Objects directly. + Where a client has written a new uncontrolled Object instance, the client is responsible for ensuring that the Object written is complete and correct before registering it with this method. All instances of an Object MUST be identical. @@ -2406,6 +2445,9 @@ paths: API instances should remove the Media Object instance from the `get_urls` list and then, if the instance is controlled, delete the Object instance from storage. API instances SHOULD prevent clients from deleting all Object instances. Additionally, API instances MAY prevent clients from deleting all controlled Object instances. Where clients wish to remove all copies of an Object from the store, they should do so by deleting all Flows or Flow Segments which reference the Object. + + API instances SHALL NOT cascade the deletion of Object instances to initialisation Objects. + Clients MUST initiate the deletion of Object instances of initialisation Objects directly. operationId: DELETE_objects-instances tags: - Objects diff --git a/api/examples/flow-get-200-video-h264-fmp4.json b/api/examples/flow-get-200-video-h264-fmp4.json new file mode 100644 index 00000000..fbefb0c9 --- /dev/null +++ b/api/examples/flow-get-200-video-h264-fmp4.json @@ -0,0 +1,54 @@ +{ + "id": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34", + "source_id": "2aa143ac-0ab7-4d75-bc32-5c00c13d186f", + "generation": 0, + "created": "2008-05-27T18:51:00Z", + "metadata_updated": "2023-09-14T09:45:26Z", + "segments_updated": "2023-09-14T09:45:26Z", + "description": "Big Buck Bunny", + "label": "bbb", + "format": "urn:x-nmos:format:video", + "created_by": "tams-dev", + "updated_by": "tams-dev", + "tags": { + "input_quality": "contribution" + }, + "codec": "video/h264", + "container": "video/mp4", + "init_segments": true, + "avg_bit_rate": 2479, + "segment_duration": { + "numerator": 10 + }, + "essence_parameters": { + "frame_rate": { + "numerator": 24, + "denominator": 1 + }, + "frame_width": 1280, + "frame_height": 720, + "bit_depth": 8, + "interlace_mode": "progressive", + "colorspace": "BT709", + "transfer_characteristic": "SDR", + "aspect_ratio": { + "numerator": 16, + "denominator": 9 + }, + "pixel_aspect_ratio": { + "numerator": 1, + "denominator": 1 + }, + "component_type": "YCbCr", + "vert_chroma_subs": 2, + "horiz_chroma_subs": 2, + "avc_parameters": { + "profile": 100, + "level": 31, + "flags": 0 + } + }, + "collected_by": [ + "e85efab4-993b-4ad6-9af3-4cd8d0d38860" + ] +} diff --git a/api/examples/flow-segments-get-200-init.json b/api/examples/flow-segments-get-200-init.json new file mode 100644 index 00000000..7f7163f3 --- /dev/null +++ b/api/examples/flow-segments-get-200-init.json @@ -0,0 +1,53 @@ +[ + { + "object_id": "846023d3-612d-5014-bc47-88f6eb2d04bb", + "timerange": "[0:0_10:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb" + } + ], + "init_object": { + "id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" + } + ] + } + }, + { + "object_id": "25be83fc-11d1-5743-9d47-6865cef5ea35", + "timerange": "[10:0_20:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/25be83fc-11d1-5743-9d47-6865cef5ea35" + } + ], + "init_object": { + "id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" + } + ] + } + }, + { + "object_id": "8b785422-6a82-5d60-b25a-f77e0a748321", + "timerange": "[20:0_30:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/8b785422-6a82-5d60-b25a-f77e0a748321" + } + ], + "init_object": { + "id": "192472f1-55fd-45dc-b355-e6833a3a140c", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/192472f1-55fd-45dc-b355-e6833a3a140c" + } + ] + } + } +] diff --git a/api/examples/objects-get-200-init.json b/api/examples/objects-get-200-init.json new file mode 100644 index 00000000..3c7c4a0b --- /dev/null +++ b/api/examples/objects-get-200-init.json @@ -0,0 +1,22 @@ +{ + "id": "846023d3-612d-5014-bc47-88f6eb2d04bb", + "referenced_by_flows": [ + "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34", + "0fde9c11-da9d-434a-a113-d3b20a2cf251" + ], + "first_referenced_by_flow": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34", + "timerange": "[150:0_200:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb" + } + ], + "init_object": { + "id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" + } + ] + } +} diff --git a/api/schemas/flow-audio.json b/api/schemas/flow-audio.json index b0632d3c..4f6a2a76 100644 --- a/api/schemas/flow-audio.json +++ b/api/schemas/flow-audio.json @@ -78,6 +78,10 @@ ] } } + }, + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" } } } diff --git a/api/schemas/flow-core.json b/api/schemas/flow-core.json index 05328f60..66e3c6da 100644 --- a/api/schemas/flow-core.json +++ b/api/schemas/flow-core.json @@ -68,7 +68,7 @@ "$ref": "mime-type.json" }, "container": { - "description": "The container MIME type for Flow Segments. Note that the `type` component of the container MIME type (i.e. the component before the `/`) may be different to the `type` component of the codec MIME type. e.g. An audio Flow may have `audio/aac` coded content may be wrapped in a `video/mp2t` container. Where multiple types exist for a subtype (e.g. `video/mp4`, `audio/mp4`, `application/mp4`), the closest MIME type to the Flow `format` should be used (e.g. `audio/mp4` for a Flow `format` of `urn:x-nmos:format:audio`). Mime types from the [IANA registry](https://www.iana.org/assignments/media-types/media-types.xhtml) should be preferred. Where multiple MIME types are possible, the most common should be preferred. Where this is insufficient, the maintainers of the TAMS repository may create an application note advising which MIME type to use. Where the Flow does not reference any Media Object(s) directly (e.g. an empty Multi Flow that serves only to collect related mono-essence Flows that do reference Media Objects), this property MUST NOT be set.", + "description": "The container MIME type for Flow Segments. Where the media format employs initialisation segments, this is the mime type of the media segments and NOT the initialisation segment(s). Note that the `type` component of the container MIME type (i.e. the component before the `/`) may be different to the `type` component of the codec MIME type. e.g. An audio Flow may have `audio/aac` coded content may be wrapped in a `video/mp2t` container. Where multiple types exist for a subtype (e.g. `video/mp4`, `audio/mp4`, `application/mp4`), the closest MIME type to the Flow `format` should be used (e.g. `audio/mp4` for a Flow `format` of `urn:x-nmos:format:audio`). Mime types from the [IANA registry](https://www.iana.org/assignments/media-types/media-types.xhtml) should be preferred. Where multiple MIME types are possible, the most common should be preferred. Where this is insufficient, the maintainers of the TAMS repository may create an application note advising which MIME type to use. Where the Flow does not reference any Media Object(s) directly (e.g. an empty Multi Flow that serves only to collect related mono-essence Flows that do reference Media Objects), this property MUST NOT be set.", "$ref": "mime-type.json" }, "avg_bit_rate": { diff --git a/api/schemas/flow-data.json b/api/schemas/flow-data.json index d8d4ab7c..8d2deb4f 100644 --- a/api/schemas/flow-data.json +++ b/api/schemas/flow-data.json @@ -30,6 +30,10 @@ "data_type": { "description": "The type of information encoded in the Flow, identified using a URN. e.g. The data_type may be urn:x-tams:data:bounding-box, and the codec `application/json`.", "type": "string" + }, + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" } } } diff --git a/api/schemas/flow-multi.json b/api/schemas/flow-multi.json index 88e283a7..c82891a8 100644 --- a/api/schemas/flow-multi.json +++ b/api/schemas/flow-multi.json @@ -18,6 +18,18 @@ "enum": [ "urn:x-nmos:format:multi" ] + }, + "essence_parameters": { + "title": "Multi Flow Essence Parameters", + "description": "Describes the parameters of the essence inside this multi Flow", + "type": "object", + "additionalProperties": false, + "properties": { + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" + } + } } } } diff --git a/api/schemas/flow-segment-post.json b/api/schemas/flow-segment-post.json index bb7d26dc..6ce05fa2 100644 --- a/api/schemas/flow-segment-post.json +++ b/api/schemas/flow-segment-post.json @@ -8,7 +8,11 @@ ], "properties": { "object_id": { - "description": "The Object identifier for the Media Object.", + "description": "The Object identifier for the Media Object. The `content-type` of the Media Object MUST match the `container` mime-type of the Flow.", + "type": "string" + }, + "init_object_id": { + "description": "The Object identifier for the initialisation segment Object required to decode the Media Object. The `content-type` of the initialisation segment Object MAY differ from the `container` mime-type of the Flow. This parameter MUST only be set where the media format makes use of initialisation segments. Initialisation Objects SHOULD be re-used where possible. This parameter SHOULD be omitted where the Object `object_id` already exists and is being re-used.", "type": "string" }, "ts_offset": { diff --git a/api/schemas/flow-segment.json b/api/schemas/flow-segment.json index df0244d6..b73592ff 100644 --- a/api/schemas/flow-segment.json +++ b/api/schemas/flow-segment.json @@ -39,11 +39,33 @@ "description": "The count of samples in the Segment (which may be fewer than in the Media Object). The count could be less than expected given the Segment duration and rate if there are gaps. If not set, every sample from sample_offset onwards is used. Note that a sample is a video frame or audio sample. A (coded) audio frame has multiple audio samples. DEPRECATED: Use object_timerange instead - see AppNote 0036. Service implementations SHOULD continue to store and return it if set.", "type": "integer", "deprecated": true + }, + "init_object": { + "title": "Initialisation Object", + "description": "The Object containing the initialisation segment required to decode the parent Media Object.", + "unevaluatedProperties": false, + "allOf": [ + { + "type": "object", + "required": [ + "id" + ], + "properties": { + "id": { + "description": "The identifier of the initialisation Object.", + "type": "string" + } + } + }, + { + "$ref": "object-core.json" + } + ] } } }, { - "$ref": "object-core.json" + "$ref": "object-media-core.json" } ] } diff --git a/api/schemas/flow-storage-post.json b/api/schemas/flow-storage-post.json index df85e17a..6dee695f 100644 --- a/api/schemas/flow-storage-post.json +++ b/api/schemas/flow-storage-post.json @@ -17,6 +17,10 @@ "storage_id": { "description": "The Storage Backend to allocate storage in. A Storage Backend identifier as advertised at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. If not set the default, as advertised at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint, will be used if available. An invalid Storage Backend identifier will result in a 400 error.", "$ref": "uuid.json" + }, + "content-type": { + "description": "The `content-type` to use for the Objects. This parameter MUST only be set where requesting storage for initialisation segments in media formats which require them, and where the mime-type of those initialisation segments differs to that of the media segments. Assumed to be the `container` type of th Flow if not set.", + "$ref": "mime-type.json" } }, "not": { diff --git a/api/schemas/flow-storage.json b/api/schemas/flow-storage.json index 2614b27a..807256a3 100644 --- a/api/schemas/flow-storage.json +++ b/api/schemas/flow-storage.json @@ -1,7 +1,7 @@ { "type": "object", "title": "Media Bucket Object Store", - "description": "Gives information on storage for Media Objects. This schema is for the `http_object_store` Storage Backend type which provides URLs for storing Media Objects in object store buckets, and is the only Storage Backend type currently implemented. URLs SHOULD support the inclusion of checksums in headers as supported by advertised Storage Backend product. See AppNote 0048 for more details.", + "description": "Gives information on storage for Media Objects. This schema is for the `http_object_store` Storage Backend type which provides URLs for storing Media Objects in object store buckets, and is the only Storage Backend type currently implemented. URLs SHOULD support the inclusion of checksums in headers as supported by advertised Storage Backend product. See AppNote 0048 for more details. Where included in the response, `content-type` MUST match the corresponding value in the request.", "properties": { "media_objects": { "type": "array", diff --git a/api/schemas/flow-video.json b/api/schemas/flow-video.json index 4a7aa0ac..a971b789 100644 --- a/api/schemas/flow-video.json +++ b/api/schemas/flow-video.json @@ -212,6 +212,10 @@ "description": "If `true`, the frame rate of the Flow is variable and `frame_rate` MUST NOT be set. If `false` or omitted, the frame rate of the Flow is fixed and `frame_rate` MUST be set.", "type": "boolean", "default": false + }, + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" } }, "if": { diff --git a/api/schemas/object-core.json b/api/schemas/object-core.json index c22779e2..e7fdb373 100644 --- a/api/schemas/object-core.json +++ b/api/schemas/object-core.json @@ -43,10 +43,6 @@ } ] } - }, - "key_frame_count": { - "description": "The number of key frames in the Media Object. This should be set greater than zero when the Media Object contains key frames that serve as a stream access point", - "type": "integer" } } } diff --git a/api/schemas/object-media-core.json b/api/schemas/object-media-core.json new file mode 100644 index 00000000..ab0e7d74 --- /dev/null +++ b/api/schemas/object-media-core.json @@ -0,0 +1,18 @@ +{ + "type": "object", + "description": "Provides the location and metadata of the media files corresponding to a Media Object.", + "title": "Object", + "allOf": [ + { + "properties": { + "key_frame_count": { + "description": "The number of key frames in the Media Object. This should be set greater than zero when the Media Object contains key frames that serve as a stream access point", + "type": "integer" + } + } + }, + { + "$ref": "object-core.json" + } + ] +} diff --git a/api/schemas/object.json b/api/schemas/object.json index 5eba2810..357d0280 100644 --- a/api/schemas/object.json +++ b/api/schemas/object.json @@ -28,11 +28,33 @@ "timerange": { "description": "The timerange covering the sample timestamps embedded in or derived from the Media Object itself, on the Media Object's timeline.", "$ref": "timerange.json" + }, + "init_object": { + "title": "Initialisation Object", + "description": "The Object containing the initialisation segment required to decode the parent Media Object.", + "unevaluatedProperties": false, + "allOf": [ + { + "type": "object", + "required": [ + "id" + ], + "properties": { + "id": { + "description": "The identifier of the initialisation Object.", + "type": "string" + } + } + }, + { + "$ref": "object-core.json" + } + ] } } }, { - "$ref": "object-core.json" + "$ref": "object-media-core.json" } ] } From b3b3ac62c58d6e4e4d1050ed109a07e1c125d2aa Mon Sep 17 00:00:00 2001 From: James Sandford Date: Mon, 18 May 2026 16:54:32 +0100 Subject: [PATCH 07/16] Use `video/iso.segment` in fmp4 Flow example. While `video/mp4` is sometimes used, `video/iso.segment` is strictly speaking correct. --- api/examples/flow-get-200-video-h264-fmp4.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/examples/flow-get-200-video-h264-fmp4.json b/api/examples/flow-get-200-video-h264-fmp4.json index fbefb0c9..e251fd59 100644 --- a/api/examples/flow-get-200-video-h264-fmp4.json +++ b/api/examples/flow-get-200-video-h264-fmp4.json @@ -14,7 +14,7 @@ "input_quality": "contribution" }, "codec": "video/h264", - "container": "video/mp4", + "container": "video/iso.segment", "init_segments": true, "avg_bit_rate": 2479, "segment_duration": { From 02a555560ce357f985da7c010565f78be19a5c02 Mon Sep 17 00:00:00 2001 From: James Sandford Date: Tue, 19 May 2026 14:06:56 +0100 Subject: [PATCH 08/16] Add Application Note 0024 describing init segment usage --- docs/README.md | 1 + docs/appnotes/0024-using-init-segments.md | 56 +++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 docs/appnotes/0024-using-init-segments.md diff --git a/docs/README.md b/docs/README.md index f98a1782..c4ba7fa5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -27,6 +27,7 @@ For more information on how we use application notes, see [here](./appnotes/READ | [0018](./appnotes/0018-managing-multiple-object-instances.md) | Managing Multiple Object Instances | | [0019](./appnotes/0019-implementing-retention-management.md) | Methods of implementing retention management | | [0021](./appnotes/0021-media-integrity.md) | Integrity model for media in TAMS, and when interacting with other systems | +| [0024](./appnotes/0024-using-init-segments.md) | Using Media with Initialisation Segments in TAMS | ## ADRs diff --git a/docs/appnotes/0024-using-init-segments.md b/docs/appnotes/0024-using-init-segments.md new file mode 100644 index 00000000..b485ee71 --- /dev/null +++ b/docs/appnotes/0024-using-init-segments.md @@ -0,0 +1,56 @@ +# 0024: Using Media with Initialisation Segments in TAMS + +This Application Note assumes basic knowledge of how to write/read media to/from a TAMS service. + +Some media formats, such as CMAF compatible MPEG-DASH and fMP4, split content into initialisation (init) segments and media segments. +The init segments contain parameters used to configure decoders common across the stream of media +The media segments generally contain the media itself with additional configuration parameters specific to that media segment. +The split of configuration parameters between init and media segments varies according to the profile used. +When the configuration changes, a new init segment may be required. +While a change in technical parameters would generally require a new Flow in TAMS, some init segment changes may be minor codec parameter changes which would not warrant a new Flow. +A change in init-segment will normally result in a new "period" in DASH, or a "discontinuity" in HLS. +Other formats may have similar concepts. + +Additionally, technologies implemented as extensions to init segments may result in a new init segment for non-technical changes. +One example being C2PA signing of live media. +C2PA uses cryptography to verify the provenance of media. +One of the two C2PA live profiles places cryptographic keys in the init segment which may be used to verify media segments and their sequencing. +When that cryptographic key is rotated, a new init segment will be published. + +TAMS support for init segments makes use of our existing Media Object resource to store and re-use init segments. +These are then associated via their ID to media segment Objects. +This allows existing Media Object re-use mechanisms (i.e. edit-by-reference) to continue to function. +Init segments follow the media. +A client may then detect if the init segment has changed by observing if the init Object ID remains the same or changes between subsequent Flow Segments. + +## General Workflows + +The general workflows described here refer to init and media segments in isolation from manifests used by specific media formats. +The approaches have, however, been designed to map to manifests where required. + +### Write + +1. Register the Flow + * The `init_segments` essence parameter MUST be set to `true` + * The `container` parameter MUST be set to the mime type of the media segments +2. Request storage for the init segment(s) + * The `content-type` MUST be set to the mime type of the init segment(s) + * If the mime type of the init segment(s) matches that of the media segments, this step may be skipped and storage for media and init segments requested together +3. Upload the init segment(s) +4. Request storage for the media segments + * `content-type` MUST NOT be set +5. Upload the media segments +6. Register the media segments as Flow Segments + * `object_id` must be set to the Object ID of the media segment + * `init_object_id` must be set to the Object ID of the init segment + * Init segment objects should be reused wherever possible + +### Read + +1. Get the Flow Segments listing for the Flow +2. For each Flow Segment + 1. If init segment Object ID has changed + 1. Get init segment Object + 2. (Re)initialise decoder with init segment + 2. Get media segment Object + 3. Play media segment From cb8121faff830d212fe997525706b481362a9582 Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 10:20:18 +0100 Subject: [PATCH 09/16] Updates based on review --- api/examples/flow-get-200-video-h264-fmp4.json | 2 +- api/schemas/flow-storage-post.json | 4 ++-- docs/adr/0045-flow-init-segments.md | 8 ++++++++ docs/appnotes/0024-using-init-segments.md | 18 ++++++++++-------- 4 files changed, 21 insertions(+), 11 deletions(-) diff --git a/api/examples/flow-get-200-video-h264-fmp4.json b/api/examples/flow-get-200-video-h264-fmp4.json index e251fd59..1de5e6d1 100644 --- a/api/examples/flow-get-200-video-h264-fmp4.json +++ b/api/examples/flow-get-200-video-h264-fmp4.json @@ -15,12 +15,12 @@ }, "codec": "video/h264", "container": "video/iso.segment", - "init_segments": true, "avg_bit_rate": 2479, "segment_duration": { "numerator": 10 }, "essence_parameters": { + "init_segments": true, "frame_rate": { "numerator": 24, "denominator": 1 diff --git a/api/schemas/flow-storage-post.json b/api/schemas/flow-storage-post.json index 6dee695f..6d4b0752 100644 --- a/api/schemas/flow-storage-post.json +++ b/api/schemas/flow-storage-post.json @@ -18,8 +18,8 @@ "description": "The Storage Backend to allocate storage in. A Storage Backend identifier as advertised at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. If not set the default, as advertised at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint, will be used if available. An invalid Storage Backend identifier will result in a 400 error.", "$ref": "uuid.json" }, - "content-type": { - "description": "The `content-type` to use for the Objects. This parameter MUST only be set where requesting storage for initialisation segments in media formats which require them, and where the mime-type of those initialisation segments differs to that of the media segments. Assumed to be the `container` type of th Flow if not set.", + "content_type": { + "description": "The `content_type` to use for the Objects. This parameter MUST only be set where requesting storage for initialisation segments in media formats which require them, and where the mime-type of those initialisation segments differs to that of the media segments. Assumed to be the `container` type of th Flow if not set.", "$ref": "mime-type.json" } }, diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md index 07760ba3..3a8c97b9 100644 --- a/docs/adr/0045-flow-init-segments.md +++ b/docs/adr/0045-flow-init-segments.md @@ -74,6 +74,7 @@ This is likely not possible with TAMS currently. The TAMS API likely does not capture all metadata media formats may need to include in init segments. * Good, because it requires no changes to the API specification +* Good, because it supports features like edit-by-reference without any modification * Bad, because initial research indicates this will not be possible due to missing metadata * Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise * Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing @@ -82,6 +83,7 @@ The TAMS API likely does not capture all metadata media formats may need to incl This option would see Option 1 extended to include the addition of required parameters to Flow metadata. +* Good, because it supports features like edit-by-reference without any modification * Bad, because it will likely require significant additions to Flow technical metadata * Bad, because it will likely require the addition of format specific parameters to Flow technical metadata * Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise @@ -97,6 +99,7 @@ This option would see the never TimeRange used to represent init segments. * Good, because it requires little/no spec changes * Good, because it doesn't require format-specific Flow parameters * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it features like edit-by-reference would require a different workflow to non-init-segment Flows * Bad, because it doesn't allow for different init segments for different parts of the Flow * Relevant formats support multiple init-segments via features such as "multi-Period" manifests * Some relevant formats may use init segments to signal minor changes to encoding parameters @@ -113,6 +116,8 @@ Implementations might use the init segment's Object ID to determine the compatib * Good, because it doesn't require format-specific Flow parameters * Good, because it re-uses existing patterns/mechanisms +* Good, because it supports features like edit-by-reference without any modification + * Though additional checks for Flow compatibility may be required * Neutral, because it requires a backwards-compatible addition to the spec * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed * Bad, because it doesn't allow for different init segments for different parts of the Flow @@ -130,6 +135,7 @@ A change in the init segment's Object ID may be used to determine a need to re-p * Good, because it allows for different init segments for different parts of the Flow * Relevant formats support multiple init-segments via features such as "multi-Period" manifests * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Good, because it supports features like edit-by-reference without any modification * Neutral, because it requires a backwards-compatible addition to the spec * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed * Bad, because it will result in a significant increase in Object metadata which makes up a large proportion of data stored in TAMS @@ -149,6 +155,7 @@ This would allow for multiple init segments to be applied to different parts of * Neutral, because it requires a backwards-compatible addition to the spec * Neutral, because it would require open-ended Flow Segments to support live Flows * Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it features like edit-by-reference would require a different workflow to non-init-segment Flows * Bad, because it may be unintuitive * Bad, because it requires Objects from multiple Flows (init segment Flow, and media segment Flow) to decode a give piece of media @@ -177,6 +184,7 @@ And are only permitted for playlist-style use cases where a media segment is use * Good, because it allows for different init segments for different parts of the Flow * Relevant formats support multiple init-segments via features such as "multi-Period" manifests * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Good, because it supports features like edit-by-reference without any modification * Bad, because media using extensions such as C2PA metadata may need to be re-encoded/re-packaged on ingest due to the sequence of init and media segments changing * Bad, because it will result in an increase in the size of Objects, which makes up a large proportion of data stored in TAMS * Such an increase is likely to be minor for individual Objects, but significant in larger deployments with relatively small Segment sizes diff --git a/docs/appnotes/0024-using-init-segments.md b/docs/appnotes/0024-using-init-segments.md index b485ee71..7a50dad8 100644 --- a/docs/appnotes/0024-using-init-segments.md +++ b/docs/appnotes/0024-using-init-segments.md @@ -2,7 +2,7 @@ This Application Note assumes basic knowledge of how to write/read media to/from a TAMS service. -Some media formats, such as CMAF compatible MPEG-DASH and fMP4, split content into initialisation (init) segments and media segments. +Some media formats, such as [CMAF](https://www.iso.org/standard/85623.html) compatible MPEG-DASH and fMP4, split content into initialisation (init) segments and media segments. The init segments contain parameters used to configure decoders common across the stream of media The media segments generally contain the media itself with additional configuration parameters specific to that media segment. The split of configuration parameters between init and media segments varies according to the profile used. @@ -12,10 +12,11 @@ A change in init-segment will normally result in a new "period" in DASH, or a "d Other formats may have similar concepts. Additionally, technologies implemented as extensions to init segments may result in a new init segment for non-technical changes. -One example being C2PA signing of live media. +One example being C2PA signing of live media (specifically fragmented MP4, applicable in segmented TAMS use cases). C2PA uses cryptography to verify the provenance of media. One of the two C2PA live profiles places cryptographic keys in the init segment which may be used to verify media segments and their sequencing. When that cryptographic key is rotated, a new init segment will be published. +See [C2PA Section 19](https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html#live-video) for more information on using C2PA with live media. TAMS support for init segments makes use of our existing Media Object resource to store and re-use init segments. These are then associated via their ID to media segment Objects. @@ -30,27 +31,28 @@ The approaches have, however, been designed to map to manifests where required. ### Write -1. Register the Flow +1. [Register the Flow](https://bbc.github.io/tams/main/index.html#/operations/PUT_flows-flowId) * The `init_segments` essence parameter MUST be set to `true` * The `container` parameter MUST be set to the mime type of the media segments -2. Request storage for the init segment(s) +2. [Request storage for the init segment(s)](https://bbc.github.io/tams/main/index.html#/operations/POST_flows-flowId-storage) * The `content-type` MUST be set to the mime type of the init segment(s) * If the mime type of the init segment(s) matches that of the media segments, this step may be skipped and storage for media and init segments requested together 3. Upload the init segment(s) -4. Request storage for the media segments +4. [Request storage for the media segments](https://bbc.github.io/tams/main/index.html#/operations/POST_flows-flowId-storage) * `content-type` MUST NOT be set 5. Upload the media segments -6. Register the media segments as Flow Segments +6. [Register the media segments as Flow Segments](https://bbc.github.io/tams/main/index.html#/operations/POST_flows-flowId-segments) * `object_id` must be set to the Object ID of the media segment * `init_object_id` must be set to the Object ID of the init segment * Init segment objects should be reused wherever possible ### Read -1. Get the Flow Segments listing for the Flow +1. [Get the Flow Segments listing for the Flow](https://bbc.github.io/tams/main/index.html#/operations/GET_flows-flowId-segments) 2. For each Flow Segment 1. If init segment Object ID has changed 1. Get init segment Object 2. (Re)initialise decoder with init segment 2. Get media segment Object - 3. Play media segment + 3. Perform [media timeline mapping](https://github.com/bbc/tams/#flow-and-media-timelines) as usual in TAMS + 4. Play media segment From ced27ab2006f40f6c47db7d336c8c7d97c63ec2c Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 10:46:33 +0100 Subject: [PATCH 10/16] Add notes on terminology to AppNote0024 --- docs/appnotes/0024-using-init-segments.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/appnotes/0024-using-init-segments.md b/docs/appnotes/0024-using-init-segments.md index 7a50dad8..a196ad0d 100644 --- a/docs/appnotes/0024-using-init-segments.md +++ b/docs/appnotes/0024-using-init-segments.md @@ -1,6 +1,20 @@ # 0024: Using Media with Initialisation Segments in TAMS -This Application Note assumes basic knowledge of how to write/read media to/from a TAMS service. +> [!NOTE] +> There is an unfortunate overlap in terminology between TAMS and CMAF/DASH/HLS/fMP4. +> As such, where Segments or Flow Segments are referred to capitalised this document is referring to the TAMS resource types. +> Where init segments or media segments are referred to in lower case, this document is referring to the CMAF resource types. +> Where this document refers to Media Objects or Objects, it is referring to the TAMS Media Object resource type. +> Objects (without Media prepended) has been preferred when referring Objects containing initialisation segments to avoid potential confusion "init Media Object" may cause. +> There is no implied distinction between Media Object or Objects in the TAMS API. + +>[!NOTE] +> This Application Note refers to CMAF, MPEG-DASH, HLS, and fMP4. +> It also uses terms of art from those formats. +> It should, however, be read as applying to the general case of any format that uses equivalent concepts of initialisation segments. + +> [!IMPORTANT] +> This Application Note assumes basic knowledge of how to write/read media to/from a TAMS service. Some media formats, such as [CMAF](https://www.iso.org/standard/85623.html) compatible MPEG-DASH and fMP4, split content into initialisation (init) segments and media segments. The init segments contain parameters used to configure decoders common across the stream of media From bc48f7b8baf90cd2ae4f30d5a6bc4ec2f41cf6e2 Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 10:58:24 +0100 Subject: [PATCH 11/16] Fix linting --- docs/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index c4ba7fa5..d1f34ef9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -87,5 +87,4 @@ For more information on how we use ADRs, see [here](./adr/README.md). | [0046](./adr/0046-governance.md) | Governance | | [0048](./adr/0048-media-integrity.md) | Integrity model for media in TAMS, and when interacting with other systems | - \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication From c092bcaeedee93011cfbf9d54a69dc03fee4776c Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 11:08:35 +0100 Subject: [PATCH 12/16] Add note regarding reference counting when deleting Objects --- api/TimeAddressableMediaStore.yaml | 8 ++++++-- api/schemas/flow-storage-post.json | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index 6b69c4f1..c2f2b8e9 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -1156,7 +1156,9 @@ paths: summary: Delete Flow description: | Deletes the Flow and associated Segments. - If Flow Segment deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress + If Flow Segment deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress. + + Services SHALL only delete Media Objects when they are no longer referenced by any Flow Segments or other Media Objects (i.e. as init Objects). operationId: DELETE_flows-flowId tags: - Flows @@ -2075,7 +2077,9 @@ paths: delete: summary: Delete Flow Segment description: | - Deletes the Flow Segments. If the deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress + Deletes the Flow Segments. If the deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress. + + Services SHALL only delete Media Objects when they are no longer referenced by any Flow Segments or other Media Objects (i.e. as init Objects). operationId: DELETE_flows-flowId-segments tags: - FlowSegments diff --git a/api/schemas/flow-storage-post.json b/api/schemas/flow-storage-post.json index 6d4b0752..b51b4f75 100644 --- a/api/schemas/flow-storage-post.json +++ b/api/schemas/flow-storage-post.json @@ -19,7 +19,7 @@ "$ref": "uuid.json" }, "content_type": { - "description": "The `content_type` to use for the Objects. This parameter MUST only be set where requesting storage for initialisation segments in media formats which require them, and where the mime-type of those initialisation segments differs to that of the media segments. Assumed to be the `container` type of th Flow if not set.", + "description": "The `content_type` to use for the Objects. This parameter MUST only be set where requesting storage for initialisation segments in media formats which require them, and where the mime-type of those initialisation segments differs to that of the media segments. Assumed to be the `container` type of the Flow if not set.", "$ref": "mime-type.json" } }, From 77e1e7f850133fb21aa47ed02cc6f241e98deabc Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 11:16:01 +0100 Subject: [PATCH 13/16] Add note on rejecting miss-use of init segments and media segments and vice-versa --- api/schemas/flow-segment-post.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/api/schemas/flow-segment-post.json b/api/schemas/flow-segment-post.json index 6ce05fa2..5f5e60fc 100644 --- a/api/schemas/flow-segment-post.json +++ b/api/schemas/flow-segment-post.json @@ -8,11 +8,11 @@ ], "properties": { "object_id": { - "description": "The Object identifier for the Media Object. The `content-type` of the Media Object MUST match the `container` mime-type of the Flow.", + "description": "The Object identifier for the Media Object. The `content-type` of the Media Object MUST match the `container` mime-type of the Flow. Service implementations SHOULD reject Objects IDs which have previously been registered as an `init_object_id` on other Flow Segments. i.e. init segments may not be used as media segments.", "type": "string" }, "init_object_id": { - "description": "The Object identifier for the initialisation segment Object required to decode the Media Object. The `content-type` of the initialisation segment Object MAY differ from the `container` mime-type of the Flow. This parameter MUST only be set where the media format makes use of initialisation segments. Initialisation Objects SHOULD be re-used where possible. This parameter SHOULD be omitted where the Object `object_id` already exists and is being re-used.", + "description": "The Object identifier for the initialisation segment Object required to decode the Media Object. The `content-type` of the initialisation segment Object MAY differ from the `container` mime-type of the Flow. This parameter MUST only be set where the media format makes use of initialisation segments. Initialisation Objects SHOULD be re-used where possible. This parameter SHOULD be omitted where the Object `object_id` already exists and is being re-used. Service implementations SHOULD reject Objects IDs which have previously been registered as an `id` on other Flow Segments. i.e. media segments may not be used as init segments.", "type": "string" }, "ts_offset": { From 92b3566378551f5a619cf07f997034ad838aaceb Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 11:23:46 +0100 Subject: [PATCH 14/16] Use `object_id` for init object on flow segments endpoint --- api/examples/flow-segments-get-200-init.json | 6 +++--- api/schemas/flow-segment.json | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/api/examples/flow-segments-get-200-init.json b/api/examples/flow-segments-get-200-init.json index 7f7163f3..eb95f4ef 100644 --- a/api/examples/flow-segments-get-200-init.json +++ b/api/examples/flow-segments-get-200-init.json @@ -8,7 +8,7 @@ } ], "init_object": { - "id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "object_id": "9ef06e23-b882-4a93-b005-480de42ef4e9", "get_urls": [ { "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" @@ -25,7 +25,7 @@ } ], "init_object": { - "id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "object_id": "9ef06e23-b882-4a93-b005-480de42ef4e9", "get_urls": [ { "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" @@ -42,7 +42,7 @@ } ], "init_object": { - "id": "192472f1-55fd-45dc-b355-e6833a3a140c", + "object_id": "192472f1-55fd-45dc-b355-e6833a3a140c", "get_urls": [ { "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/192472f1-55fd-45dc-b355-e6833a3a140c" diff --git a/api/schemas/flow-segment.json b/api/schemas/flow-segment.json index b73592ff..7226f4e9 100644 --- a/api/schemas/flow-segment.json +++ b/api/schemas/flow-segment.json @@ -48,10 +48,10 @@ { "type": "object", "required": [ - "id" + "object_id" ], "properties": { - "id": { + "object_id": { "description": "The identifier of the initialisation Object.", "type": "string" } From 89e88a3744951ea0942494b4034f09d595e76f8e Mon Sep 17 00:00:00 2001 From: James Sandford Date: Thu, 21 May 2026 11:32:11 +0100 Subject: [PATCH 15/16] Add note to AppNote0024 on minimizing creation of new init segments. --- docs/appnotes/0024-using-init-segments.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/appnotes/0024-using-init-segments.md b/docs/appnotes/0024-using-init-segments.md index a196ad0d..83b2b1c7 100644 --- a/docs/appnotes/0024-using-init-segments.md +++ b/docs/appnotes/0024-using-init-segments.md @@ -38,6 +38,10 @@ This allows existing Media Object re-use mechanisms (i.e. edit-by-reference) to Init segments follow the media. A client may then detect if the init segment has changed by observing if the init Object ID remains the same or changes between subsequent Flow Segments. +> [!IMPORTANT] +> Encoder implementations/configurations should minimise the creation of new init segments as much possible. +> Over-use of init segments may result in un-needed re-initialisation of decoders which may adversely affect performance and/or playback. + ## General Workflows The general workflows described here refer to init and media segments in isolation from manifests used by specific media formats. From c09b54b73e202078a35c013a5c1828debec4a545 Mon Sep 17 00:00:00 2001 From: James Sandford Date: Fri, 12 Jun 2026 15:09:34 +0100 Subject: [PATCH 16/16] Specify that Object timerange MUST NOT be set for init segment objects --- api/schemas/object.json | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/api/schemas/object.json b/api/schemas/object.json index 357d0280..252684bd 100644 --- a/api/schemas/object.json +++ b/api/schemas/object.json @@ -6,8 +6,7 @@ "type": "object", "required": [ "id", - "referenced_by_flows", - "timerange" + "referenced_by_flows" ], "properties": { "id": { @@ -26,7 +25,7 @@ "$ref": "uuid.json" }, "timerange": { - "description": "The timerange covering the sample timestamps embedded in or derived from the Media Object itself, on the Media Object's timeline.", + "description": "The timerange covering the sample timestamps embedded in or derived from the Media Object itself, on the Media Object's timeline. This parameter MUST be set where the Object contains media. It MUST NOT be set where the Object contains an init segment.", "$ref": "timerange.json" }, "init_object": {