Add temporal semantics

[frlai]

Summary

Adds graph-level and temporal semantic metadata support to LEAPP YAML export.

Changes

• Added GraphConfigs for graph-level metadata passed through leapp.compile_graph(...).
• Emits graph-level config fields under pipeline.configs, e.g. frequency and flattened extra values.
• Added TemporalPeriodMs for marking a tensor axis as temporal through TensorSemantics.element_names.
• Serializes temporal axes with the reserved __temporal_axis__ sentinel and temporal_period_ms.
• Exported GraphConfigs and TemporalPeriodMs from the public leapp package.
• Updated semantic metadata docs and API docs with graph-level semantics and temporal-axis examples.
• Added functional coverage for GraphConfigs and TemporalPeriodMs YAML output.

Example

leapp.compile_graph(
    graph_configs=GraphConfigs(
        frequency=50,
        extra={"runtime": "isaac_lab"},
    ),
)

TensorSemantics(
    name="actions",
    ref=actions,
    element_names=[TemporalPeriodMs(100), ["hip", "knee", "ankle"]],
)

Final yaml:

...

outputs:
- name: actions
  element_names:
  - [__temporal_axis__, [hip, knee, ankle]]
  temporal_period_ms: 100
  
pipeline:
  configs:
    frequency: 50
    runtime: isaac_lab
 ...

[lgulich]

Looks good overall, just some small comments

[lgulich]

So here this would mean that the first axis is time, the second axis is joints?

When I initially read this line (without reading the documentaiton above) I found it a bit unintuitive that TemporalPeriodMs is part of element_names.

But now after reading the comment above it makes sense. Any idea how we could make this more clear at a glance?

[frlai]

what is your opinion on
TensorSemantics( name="actions", ref=actions, element_names=[TemporalAxis(period_ms=100), ["hip", "knee", "ankle"]],
or is the main issue we have a non-string defined in a field that is called element_names

[lgulich]

yeah I think mainly "element_names" is confusing, since both TemporalPeriodMs / TemporalAxis are both not "names".

Something like the following would be the most natural to me, but this is not back compatible :/

  axis_kind=[TemporalAxis(period_ms=100), IndexAxis(names=['hip','knee','ankle'])]

[frlai]

what if we did this:

TensorSemantics(
    name="actions",
    ref=actions,
    dims=[
        TemporalDim(period_ms=100),
        NamedDim(names=["hip", "knee", "ankle"]),
    ],
)

this will be the new recommended api. however we keep the original element_names as a backward compatible wrapper that is the same as dims.
we also assume any list passed in is a NamedDim and normalize it as such. so then the original
element_names= [['a', 'b', 'c'], ['1', '2', '3']]
and
dims = [NamedDim(names=['a', 'b', 'c']), NamedDim(names=['1', '2', '3']]
are equivalent and we can even keep the raw list passing as a short hand.

in the future this also gives us space to add batch dims or make it so some dimensions are variable in size etc. but this will mean the final yaml specs need to change.
element_names field should be something like dim semantics. might organize it like this

dim_semantics: [{type: temporal_dim, period_ms: 100}, {type: batch_dim}, {type: None}, {type: named_dim, names=["hip", "knee", "ankle"]}]

but I am not sure if it is worth it to do this right now or at all. especially because this will for sure require a change on the deployment framework side.

my vote is for now we can just go with the element names idea and in the future if the need for more dimensions arises we can consider updating.

[lgulich]

do we need the temporal_period_ms here too? if it's set here which axis would this apply to?

I wonder if it's enought to just have the TemporalPeriodMs for element_names?

[frlai]

you're right. this isn't intended. I think we can get rid of this.