Skip to content

Store test case time as a number, not a string #1954

Description

@ubmarco

Today

time is a string everywhere a need can see it:

  • the field is declared a string in sphinxcontrib/test_reports/fields.py
    ("time": ("string", "Test execution time, in seconds")), the table both the
    extension's field registration and the converter's needs_schema read;
  • the test-case directive parses the XML attribute to a float and writes
    str(...) back — sphinxcontrib/test_reports/directives/test_case.py:139-153,
    with the comment "SN 6.0.0 requires to be in one specific type and it is set
    to string for backwards compatibility"
    ;
  • the converter mirrors the directive —
    sphinxcontrib/test_reports/needs_export.py, optional(case.get("time"), -1).

The parser already holds a float: junitparser.py:165,
float(testcase.attrib.get("time", -1)). The value is stringified only on its
way into a need.

So a needs.json says "time": "0.001", and nothing downstream can compare,
sum or average it without parsing it back. A needtable sorted by time sorts
lexicographically — "0.01" before "0.002".

Wanted

time declared as {"type": ["number", "null"]} and written as a JSON number:
"time": 0.001.

What has to change together

The declaration and both writers are one change and cannot move separately:
sphinx-needs type-checks a field's value against its declared type when the
need is created, and coerces only from a string to the declared type, never the
other way (needs_schema.py, convert_or_type_check). A number written against
a string declaration is an import-time error.

  1. fields.py: "time": ("number", ...) — flows into both the extension's
    registration and the converter's needs_schema.
  2. directives/test_case.py: pass the float, drop time_str.
  3. needs_export.py: write the number instead of optional(...).
  4. directives/test_suite.py and directives/test_file.py, which carry time
    on their needs too.

Migration notes

  • An existing needs.json carrying "0.001" keeps importing: sphinx-needs
    coerces the string to the declared type.
  • Filters and schemas in the field break — time == "0.001" and a
    {"type": "string"} entry in a needs_schema_definitions_from_json have to
    become numeric. Breaking change, worth its own changelog entry.
  • Absent time needs one answer for both writers, which differ today: the
    parser reports -1, the directive writes "0.0", the converter writes "".
    With a nullable number, null is the honest value for "the report did not
    say", where 0.0 claims a measurement that never happened.
  • case_line has the same shape of problem — declared a string, an integer in
    the XML. Same fix, separate decision.

Noticed while adding the needs_schema block to the converter's output
(#148), which is what makes the declared types visible in the artifact.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementpkg: sphinx-test-reportsConcerns the sphinx-test-reports package (packages/sphinx-test-reports)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions