Skip to content

test(orchestrator): redirect IPoIB validation logs to report_path and remove them after each case - #5802

Merged
abhishek-sa1 merged 8 commits into
dell:stagingfrom
mithileshreddy04:staging
Nov 13, 2026
Merged

abhishek-sa1 merged 8 commits into
dell:stagingfrom
mithileshreddy04:staging

Conversation

@mithileshreddy04

@mithileshreddy04 mithileshreddy04 commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

I'll check the actual working-tree state so the file list is accurate.

Only one file differs from HEAD, so the other changes may already be committed. I'll check the recent history.

The history shows a fold commit f7debbe9f, which includes the role file. The role file is now identical to its pre-redirect version, so I'll check the net diff against the base commit to get the real file count.

Net of the base commit 8acb63820, the PR changes two files. I'll write the description for those two only. The artifacts sweep is a third, uncommitted change, so I'm leaving it out.

Description of the Solution

The IPoIB input FVT validates each case against a scratch project named omnia_fvt_ipoib_<hex>. The validator writes its log under <OMNIA_DATA_PATH>/orchestrator/log/<scratch>/, and the teardown removed only the scratch input and output. Every run therefore left a new log/omnia_fvt_ipoib_<hex>/ directory that blocks omnia cleanup --all.

This PR changes only the test code. After each validation, the FVT copies the scratch log into the active project's single validation log, <OMNIA_DATA_PATH>/orchestrator/log/<project>/orchestrator_validation_<project>.log. The first case of a session rewrites that file, and later cases append to it, each under a header naming the scratch project. The teardown removes the scratch input, output, and log directories. The aggregate file is kept, and the next run rewrites it.

Related Issue

  • Fixes #

Changes

IPoIB input FVT (test/orchestrator/library/functions/ipoib_inputs_precheck_func.py)

  • _read_live returns a log_root key, derived from the resolved input directory as <shared>/log.
  • New _append_remote writes base64 chunks, truncating on the first write and appending after.
  • New _fold_scratch_log copies log/<scratch>/orchestrator_validation_<scratch>.log into log/<project>/orchestrator_validation_<project>.log. It creates the project directory with mkdir -p if needed, and returns False when the scratch log is absent.
  • New module flag _AGGREGATE_LOG_STARTED decides whether the next write truncates or appends. It is set only after a log has been written.
  • _run_validate runs the clone and validation as before, folds the scratch log, and removes the scratch input, output, and log directories in the finally block.

IPoIB FVT constants (test/orchestrator/library/vars/ipoib_vars.py)

  • Adds the make_parent_dirs command (mkdir -p -- %s).

Files Changed

File Change Type Description
test/orchestrator/library/functions/ipoib_inputs_precheck_func.py Modified Folds each scratch validation log into the active project's log file, and removes the scratch directories at teardown.
test/orchestrator/library/vars/ipoib_vars.py Modified Adds the make_parent_dirs command.

Testing

  • Not run: Python and Ansible are not available in the development shell, so the change was checked by reading the code only.
  • Required before merge, on a test OIM:
    • Run ./run_validation.sh fvt_orchestrator precheck verify --suite ipoib_inputs.
    • Confirm that log/<project>/orchestrator_validation_<project>.log contains a header and output for each case.
    • Confirm that no omnia_fvt_ipoib_* directory remains under /opt/omnia/orchestrator/log/.
    • Confirm that log/<project>/ is rewritten, not appended to, at the start of a new run.

Backward Compatibility

  • No product code changes. The validator and role are unchanged, so deploy, precheck, and validate behave as before.
  • The aggregate file sits at the active project's standard log path. A later real validation of that project overwrites it, which is expected.
  • If an interrupted run never reaches teardown, its scratch directory stays under orchestrator/log/ and blocks cleanup --all until removed.
  • Existing omnia_fvt_ipoib_* directories from earlier runs are not removed by this PR and need manual cleanup.

mithileshreddy04 and others added 5 commits November 12, 2026 16:20
Telemetry cleanup skipped every removal when kube_vip could not be
resolved, leaving the validator log and generated output files behind.
cleanup --all then aborted in its preflight on
validate_telemetry_input.log under telemetry/log.

Move the log, output, and stray iDRAC report removal outside the
kube_vip gate. These are local files with no cluster dependency.
Keep input removal inside the gate so staged configuration is not
deleted while sink volumes may still remain in the cluster. The input
directory is already allowed by the cleanup preflight.

Revert the preflight exclusion for validate_telemetry_input.log added
in the previous commit, since the log is now removed by cleanup.

Output removal deletes telemetry_status.yml when Delete_sinks_volume is
set. output-contract.md still says this file is preserved and needs an
update to match.
                                                                                                                                    Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>
…ter each case

The IPoIB input FVT validates each case against a scratch project named
omnia_fvt_ipoib_<hex>. The validator wrote its log to
<OMNIA_DATA_PATH>/orchestrator/log/<scratch>/, and the teardown removed only
the scratch input and output directories, so every run left a log directory
behind that blocked omnia cleanup --all.

The validate_orchestrator_input role now takes an optional
orchestrator_validation_log_dir variable for the validator's log directory.
When it is not set, the default path is used, so deploy, precheck, and
validate runs are unchanged.

The FVT passes its scratch log directory under report_path
(<report_path>/ipoib_inputs/logs/<scratch>) through that variable. The
teardown removes that directory with the scratch input and output. The test
does not read the log, so removing it does not change any result.

Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>
@mithileshreddy04 mithileshreddy04 changed the title test(orchestrator): remove IPoIB scratch project logs after input checks test(orchestrator): redirect IPoIB validation logs to report_path and remove them after each case Nov 13, 2026
…e project log

The IPoIB input FVT validates each case against a scratch project named
omnia_fvt_ipoib_<hex>. The validator writes its log under
orchestrator/log/<scratch>/, and teardown removed only the scratch input and
output, so every case left a new log directory behind that blocked
omnia cleanup --all.

After each validation, copy that scratch log into the active project's
single validation log, orchestrator/log/<project>/orchestrator_validation_<project>.log.
The first case of a run rewrites the file and later cases append to it, each
with a header naming its scratch project. Teardown removes the scratch input,
output, and log directories. The aggregate file is kept, and the next run
rewrites it.

No product code changes: the validator and role are unchanged.

Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>
@abhishek-sa1
abhishek-sa1 merged commit 40c86a7 into dell:staging Nov 13, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants