Skip to content

DataUpload/DataDownload progress updates for kubevirt-dm - #252

Open
sseago wants to merge 37 commits into
migtools:oadp-devfrom
sseago:bytes-done
Open

sseago wants to merge 37 commits into
migtools:oadp-devfrom
sseago:bytes-done

Conversation

@sseago

@sseago sseago commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #194

Progress Reporting

The controller reports real-time progress on DataUpload and DataDownload CRs. Because the KubeVirt datamover transfers .qcow2 files rather than scanning filesystems like Velero's built-in Kopia datamover, the progress fields represent slightly different metrics:

DataUpload (Backup)

  • Status.Progress.TotalBytes: The raw capacity of the source PVCs being backed up.
  • Status.IncrementalBytes: The actual size of the .qcow2 payload being transferred to S3. For incremental backups, this will be significantly smaller than TotalBytes, reflecting the efficiency of CBT.
  • Status.Progress.BytesDone: The number of .qcow2 bytes successfully uploaded to S3 so far.

DataDownload (Restore)

  • Status.Progress.TotalBytes: The total size of all .qcow2 files in the checkpoint chain that need to be downloaded from S3.
  • Status.Progress.BytesDone: The number of .qcow2 bytes successfully downloaded so far.
  • (Note: IncrementalBytes is not populated during restore, as the entire chain must be downloaded to reconstruct the flat raw disk).

Quirk: During a restore, after BytesDone reaches TotalBytes (100%), the progress bar will pause while the downloader pod runs qemu-img rebase and qemu-img commit to flatten the chain into a raw image. This reconstruction phase can take time, but the progress will remain at 100% until it finishes.

Example backup output:

oc get du -o wide

NAME                                                          STATUS      STARTED   BYTES DONE   TOTAL BYTES   INCREMENTAL BYTES   STORAGE LOCATION   AGE   NODE
du-backup-test-progress-08-cirros-test-cirros-test-46507dc6   Completed   20h       1075052544   1075052544    1075052544          velero-sample-2    20h   
du-backup-test-progress-09-cirros-test-cirros-test-ef4bd269   Completed   20h                                                      velero-sample-2    20h   
du-backup-test-progress-10-cirros-test-cirros-test-7347dee2   Completed   19h       786440       1073741824    786440              velero-sample-2    19h   

Example restore output:

oc get dd 

NAME                                                                   STATUS      STARTED   BYTES DONE   TOTAL BYTES   STORAGE LOCATION   AGE   NODE
dd-restore-from-full-01-cirros-test-cirros-test-disk-d42643bc          Completed   17h       1075052544   1075052544    velero-sample-2    17h   
dd-restore-from-incremental-01-cirros-test-cirros-test-disk-606e5cca   Completed   17h       1076625424   1076625424    velero-sample-2    17h 

sseago and others added 27 commits October 26, 2026 17:32
…ions

Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>

fix: restore truncated helper functions and add nil-check for progress

Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>

removed extraneous progress nil check
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
…error

Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
@openshift-ci

openshift-ci Bot commented Oct 29, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sseago

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fe6485f0-5f7f-4180-a78f-83805ae30611
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Scott Seago <sseago@redhat.com>
sseago and others added 6 commits October 29, 2026 17:55
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
@sseago

sseago commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator Author

/retest

sseago and others added 2 commits October 30, 2026 14:15
…phase

Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
Co-authored-by: aider (vertex_ai/gemini-3.1-pro-preview) <aider@aider.chat>
@openshift-ci

openshift-ci Bot commented Oct 30, 2026

Copy link
Copy Markdown

@sseago: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/virt-kdm-e2e-test-aws 20bb87c link true /test virt-kdm-e2e-test-aws

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dataupload CR doesn't display bytes done and total bytes

1 participant