Fix KVM incremental volume snapshot migration between secondary storages

[hsato03]

Description

ACS allows data to be migrated from one secondary storage to another; however, this process disregards incremental volume snapshots created using the KVM hypervisor.

Since then, the migration process has been extended to also allow the migration of these resources.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

N°	Test	API	Result	Expected result (Y/N)
1	Migrate a single incremental volume snapshot	migrateResourceToAnotherSecondaryStorage	The snapshot, along with its checkpoint file, was migrated to the destination storage, and its child snapshot, which remained in the old storage, was rebased onto the snapshot in the new storage	Y
2	Migrate the entire chain of incremental volume snapshots to another secondary storage	migrateSecondaryStorageData	All snapshots were migrated and the chain was correctly updated with the files from the new storage	Y
3	Migrate a chain of incremental volume snapshots split between 2 secondary storages	migrateSecondaryStorageData	The chain was updated based on existing snapshots in the destination storage and the new file from the migrated snapshots	Y

How did you try to break this feature and the system with this change?

[codecov]

Codecov Report

❌ Patch coverage is 0.70922% with 280 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.92%. Comparing base (30e6c22) to head (7d71730).

Files with missing lines	Patch %	Lines
...apshotsBetweenSecondaryStoragesCommandWrapper.java	1.01%	98 Missing ⚠️
...tack/engine/orchestration/StorageOrchestrator.java	0.00%	83 Missing ⚠️
...ack/engine/orchestration/DataMigrationUtility.java	0.00%	33 Missing ⚠️
...grateSnapshotsBetweenSecondaryStoragesCommand.java	0.00%	24 Missing ⚠️
...he/cloudstack/storage/snapshot/SnapshotObject.java	0.00%	17 Missing ⚠️
...java/org/apache/cloudstack/utils/qemu/QemuImg.java	0.00%	12 Missing ⚠️
.../MigrateBetweenSecondaryStoragesCommandAnswer.java	0.00%	9 Missing ⚠️
...tack/storage/snapshot/SnapshotDataFactoryImpl.java	0.00%	3 Missing ⚠️
.../hypervisor/kvm/storage/LibvirtStorageAdaptor.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12086      +/-   ##
============================================
- Coverage     17.92%   17.92%   -0.01%     
- Complexity    16154    16158       +4     
============================================
  Files          5939     5942       +3     
  Lines        533181   533435     +254     
  Branches      65237    65257      +20     
============================================
+ Hits          95585    95597      +12     
- Misses       426856   427098     +242     
  Partials      10740    10740              

Flag	Coverage Δ
uitests	3.67% <ø> (ø)
unittests	19.03% <0.70%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hsato03]

@blueorangutan package

[Copilot]

Pull request overview

This PR fixes a bug where KVM incremental volume snapshots were not being migrated when data was transferred between secondary storages. The implementation extends the existing migration process to handle KVM-specific incremental snapshots and their checkpoint files, ensuring the entire snapshot chain is correctly migrated and rebased.

Key changes:

• Added support for migrating KVM incremental snapshot chains with their checkpoint files
• Implemented single-threaded execution per zone for KVM incremental snapshot migrations to maintain chain integrity
• Added new command classes and wrapper for handling snapshot migration between secondary storages

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
StorageManagerImpl.java	Added new configuration key for agent migration timeout
QemuImg.java	Enhanced convert method to support backing file specification during conversion
LibvirtStorageAdaptor.java	Updated convert call to include new backing file parameter
LibvirtMigrateSnapshotsBetweenSecondaryStoragesCommandWrapper.java	New handler for executing snapshot migration commands on KVM agents
SnapshotObject.java	Added methods to retrieve parent and child snapshot chains
SnapshotDataFactoryImpl.java	Sets KVM incremental snapshot flag based on checkpoint path presence
StorageOrchestrator.java	Core migration orchestration logic for handling KVM incremental snapshots
DataMigrationUtility.java	Updated to identify and build KVM incremental snapshot chains for migration
StorageManager.java	Defined new configuration key for migration timeout
SnapshotInfo.java	Added interface methods for retrieving snapshot chain relationships
MigrateSnapshotsBetweenSecondaryStoragesCommand.java	New command class for snapshot migration requests
MigrateBetweenSecondaryStoragesCommandAnswer.java	New answer class for migration command responses

Comments suppressed due to low confidence (1)

plugins/hypervisors/kvm/src/main/java/org/apache/cloudstack/utils/qemu/QemuImg.java:1

• [nitpick] Extra whitespace before 'new QemuImageOptions' parameter. Should have consistent single-space separation between parameters.

// Licensed to the Apache Software Foundation (ASF) under one

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[hsato03]

@blueorangutan package

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15849

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian Build Failed (tid-14880)

[blueorangutan]

[SF] Trillian test result (tid-14882)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 53722 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12086-t14882-kvm-ol8.zip
Smoke tests completed. 149 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_list_cpvm_vm	Failure	0.04	test_ssvm.py
test_04_cpvm_internals	Failure	0.05	test_ssvm.py

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[hsato03]

@blueorangutan package

[blueorangutan]

@hsato03 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16432

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[hsato03]

@blueorangutan package

[blueorangutan]

@hsato03 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16929

[DaanHoogland]

@hsato03 , Do you want thios in 23 or an earlier release? (seems like 22.1 would make sense to me)

[hsato03]

@hsato03 , Do you want thios in 23 or an earlier release? (seems like 22.1 would make sense to me)

@DaanHoogland I agree. Since it is a bug fix, I think it could be included in 4.22.1. I will rebase and change the target branch.

[hsato03]

@blueorangutan package

[blueorangutan]

@hsato03 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.