feat: add phys-bits (MAXPHYADDR) as a second live-migration safety axis#3
Open
maxgashkov wants to merge 1 commit into
Open
feat: add phys-bits (MAXPHYADDR) as a second live-migration safety axis#3maxgashkov wants to merge 1 commit into
maxgashkov wants to merge 1 commit into
Conversation
ProxCLMC computed only the lowest common ISA baseline (x86-64-vN). That is necessary but not sufficient for safe live migration: the guest's advertised physical-address width (phys-bits / MAXPHYADDR) must also be <= every host's. The two axes are independent — a cluster standardized on x86-64-v2-AES can still panic guests on live migration when a node is narrower than the guest's default width (e.g. Intel 39-bit alongside AMD 48-bit). An unpinned guest on a named CPU model advertises QEMU's default 40 phys-bits. A VM booted on a node at least that wide can place virtio MMIO BARs above a narrower node's limit; the aperture is fixed at boot and travels unchanged across a live migration, so the next virtqueue kick on the narrow node faults (#PF, reserved bit) and the guest kernel-panics. Offline migration, HA failover and reboots cold-boot on the destination and are unaffected, which makes this look random and easy to misdiagnose. This patch: - parses the "address sizes" line from /proc/cpuinfo into per-node phys-bits - computes the cluster's lowest common phys-bits (ignoring undetected nodes) - pins the reported CPU type to that minimum when a node sits below the guest default (min < min(GUEST_DEFAULT_PHYS_BITS, max)), e.g. Cluster CPU type: x86-64-v3,phys-bits=39 - --list-only carries the same suffix; output stays single-line and unchanged for uniform clusters and clusters all at or above the guest default - unit tests for parsing (real /proc/cpuinfo address-size lines), cluster min/max, and the pin predicate (one case per branch) - documents the second axis in README, deferring the deep mechanism to QEMU's physical-address-space notes - adds a 1.3.0 changelog fragment GUEST_DEFAULT_PHYS_BITS is hardcoded to 40 (QEMU's default for named models, verified on pve-qemu 11.0.0 via qom-get/query-cpu-model-expansion). If QEMU ever raises that default this can under-report until the constant is bumped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add phys-bits (MAXPHYADDR) as a second live-migration safety axis
Problem
ProxCLMC computes the lowest common ISA baseline (
x86-64-vN). That's necessary but not sufficient: the guest's advertised physical-address width (phys-bits / MAXPHYADDR) must also be<=every host's. The two axes are independent — a cluster cleanly standardized onx86-64-v2-AEScan still corrupt guests on live migration.An unpinned guest on a named CPU model advertises QEMU's default 40 phys-bits. On a mixed cluster (e.g. Intel nodes at 39-bit, AMD at 48-bit) a VM booted on a node at least 40 bits wide can place its virtio MMIO BARs above a narrower node's limit. The aperture is fixed at boot and travels unchanged across a live migration; when it lands on the narrow node, the next virtqueue kick faults (
#PF, reserved bit) and the guest kernel-panics. Offline migration, HA failover and reboots cold-boot on the destination and are unaffected.Change
address sizesline from/proc/cpuinfointo a per-node phys-bits.min < min(GUEST_DEFAULT_PHYS_BITS, max)— pin the reported CPU type to the cluster minimum, e.g.Cluster CPU type: x86-64-v3,phys-bits=39.--list-onlycarries the same suffix.GUEST_DEFAULT_PHYS_BITSis hardcoded to 40 (QEMU's default for named models, verified on pve-qemu 11.0.0 viaqom-get/query-cpu-model-expansion). If QEMU ever raises that default this can under-report until the constant is bumped.Testing
cargo test— unit tests for cpuinfo parsing, cluster min/max, and the pin predicate.cargo clippyclean.phys-bits=39:Notes