Skip to content

feat(qemu-acpi): model guest NUMA nodes without PXB expanders - #1458

Merged
kvinwang merged 1 commit into
nextfrom
feat/numa-without-pxb
Oct 19, 2026
Merged

kvinwang merged 1 commit into
nextfrom
feat/numa-without-pxb

Conversation

@kvinwang

Copy link
Copy Markdown
Collaborator

Summary

Lets VmConfig.numa_nodes describe guest NUMA nodes without pxb-pcie expanders: SRAT plus CPU _PXM, all devices on the root bus. A QEMU wrapper (like the one that already reports extra_root_devices) can add such nodes and record them in .sys-config.json, and the KMS can verify the resulting RTMR0. The multi-PXB layout can't be modeled this way, because each expander's _CRS comes from firmware enumeration.

  • NumaNodeConfig.pxb_bus becomes Option<u8> and is skipped when None.
  • qemu-acpi accepts nodes where every node has an expander or none does. Mixed layouts are still rejected.
  • dstack-mr measure --numa-nodes N.
  • Both differential harnesses cover the new layout. Adds a golden fixture dumped from the pinned reference QEMU.

Compatibility

  • Existing vm_configs parse and serialize byte for byte as before. RTMR0 and verifier cache keys don't change: dstack-mr measure on next and on this branch agrees for 8 legacy layouts (plain, GPUs, hugepages, 1/2/4 PXBs, extra root devices).
  • An older KMS fails to parse a node without pxb_bus and refuses the CVM (fails closed).

Testing

  • differential-random.py against the CI reference image, seed 0x1053, 200 random cases: 243/243 pass, 66 of them pxb-less.
  • differential-qemu.sh with the pinned reference QEMU (dump-all-blobs patch), 10 compat versions × 6 topologies: 60/60 three-blob matches, including the new numa2-plain.
  • Real TDX host: QEMU 8.2.2, dstack-0.6.0 image, 4 vCPU, 4 GiB, two pxb-less nodes plus one extra root endpoint, key provider none. The guest sees both nodes. dstack-mr diagnose reports MRTD, RTMR0-2 and the RTMR0 event log as MATCH. Without numa_nodes, RTMR0 mismatches.

A QEMU wrapper can give a CVM guest NUMA nodes (SRAT and CPU _PXM) while
leaving every device on the root bus. Such a layout depends only on the
vCPU count and memory size, so the KMS can reproduce it, unlike the
multi-PXB layout whose expander _CRS comes from firmware enumeration.

NumaNodeConfig.pxb_bus becomes optional and is omitted when absent, so
existing vm_configs deserialize and serialize unchanged and their MRs and
verifier cache keys stay the same. Mixed layouts are still rejected. An
older KMS fails to parse a node without pxb_bus and refuses the CVM.

dstack-mr measure gains --numa-nodes, and both differential harnesses
cover the new layout.

Signed-off-by: Kevin Wang <wy721@qq.com>
@kvinwang
kvinwang force-pushed the feat/numa-without-pxb branch from d1a7d14 to 4e60f32 Compare October 19, 2026 16:13
@kvinwang
kvinwang merged commit e4add34 into next Oct 19, 2026
13 checks passed
@kvinwang
kvinwang deleted the feat/numa-without-pxb branch October 19, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant