[johnahull]

Summary

PR682 is the OpenShift hardening layer on top of PR681.

PR681 establishes the GIM/VFIO worker lifecycle. PR682 makes passthrough readiness authoritative on OpenShift by ensuring that host configuration, security permissions, device-plugin rollout, and AMD resource advertisement are all complete before the operator reports readiness.

The AMD device-plugin repository is unchanged. This PR integrates with an externally supplied device-plugin image through the existing DeviceConfig fields.

What PR682 adds

1. OpenShift owns host configuration

• Make the OpenShift Machine Config Operator the owner of GIM host configuration.
• Stop the node labeller from mutating host modprobe configuration.
• Document the required MCO/IOMMU/GIM prerequisites and ownership boundaries.

2. Scoped security permissions

• Add narrowly scoped SCC bindings for the KMM device-plugin, KMM module-loader, and node-labeller service accounts.
• Keep privileged access limited to the operands that need it for GIM/VFIO operation.

3. Resource-gated VF passthrough readiness

For VF passthrough, the operator now waits for all of the following:

• The current device-plugin DaemonSet rollout to be complete.
• The AMD resource to be advertised as allocatable on the selected node.
• The existing GIM/module-loader and VFIO worker conditions from PR681.

This prevents a stale readiness label or an old device-plugin state from making a node appear ready for VM allocation.

4. Reconciliation and watcher hardening

• Requeue when device-plugin DaemonSets are created, deleted, rolling, or change status.
• Requeue when AMD node allocatable resources change.
• Preserve correct readiness behavior for stale and terminating worker pods.
• Cover both RollingUpdate and OnDelete DaemonSet strategies.
• Handle resource-naming variations without declaring readiness from the wrong resource.

5. Documentation and regression coverage

• Clarify GIM, VFIO, IOMMU, KubeVirt, and external device-plugin prerequisites.
• Add regression tests for readiness, DaemonSet watchers, node resource changes, SCC manifests, and lifecycle edge cases.

Scope boundary

Included:

• OpenShift MCO/SCC integration for GIM/VFIO.
• Device-plugin rollout and AMD resource readiness checks.
• Reconciliation and regression coverage for the complete passthrough readiness contract.

Not included:

• Changes to the AMD device-plugin implementation.
• The gpu-virtualization device-plugin branch or its container images.
• KubeVirt VM creation or guest-driver validation.
• A replacement for the external device-plugin resource-allocation implementation.

Validation

The following checks pass on this branch:

• Non-E2E Go tests.
• go vet.
• Operator manager build.
• Helm lint, with the existing chart SemVer/dependency warnings.

Full cluster E2E remains environment and hardware dependent. The currently unavailable suites require E2E_UBUNTU_BASE_IMAGE and GPU_OPERATOR_CHART plus a reachable test API endpoint.

Relationship to other PRs

• PR674 and PR675 provide the prerequisite GIM/VFIO functionality.
• PR681 is the foundational GIM/VFIO operator lifecycle change.
• PR682 is stacked on PR681 and should be rebased after PR681 merges so GitHub shows only this incremental hardening diff.
• PR673 is already merged.
• The AMD device-plugin repository is unchanged.

[johnahull]

OpenShift validation

Functional cluster validation was completed on the equivalent pre-rebase head 39b6b7ab; the current head 68cf85a3 contains the same behavior plus the PR-specific lint cleanup and the PR 681 ancestry update.

• Built and pushed the operator image pr682-39b6b7ab to the OpenShift integrated registry and deployed it with Helm.
• Created a vf-passthrough DeviceConfig using GIM 9.3.0.K, VFIO device ID 74b5, and devicePlugin.enableDevicePlugin=false.
• KMM completed a fresh GIM build into a unique image repository; moduleAvailable=1 and moduleDesired=1.
• On the node, gim loaded and all 8 MI300X PFs created 74b5 VFs bound to vfio-pci.
• The operator set the VFIO-ready label and reported the DeviceConfig Ready condition.
• Teardown passed: deleting the DeviceConfig removed the KMM module, VFs, readiness labels, and GIM module cleanly.
• Unit tests passed under Go 1.27.3; helm lint passed with the existing chart-version warning.
• The PR-specific staticcheck finding was fixed and is no longer reported. Lint now reports only the pre-existing unused useSourceImage field in cmd/prepareimage/main.go.

Device-plugin, DRA, and VM end-to-end tests were intentionally deferred.

[johnahull]

Test environment details

The reported testing used this OpenShift cluster:

• OpenShift Server 4.22.20 / Kubernetes v1.35.9
• One RHCOS 9.8 node, serving as control-plane, master, and worker
• Kernel 5.14.0-687.60.1.el9_8.x86_64
• Dual Intel Xeon Gold 6448Y sockets, 32 cores/socket, 2 threads/core (128 logical CPUs)
• Approximately 2 TiB RAM
• Eight AMD Instinct MI300X GPU PFs (1002:74a1)
• During the GIM/VFIO run, eight MI300X VFs (1002:74b5) were created and bound to vfio-pci, then removed successfully during teardown
• IOMMU was enabled with intel_iommu=on iommu=pt; amdgpu was blacklisted while testing GIM