Add the vendor VFIO vGPU device backend - #364
Conversation
2bb8e86 to
7fc3b49
Compare
7fc3b49 to
f661e63
Compare
f661e63 to
d1207d0
Compare
d1207d0 to
5b47670
Compare
5b47670 to
ffbf8a0
Compare
ffbf8a0 to
78801e7
Compare
78801e7 to
b4651f3
Compare
b4651f3 to
d2bf3fa
Compare
d2bf3fa to
495ab62
Compare
|
added 7f5233f: report |
7f5233f to
0baacb3
Compare
-->
✱ stlc build✅ go code · compare
✅ typescript code · compare
Diagnostics: 💡 0 new / 5 total note
Build metadata
This comment is auto-generated by stlc and is kept up to date as you push. |
eb00c67 to
0fb3c6a
Compare
0fb3c6a to
5beb0fb
Compare
5beb0fb to
ea22b71
Compare
ea22b71 to
49cb877
Compare
Linux 6.8 hosts with NVIDIA R580 drop the mdev interface: vGPUs are assigned by writing a type ID to a VF's nvidia/current_vgpu_type and passed to QEMU as a plain VFIO PCI device. Add a vendor VFIO backend behind the existing framework dispatch: profile discovery from the capacity-dependent creatable catalogs, least-loaded VF placement, create/verify/rollback, and release. Because the same VF path is reused across assignments (unlike mdev UUIDs), release is guarded: an in-process owner map covers the window before QEMU opens the device, and an open-VFIO-handle scan refuses to clear a VF a running VM still holds. Reconciliation clears orphaned assignments on startup, skipping VFs protected by the caller and failing closed when the protected set is unavailable. Branch the vGPU integration test by discovered framework and extend it to cover release on stop and reacquisition on start.
Sort GPUs with unaccountable load last instead of rejecting placement, and stop reporting passthrough capacity when vGPU discovery fails.
…nership through releases
The instance lifecycle already routes create/start/stop/delete through CreateVGPU/DestroyVGPU, so dispatching vendor VFIO creates here would activate the backend before assignment durability and release guards exist. Reject vendor VFIO creates for now; destroy stays wired so existing assignments remain releasable. The integration test skips on vendor VFIO hosts at this layer and no longer asserts the transitional stop-retention behavior.
Counting every free VF advertising a type overreports concurrent capacity: sibling VFs share their parent GPU's framebuffer, so one 48Q assignment revokes the type from every other VF on that GPU. Bound each GPU's contribution by both its free VFs and how many times the profile framebuffer fits into the GPU's remaining framebuffer, using the largest still-creatable profile as a lower bound on what remains.
This reverts commit d6adc4c.
49cb877 to
1438015
Compare
…-backend # Conflicts: # lib/oapi/oapi.go
Summary
Layer 2 of the vendor VFIO vGPU stack (
generalize-vgpu-device←hypervisor-liveness← this ←vendor-vfio-vgpu). Self-contained inlib/devices+lib/resources; nothing in the instance lifecycle calls it yet (that's the top layer).Linux 6.8 hosts with NVIDIA R580 drop the mdev interface: vGPUs are assigned by writing a type ID to a VF's
nvidia/current_vgpu_typeand passed to QEMU as a plain VFIO PCI device. This adds that backend behind the framework dispatch introduced in #322:creatable_vgpu_typescatalogs, least-loaded-GPU VF selection, create/verify/rollback. Profile availability counts free VFs currently advertising each type as a best-effort snapshot; creating one assignment may change sibling catalogs.Testing
go build ./...,go vetcleango test -race ./lib/devices/ ./lib/resources/passNote
Medium Risk
Touches host GPU sysfs assignment and release with VFIO in-use checks, but vendor VFIO instance creation remains disabled until a follow-up integration layer lands.
Overview
Adds NVIDIA vendor VFIO vGPU support for newer hosts (assign via
current_vgpu_type/creatable_vgpu_types) alongside the existing mdev path, unified behindDiscoverVGPUand framework-aware profile listing, create/destroy, and reconciliation.vendor_vfio_linux.goimplements sysfs discovery, profile availability from free VFs, least-loaded-GPU placement, create/verify/rollback, and teardown that refuses to clear a VF when open VFIO handles exist or a stale instance does not own the assignment.ReconcileVGPUsclears orphaned vendor assignments while honoring a protected device-path set.CreateVGPUstill errors on vendor VFIO until instance lifecycle integration lands; destroy/reconcile and resources reporting already route through the new stack.GetGPUStatusnow takes context, probes vGPU before falling back to passthrough, and documents profile availability as a best-effort VF snapshot.The vGPU integration test is framework-aware (sysfs checks, stop/release, start/reacquire) but skips vendor VFIO hosts until create is wired. Docs/OpenAPI reflect mdev vs vendor VFIO and ephemeral assignment lifecycle.
Reviewed by Cursor Bugbot for commit 9769a33. Bugbot is set up for automated code reviews on this repo. Configure here.