CVE-2024-26976

In the Linux kernel, the following vulnerability has been resolved:

KVM: Always flush async #PF workqueue when vCPU is being destroyed

Always flush the per-vCPU async #PF workqueue when a vCPU is clearing its completion queue, e.g. when a VM and all its vCPUs is being destroyed. KVM must ensure that none of its workqueue callbacks is running when the last reference to the KVM module is put. Gifting a reference to the associated VM prevents the workqueue callback from dereferencing freed vCPU/VM memory, but does not prevent the KVM module from being unloaded before the callback completes.

Drop the misguided VM refcount gifting, as calling kvmputkvm() from asyncpfexecute() if kvmputkvm() flushes the async #PF workqueue will result in deadlock. asyncpfexecute() can't return until kvmputkvm() finishes, and kvmputkvm() can't return until asyncpfexecute() finishes:

WARNING: CPU: 8 PID: 251 at virt/kvm/kvmmain.c:1435 kvmputkvm+0x2d/0x320 [kvm] Modules linked in: vhostnet vhost vhostiotlb tap kvmintel kvm irqbypass CPU: 8 PID: 251 Comm: kworker/8:1 Tainted: G W 6.6.0-rc1-e7af8d17224a-x86/gmem-vm #119 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 Workqueue: events asyncpfexecute [kvm] RIP: 0010:kvmputkvm+0x2d/0x320 [kvm] Call Trace: <TASK> asyncpfexecute+0x198/0x260 [kvm] processonework+0x145/0x2d0 workerthread+0x27e/0x3a0 kthread+0xba/0xe0 retfromfork+0x2d/0x50 retfromforkasm+0x11/0x20 </TASK> ---[ end trace 0000000000000000 ]--- INFO: task kworker/8:1:251 blocked for more than 120 seconds. Tainted: G W 6.6.0-rc1-e7af8d17224a-x86/gmem-vm #119 "echo 0 > /proc/sys/kernel/hungtasktimeoutsecs" disables this message. task:kworker/8:1 state:D stack:0 pid:251 ppid:2 flags:0x00004000 Workqueue: events asyncpfexecute [kvm] Call Trace: <TASK> _schedule+0x33f/0xa40 schedule+0x53/0xc0 scheduletimeout+0x12a/0x140 _waitforcommon+0x8d/0x1d0 _flushwork.isra.0+0x19f/0x2c0 kvmclearasyncpfcompletionqueue+0x129/0x190 [kvm] kvmarchdestroyvm+0x78/0x1b0 [kvm] kvmputkvm+0x1c1/0x320 [kvm] asyncpfexecute+0x198/0x260 [kvm] processonework+0x145/0x2d0 workerthread+0x27e/0x3a0 kthread+0xba/0xe0 retfromfork+0x2d/0x50 retfromforkasm+0x11/0x20 </TASK>

If kvmclearasyncpfcompletionqueue() actually flushes the workqueue, then there's no need to gift asyncpfexecute() a reference because all invocations of asyncpfexecute() will be forced to complete before the vCPU and its VM are destroyed/freed. And that in turn fixes the module unloading bug as _fput() won't do module_put() on the last vCPU reference until the vCPU has been freed, e.g. if closing the vCPU file also puts the last reference to the KVM module.

Note that kvmcheckasyncpfcompletion() may also take the work item off the completion queue and so also needs to flush the work queue, as the work will not be seen by kvmclearasyncpfcompletionqueue(). Waiting on the workqueue could theoretically delay a vCPU due to waiting for the work to complete, but that's a very, very small chance, and likely a very small delay. kvmarchasyncpagepresentqueued() unconditionally makes a new request, i.e. will effectively delay entering the guest, so the remaining work is really just:

    trace_kvm_async_pf_completed(addr, cr2_or_gpa);

    __kvm_vcpu_wake_up(vcpu);

    mmput(mm);

and mmput() can't drop the last reference to the page tables if the vCPU is still alive, i.e. the vCPU won't get stuck tearing down page tables.

Add a helper to do the flushing, specifically to deal with "wakeup all" work items, as they aren't actually work items, i.e. are never placed in a workqueue. Trying to flush a bogus workqueue entry rightly makes _flushwork() complain (kudos to whoever added that sanity check).

Note, commit 5f6de5cbebee ("KVM: Prevent module exit until al ---truncated---