linux.git/include/linux/kvm_host.h, branch v7.3-rc2 Linux kernel source tree Merge tag 'kvm-x86-misc-7.3' of https://github.com/kvm-x86/linux into HEAD 2026-08-18T11:41:51+00:00 Paolo Bonzini pbonzini@redhat.com 2026-08-18T11:41:51+00:00 d75b48460559423f8c38aafed7a9cdec91d26d57 KVM x86 misc changes for 7.3 - Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs. - Harden the SNP and TDX "populate" ioctls against bad input, and to prepare for supporting in-place private<=>shared conversion. - Fix a variety of #DB priority bugs. - Fix a class of races related to enabling Hyper-V emulation on a vCPU after the vCPU is visible to the rest of KVM. - Use static calls for nested virtualization ops. - Move more KVM-internal code out of x86's kvm_host.h. - Enumerate support for a variety of Zhaoxin instructions that don't require explicit virtualization. - Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path. - Harden kvm_vcpu_map() against double-mapping and thus leaking references. - Misc fixes and cleanups, e.g. for largely benign syzkaller splats. 
KVM x86 misc changes for 7.3

 - Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs.

 - Harden the SNP and TDX "populate" ioctls against bad input, and to prepare
   for supporting in-place private<=>shared conversion.

 - Fix a variety of #DB priority bugs.

 - Fix a class of races related to enabling Hyper-V emulation on a vCPU after
   the vCPU is visible to the rest of KVM.

 - Use static calls for nested virtualization ops.

 - Move more KVM-internal code out of x86's kvm_host.h.

 - Enumerate support for a variety of Zhaoxin instructions that don't require
   explicit virtualization.

 - Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path.

 - Harden kvm_vcpu_map() against double-mapping and thus leaking references.

 - Misc fixes and cleanups, e.g. for largely benign syzkaller splats.
Merge tag 'kvm-x86-coco-7.3' of https://github.com/kvm-x86/linux into HEAD 2026-08-18T11:25:39+00:00 Paolo Bonzini pbonzini@redhat.com 2026-08-18T11:25:39+00:00 159ed1ae35b16a2d4603742fc32a6616fe42a78a KVM guest_memfd and x86 CoCo changes for 7.3 - Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State Change request. - Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare for in-place private<=>shared conversion, and clean up a few warts along the way. 
KVM guest_memfd and x86 CoCo changes for 7.3

 - Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is
   zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State
   Change request.

 - Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare
   for in-place private<=>shared conversion, and clean up a few warts along the
   way.
KVM: Check for duplicate vcpu_id as early as possible 2026-07-31T22:13:30+00:00 Dmytro Maluka dmaluka@chromium.org 2026-07-29T17:06:20+00:00 97d65b544f48b2ee49f6aea32145e3e7969955dc If userspace tries to create a vCPU with the same vcpu_id as an existing one, kvm_vm_ioctl_create_vcpu() checks for that and fails with -EEXIST only after it already created the vCPU via kvm_arch_vcpu_create(). As a result, even though this newly created vCPU is destroyed in the failure path, the fact that it is temporarily created with an invalid vcpu_id and that there are temporarily two vCPUs with the same vcpu_id is a potential source of subtle issues. In particular, this prevents fixing an VMX IPIv issue where a stale entry left in the VM's PI descriptor table after the vCPU is destroyed in the failure path. The right way to fix that issue is to clear that entry when destroying the vCPU, however right now that would have a nasty side effect: since the same entry is used for the other, previously created vCPU with same vcpu_id, clearing it would mean effectively disabling IPIv for that existing good vCPU. So to avoid this and similar problems, check for duplicate vcpu_id as early in the vCPU creation path as possible, before kvm_arch_vcpu_create() and even before kvm_arch_vcpu_precreate(). Simply moving the existing kvm_get_vcpu_by_id() check earlier doesn't work, as kvm->lock is dropped and reacquired, i.e. moving kvm_get_vcpu_by_id() would introduce a race: 1. vCPU A is being created but not installed in kvm->vcpu_array yet. 2. vCPU B with the same vcpu_id is being created. It passes the duplicated vcpu_id check, since the check doesn't find vCPU A in kvm->vcpu_array. 3. vCPU A is installed in kvm->vcpu_array, vCPU creation succeeds. 4. vCPU B with the same vcpu_id is installed in kvm->vcpu_array, vCPU creation succeeds. So introduce the bitmap of vcpu_ids used by the VM, in order to safely check if the given vcpu_id is used and mark is as used before releasing kvm->lock first time. Alternatively, KVM could use another Xarray[*] for roughly the same code complexity, which would minimize KVM's steady state memory footprint at the cost of higher runtime latency (to allocate and free entries). Given that the worst case scenario is 256 bytes per-VM (on x86, which allows up to 16KiB vCPU IDs), go with the slightly simpler approach until there's a need to save memory. Suggested-by: Sean Christopherson <seanjc@google.com> Link: https://lore.kernel.org/kvm/al6eg7C-2sDBEAFD@google.com [*] Signed-off-by: Dmytro Maluka <dmaluka@chromium.org> Reviewed-by: Kai Huang <kai.huang@intel.com> Link: https://patch.msgid.link/20260729170621.308809-2-dmaluka@chromium.org [sean: massage changelog] Signed-off-by: Sean Christopherson <seanjc@google.com> 
If userspace tries to create a vCPU with the same vcpu_id as an existing
one, kvm_vm_ioctl_create_vcpu() checks for that and fails with -EEXIST
only after it already created the vCPU via kvm_arch_vcpu_create(). As a
result, even though this newly created vCPU is destroyed in the failure
path, the fact that it is temporarily created with an invalid vcpu_id
and that there are temporarily two vCPUs with the same vcpu_id is a
potential source of subtle issues.

In particular, this prevents fixing an VMX IPIv issue where a stale entry
left in the VM's PI descriptor table after the vCPU is destroyed in the
failure path.  The right way to fix that issue is to clear that entry when
destroying the vCPU, however right now that would have a nasty side effect:
since the same entry is used for the other, previously created vCPU with
same vcpu_id, clearing it would mean effectively disabling IPIv for that
existing good vCPU.

So to avoid this and similar problems, check for duplicate vcpu_id as
early in the vCPU creation path as possible, before
kvm_arch_vcpu_create() and even before kvm_arch_vcpu_precreate().

Simply moving the existing kvm_get_vcpu_by_id() check earlier doesn't work,
as kvm->lock is dropped and reacquired, i.e. moving kvm_get_vcpu_by_id()
would introduce a race:

  1. vCPU A is being created but not installed in kvm->vcpu_array yet.
  2. vCPU B with the same vcpu_id is being created. It passes the
     duplicated vcpu_id check, since the check doesn't find vCPU A in
     kvm->vcpu_array.
  3. vCPU A is installed in kvm->vcpu_array, vCPU creation succeeds.
  4. vCPU B with the same vcpu_id is installed in kvm->vcpu_array, vCPU
     creation succeeds.

So introduce the bitmap of vcpu_ids used by the VM, in order to safely
check if the given vcpu_id is used and mark is as used before releasing
kvm->lock first time.

Alternatively, KVM could use another Xarray[*] for roughly the same code
complexity, which would minimize KVM's steady state memory footprint at
the cost of higher runtime latency (to allocate and free entries).  Given
that the worst case scenario is 256 bytes per-VM (on x86, which allows up
to 16KiB vCPU IDs), go with the slightly simpler approach until there's a
need to save memory.

Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/kvm/al6eg7C-2sDBEAFD@google.com [*]
Signed-off-by: Dmytro Maluka <dmaluka@chromium.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260729170621.308809-2-dmaluka@chromium.org
[sean: massage changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: nSVM: Add CLASS()es for automagically handling local kvm_vcpu_map() usage 2026-07-28T20:48:43+00:00 Sean Christopherson seanjc@google.com 2026-07-24T00:47:53+00:00 d196e4d91e7845b9e65436574bff63234a6f4d1d Add CLASS() definitions for locally mapping a PFN using kvm_vcpu_map() given a vCPU+gfn pair. In addition to eliminating the need to manually do unmap(), e.g. in error paths, this will allow hardening KVM against double-mapping without having to manually ensure every on-stack declaration is zero-initialized. Use "map local" as the primary terminology as the basic concept is more or less the same as kmap_local(): ensure the current context has a kernel mapping to the underlying memory. Immediately convert the relatively straightforward nested SVM flows, and defer converting the more involved SMM flows to a separate change. No functional change intended. Cc: Yosry Ahmed <yosry@kernel.org> Link: https://patch.msgid.link/20260724004757.131420-3-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
Add CLASS() definitions for locally mapping a PFN using kvm_vcpu_map()
given a vCPU+gfn pair.  In addition to eliminating the need to manually
do unmap(), e.g. in error paths, this will allow hardening KVM against
double-mapping without having to manually ensure every on-stack declaration
is zero-initialized.

Use "map local" as the primary terminology as the basic concept is more or
less the same as kmap_local(): ensure the current context has a kernel
mapping to the underlying memory.

Immediately convert the relatively straightforward nested SVM flows, and
defer converting the more involved SMM flows to a separate change.

No functional change intended.

Cc: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260724004757.131420-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: guest_memfd: Explicitly pass number of pages to make_private() hook 2026-07-27T16:43:19+00:00 Sean Christopherson seanjc@google.com 2026-07-23T21:08:10+00:00 fb50ca77b672fd359453728fc59730105f2bf7eb Tweak the guest_memfd make_private() hook to explicitly pass the number of pages to align with the signature of the make_shared() hook, and because the existing code is outright broken if a guest_memfd folio is comprised of more than one page (which can't happen, yet). The SNP code *tries* to create a corresponding huge entry, but if the RMP must use 4KiB entries for whatever reason, KVM will only convert the first pfn, and not the entire range of pfns that will be mapped into the guest. Alternatively, @max_order could simply be repurposed as _the_ @order, but that will fall apart when in-place conversion comes along, at which point KVM will need to deal with conversions that aren't bound 1:1 to a folio. I.e. the number of pages to convert may not be exactly a power-of-2 (and folios don't strictly guarantee power-of-2 pages anyways). WARN in the SNP code if the number of pages to prepare is anything other than '1', i.e. if guest_memfd is trying to prepare/convert more than a single 4KiB page, as sev_gmem_prepare() doesn't actually handle conversion greater than order-0 folios. Opportunistically swap the ordering of @pfn and @gfn params for kvm_x86_ops.gmem_make_private() to match kvm_arch_gmem_make_private(). Fixes: b85524314a3d ("KVM: guest_memfd: delay kvm_gmem_prepare_folio() until the memory is passed to the guest") Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Ackerley Tng <ackerleytng@google.com> Link: https://patch.msgid.link/20260723210811.72720-9-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
Tweak the guest_memfd make_private() hook to explicitly pass the number of
pages to align with the signature of the make_shared() hook, and because
the existing code is outright broken if a guest_memfd folio is comprised of
more than one page (which can't happen, yet).  The SNP code *tries* to
create a corresponding huge entry, but if the RMP must use 4KiB entries for
whatever reason, KVM will only convert the first pfn, and not the entire
range of pfns that will be mapped into the guest.

Alternatively, @max_order could simply be repurposed as _the_ @order, but
that will fall apart when in-place conversion comes along, at which point
KVM will need to deal with conversions that aren't bound 1:1 to a folio.
I.e. the number of pages to convert may not be exactly a power-of-2 (and
folios don't strictly guarantee power-of-2 pages anyways).

WARN in the SNP code if the number of pages to prepare is anything other
than '1', i.e. if guest_memfd is trying to prepare/convert more than a
single 4KiB page, as sev_gmem_prepare() doesn't actually handle conversion
greater than order-0 folios.

Opportunistically swap the ordering of @pfn and @gfn params for
kvm_x86_ops.gmem_make_private() to match kvm_arch_gmem_make_private().

Fixes: b85524314a3d ("KVM: guest_memfd: delay kvm_gmem_prepare_folio() until the memory is passed to the guest")
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260723210811.72720-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT 2026-07-27T16:42:42+00:00 Sean Christopherson seanjc@google.com 2026-07-23T21:08:09+00:00 2131c4f763d2e4cbbe5c227a406488c38e23f806 Rework guest_memfd's prepare() hook into a more accurate make_private(), and rework its Kconfig from PREPARE to a more generic CONVERT. This will allow x86 to share (pun intended) a kvm_x86_ops.gmem_make_shared() hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP. No functional change intended. Reviewed-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Link: https://patch.msgid.link/20260723210811.72720-8-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
Rework guest_memfd's prepare() hook into a more accurate make_private(),
and rework its Kconfig from PREPARE to a more generic CONVERT.  This will
allow x86 to share (pun intended) a kvm_x86_ops.gmem_make_shared() hook
between the "convert to shared" and "reclaim" flows, which are one and the
same for SNP.

No functional change intended.

Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260723210811.72720-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it 2026-07-27T16:42:42+00:00 Sean Christopherson seanjc@google.com 2026-07-23T21:08:04+00:00 7b8529e70a11ed110a15568f6b391a5f32eef67f Rename guest_memfd's invalidate() hook to reclaim() and isolate it via its own RECLAIM Kconfig, as the hook is called when a folio is freed, which is far too late and lacks sufficient information for KVM to actually invalidate its usage of the memory. E.g. SNP uses the hook to convert memory back to SHARED so that it can be safely accessed by the host, there is no invalidation of guest mappings anywhere. Isolating the hook will also allow pKVM on arm64 to opt-in to reclaim() without also having to differentiate between reclaim and conversions to shared for active VMs. Keep guest_memfd's trampoline, even though it would be trivial to wire up .free_folio() directly to an arch callback, to avoid bleeding guest_memfd internals into arch code (specifically, avoid referencing folios in arch code). Leave the kvm_x86_ops hook as-is for the moment, as "reclaim" on SNP is the same as convert-to-shared, i.e. using a different name for the x86 hook will allow reusing it for in-place conversion. Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Ackerley Tng <ackerleytng@google.com> Link: https://patch.msgid.link/20260723210811.72720-3-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
Rename guest_memfd's invalidate() hook to reclaim() and isolate it via its
own RECLAIM Kconfig, as the hook is called when a folio is freed, which is
far too late and lacks sufficient information for KVM to actually
invalidate its usage of the memory.  E.g. SNP uses the hook to convert
memory back to SHARED so that it can be safely accessed by the host, there
is no invalidation of guest mappings anywhere.  Isolating the hook will
also allow pKVM on arm64 to opt-in to reclaim() without also having to
differentiate between reclaim and conversions to shared for active VMs.

Keep guest_memfd's trampoline, even though it would be trivial to wire up
.free_folio() directly to an arch callback, to avoid bleeding guest_memfd
internals into arch code (specifically, avoid referencing folios in arch
code).

Leave the kvm_x86_ops hook as-is for the moment, as "reclaim" on SNP is
the same as convert-to-shared, i.e. using a different name for the x86 hook
will allow reusing it for in-place conversion.

Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260723210811.72720-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: guest_memfd: Pass the number of pages instead of the end pfn into .invalidate() 2026-07-27T16:42:42+00:00 Sean Christopherson seanjc@google.com 2026-07-23T21:08:03+00:00 0f0893ac5af3ac9d18440955f302bd20a35ccab2 Pass the number of pages to "invalidate", i.e. reclaim, instead of the end pfn, as a first step towards aligning the function prototypes between the de facto "to private" and "to shared" arch hooks. Eventually, the goal is to end up with kvm_gmem_arch_make_{private,shared}(), and in both cases, providing the number of pages makes the call sites slightly nicer, and also avoids any confusion over whether the end pfn is inclusive or exclusive. Opportunistically rename "start" to "pfn", again to align with the expected signature of make_private() (which needs to pass a starting gfn as well, at which point the "start" becomes noise). No functional change intended. Cc: Fuad Tabba <fuad.tabba@linux.dev> Cc: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Link: https://patch.msgid.link/20260723210811.72720-2-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
Pass the number of pages to "invalidate", i.e. reclaim, instead of the end
pfn, as a first step towards aligning the function prototypes between the
de facto "to private" and "to shared" arch hooks.  Eventually, the goal
is to end up with kvm_gmem_arch_make_{private,shared}(), and in both cases,
providing the number of pages makes the call sites slightly nicer, and also
avoids any confusion over whether the end pfn is inclusive or exclusive.

Opportunistically rename "start" to "pfn", again to align with the expected
signature of make_private() (which needs to pass a starting gfn as well, at
which point the "start" becomes noise).

No functional change intended.

Cc: Fuad Tabba <fuad.tabba@linux.dev>
Cc: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260723210811.72720-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: SEV: Forcefully invalidate SNP VMSA if its backing gmem page is zapped 2026-07-10T16:01:11+00:00 Sean Christopherson seanjc@google.com 2026-07-09T20:49:40+00:00 d1a3c216233413f57f5341a9b878b7e2dde7e785 Wire up a gmem_invalidate_range() call for SNP VMs, and use it to force vCPUs to reload/recheck their guest-provided VMSA if the backing gmem page is being invalidated, e.g. is being PUNCH_HOLE'd. Use the same core logic to handle invalidations as VMX does for the APIC-access page, as the two concepts are nearly identical: shove the physical address of a page into the vCPU's control structure: 1. Snapshot the invalidation sequence counter 2. Grab the pfn (from guest_memfd in this case) 3. Acquire mmu_lock for read 4. Re-request reload if retry is needed, otherwise commit the change. Note, the re-request action in #4 is necessary as KVM's retry logic is fuzzy, i.e. can get false positives. If the guest_memfd page has been dropped, at some point a subsequent reload will fail to get a PFN from guest_memfd, and KVM will fail KVM_RUN. If the retry was due to a false positive, KVM will retry until there are no relevant MMU notifier events (and will retry in the "outer" loop, i.e. will drop locks and resched as needed). Note #2! Take care to invalidate the VMSA when a relevant memslot is DELETED or MOVED, as invalidations in response to PUNCH_HOLE are predicated on memslot bindings (KVM doesn't know what GFN range(s) to invalidate without a binding). And more importantly, the VMSA mapping requires a memslot, i.e. must be invalidated if its memslots disappears, regardless of the state of the underlying guest_memfd inode. Failure to invalidate the vCPU's control.vmsa_pa (which is checked by pre_sev_run()) can prevent KVM from properly freeing the page as firmware will reject the RMPUPDATE to reclaim the page with FAIL_INUSE if the vCPU is actively running, i.e. if VMSA page is in-use. That in turn leads to an RMP #PF on the next use, as the page will still be assigned to the SNP VM. SEV-SNP: RMPUPDATE failed for PFN 78d198, pg_level: 1, ret: 3 SEV-SNP: PFN 0x78d198, RMP entry: [0xfff0000000144001 - 0x000000000000000f] CPU: 3 UID: 0 PID: 31345 Comm: sev_snp_vmsa_pu Tainted: G U O Tainted: [U]=USER, [O]=OOT_MODULE Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026 Call Trace: <TASK> dump_stack_lvl+0x54/0x70 rmpupdate+0x12c/0x140 rmp_make_shared+0x3b/0x60 sev_gmem_invalidate+0xe0/0x170 [kvm_amd] delete_from_page_cache_batch+0x1d8/0x220 truncate_inode_pages_range+0x120/0x3d0 kvm_gmem_fallocate+0x19a/0x270 [kvm] vfs_fallocate+0x1bc/0x1f0 __x64_sys_fallocate+0x48/0x70 do_syscall_64+0x10a/0x480 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x496c7e </TASK> ------------[ cut here ]------------ SEV: Failed to update RMP entry for PFN 0x78d198 error -14 WARNING: arch/x86/kvm/svm/sev.c:5160 at sev_gmem_invalidate+0x126/0x170 [kvm_amd], CPU#3: sev_snp_vmsa_pu/31345 CPU: 3 UID: 0 PID: 31345 Comm: sev_snp_vmsa_pu Tainted: G U O Tainted: [U]=USER, [O]=OOT_MODULE Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026 RIP: 0010:sev_gmem_invalidate+0x12b/0x170 [kvm_amd] Call Trace: <TASK> delete_from_page_cache_batch+0x1d8/0x220 truncate_inode_pages_range+0x120/0x3d0 kvm_gmem_fallocate+0x19a/0x270 [kvm] vfs_fallocate+0x1bc/0x1f0 __x64_sys_fallocate+0x48/0x70 do_syscall_64+0x10a/0x480 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x496c7e </TASK> irq event stamp: 20689 hardirqs last enabled at (20699): [<ffffffff8e76092c>] __console_unlock+0x5c/0x60 hardirqs last disabled at (20708): [<ffffffff8e760911>] __console_unlock+0x41/0x60 softirqs last enabled at (20722): [<ffffffff8e6cd74e>] __irq_exit_rcu+0x7e/0x140 softirqs last disabled at (20717): [<ffffffff8e6cd74e>] __irq_exit_rcu+0x7e/0x140 ---[ end trace 0000000000000000 ]--- BUG: unable to handle page fault for address: ffff99a64d198000 #PF: supervisor write access in kernel mode #PF: error_code(0x80000003) - RMP violation PGD 13eb001067 P4D 13eb001067 PUD 78d1d1063 PMD 1184e0063 PTE 800000078d198163 SEV-SNP: PFN 0x78d198, RMP entry: [0x6030000000144001 - 0x000000000000000f] Oops: Oops: 0003 [#1] SMP CPU: 3 UID: 0 PID: 31407 Comm: highlanderd_hea Tainted: G U W O Tainted: [U]=USER, [W]=WARN, [O]=OOT_MODULE Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026 RIP: 0010:prep_new_page+0x67/0x220 Call Trace: <TASK> get_page_from_freelist+0x1c40/0x1c70 __alloc_frozen_pages_noprof+0xca/0x1f0 alloc_pages_mpol+0x10b/0x1b0 alloc_pages_noprof+0x81/0x90 pte_alloc_one+0x1b/0xd0 do_pte_missing+0xdf/0x1020 handle_mm_fault+0x7c7/0xb20 do_user_addr_fault+0x268/0x6b0 exc_page_fault+0x67/0xa0 asm_exc_page_fault+0x26/0x30 RIP: 0033:0x4a6b1e </TASK> gsmi: Log Shutdown Reason 0x03 CR2: ffff99a64d198000 ---[ end trace 0000000000000000 ]--- RIP: 0010:prep_new_page+0x67/0x220 Drop the pseudo-TODO comment about needing to pin the page if guest_memfd every supports migration, as integrating with invalidations events means KVM will Just Work if/when page migration is ever supported (assuming SNP hardware supports migrating VMSA pages). Note #3, invalidate() and invalidate_range() have _completely_ different semantics; the new invalidate_range() is a true invalidation, whereas the existing invalidate() is really a "make shared" operation. Ignore the confusing naming and poor Kconfig bundling for the moment to minimize the delta for LTS kernels, the mess will be cleaned up shortly. Reported-by: Hyunwoo Kim <imv4bel@gmail.com> Closes: https://lore.kernel.org/all/aimMWzAf5b3luM0b@v4bel Fixes: e366f92ea99e ("KVM: SEV: Support SEV-SNP AP Creation NAE event") Cc: stable@vger.kernel.org Cc: Tom Lendacky <thomas.lendacky@amd.com> Cc: Michael Roth <michael.roth@amd.com> Cc: Jörg Rödel <joro@8bytes.org> Cc: Fuad Tabba <tabba@google.com> Cc: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Link: https://patch.msgid.link/20260709204948.1988414-11-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
Wire up a gmem_invalidate_range() call for SNP VMs, and use it to force
vCPUs to reload/recheck their guest-provided VMSA if the backing gmem
page is being invalidated, e.g. is being PUNCH_HOLE'd.  Use the same core
logic to handle invalidations as VMX does for the APIC-access page, as the
two concepts are nearly identical: shove the physical address of a page
into the vCPU's control structure:

 1. Snapshot the invalidation sequence counter
 2. Grab the pfn (from guest_memfd in this case)
 3. Acquire mmu_lock for read
 4. Re-request reload if retry is needed, otherwise commit the change.

Note, the re-request action in #4 is necessary as KVM's retry logic is
fuzzy, i.e. can get false positives.  If the guest_memfd page has been
dropped, at some point a subsequent reload will fail to get a PFN from
guest_memfd, and KVM will fail KVM_RUN.  If the retry was due to a false
positive, KVM will retry until there are no relevant MMU notifier events
(and will retry in the "outer" loop, i.e. will drop locks and resched as
needed).

Note #2!  Take care to invalidate the VMSA when a relevant memslot is
DELETED or MOVED, as invalidations in response to PUNCH_HOLE are predicated
on memslot bindings (KVM doesn't know what GFN range(s) to invalidate
without a binding).  And more importantly, the VMSA mapping requires a
memslot, i.e. must be invalidated if its memslots disappears, regardless of
the state of the underlying guest_memfd inode.

Failure to invalidate the vCPU's control.vmsa_pa (which is checked by
pre_sev_run()) can prevent KVM from properly freeing the page as firmware
will reject the RMPUPDATE to reclaim the page with FAIL_INUSE if the vCPU
is actively running, i.e. if VMSA page is in-use.  That in turn leads to an
RMP #PF on the next use, as the page will still be assigned to the SNP VM.

  SEV-SNP: RMPUPDATE failed for PFN 78d198, pg_level: 1, ret: 3
  SEV-SNP: PFN 0x78d198, RMP entry: [0xfff0000000144001 - 0x000000000000000f]
  CPU: 3 UID: 0 PID: 31345 Comm: sev_snp_vmsa_pu Tainted: G     U     O
  Tainted: [U]=USER, [O]=OOT_MODULE
  Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026
  Call Trace:
   <TASK>
   dump_stack_lvl+0x54/0x70
   rmpupdate+0x12c/0x140
   rmp_make_shared+0x3b/0x60
   sev_gmem_invalidate+0xe0/0x170 [kvm_amd]
   delete_from_page_cache_batch+0x1d8/0x220
   truncate_inode_pages_range+0x120/0x3d0
   kvm_gmem_fallocate+0x19a/0x270 [kvm]
   vfs_fallocate+0x1bc/0x1f0
   __x64_sys_fallocate+0x48/0x70
   do_syscall_64+0x10a/0x480
   entry_SYSCALL_64_after_hwframe+0x4b/0x53
  RIP: 0033:0x496c7e
   </TASK>
  ------------[ cut here ]------------
  SEV: Failed to update RMP entry for PFN 0x78d198 error -14
  WARNING: arch/x86/kvm/svm/sev.c:5160 at sev_gmem_invalidate+0x126/0x170 [kvm_amd], CPU#3: sev_snp_vmsa_pu/31345
  CPU: 3 UID: 0 PID: 31345 Comm: sev_snp_vmsa_pu Tainted: G     U     O
  Tainted: [U]=USER, [O]=OOT_MODULE
  Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026
  RIP: 0010:sev_gmem_invalidate+0x12b/0x170 [kvm_amd]
  Call Trace:
   <TASK>
   delete_from_page_cache_batch+0x1d8/0x220
   truncate_inode_pages_range+0x120/0x3d0
   kvm_gmem_fallocate+0x19a/0x270 [kvm]
   vfs_fallocate+0x1bc/0x1f0
   __x64_sys_fallocate+0x48/0x70
   do_syscall_64+0x10a/0x480
   entry_SYSCALL_64_after_hwframe+0x4b/0x53
  RIP: 0033:0x496c7e
   </TASK>
  irq event stamp: 20689
  hardirqs last  enabled at (20699): [<ffffffff8e76092c>] __console_unlock+0x5c/0x60
  hardirqs last disabled at (20708): [<ffffffff8e760911>] __console_unlock+0x41/0x60
  softirqs last  enabled at (20722): [<ffffffff8e6cd74e>] __irq_exit_rcu+0x7e/0x140
  softirqs last disabled at (20717): [<ffffffff8e6cd74e>] __irq_exit_rcu+0x7e/0x140
  ---[ end trace 0000000000000000 ]---
  BUG: unable to handle page fault for address: ffff99a64d198000
  #PF: supervisor write access in kernel mode
  #PF: error_code(0x80000003) - RMP violation
  PGD 13eb001067 P4D 13eb001067 PUD 78d1d1063 PMD 1184e0063 PTE 800000078d198163
  SEV-SNP: PFN 0x78d198, RMP entry: [0x6030000000144001 - 0x000000000000000f]
  Oops: Oops: 0003 [#1] SMP
  CPU: 3 UID: 0 PID: 31407 Comm: highlanderd_hea Tainted: G     U  W  O
  Tainted: [U]=USER, [W]=WARN, [O]=OOT_MODULE
  Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026
  RIP: 0010:prep_new_page+0x67/0x220
  Call Trace:
   <TASK>
   get_page_from_freelist+0x1c40/0x1c70
   __alloc_frozen_pages_noprof+0xca/0x1f0
   alloc_pages_mpol+0x10b/0x1b0
   alloc_pages_noprof+0x81/0x90
   pte_alloc_one+0x1b/0xd0
   do_pte_missing+0xdf/0x1020
   handle_mm_fault+0x7c7/0xb20
   do_user_addr_fault+0x268/0x6b0
   exc_page_fault+0x67/0xa0
   asm_exc_page_fault+0x26/0x30
  RIP: 0033:0x4a6b1e
   </TASK>
  gsmi: Log Shutdown Reason 0x03
  CR2: ffff99a64d198000
  ---[ end trace 0000000000000000 ]---
  RIP: 0010:prep_new_page+0x67/0x220

Drop the pseudo-TODO comment about needing to pin the page if guest_memfd
every supports migration, as integrating with invalidations events means
KVM will Just Work if/when page migration is ever supported (assuming SNP
hardware supports migrating VMSA pages).

Note #3, invalidate() and invalidate_range() have _completely_ different
semantics; the new invalidate_range() is a true invalidation, whereas the
existing invalidate() is really a "make shared" operation.  Ignore the
confusing naming and poor Kconfig bundling for the moment to minimize the
delta for LTS kernels, the mess will be cleaned up shortly.

Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Closes: https://lore.kernel.org/all/aimMWzAf5b3luM0b@v4bel
Fixes: e366f92ea99e ("KVM: SEV: Support SEV-SNP AP Creation NAE event")
Cc: stable@vger.kernel.org
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Michael Roth <michael.roth@amd.com>
Cc: Jörg Rödel <joro@8bytes.org>
Cc: Fuad Tabba <tabba@google.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM: x86: Treat a vCPU as unreachable if its index is invalid 2026-07-08T20:41:13+00:00 Sean Christopherson seanjc@google.com 2026-06-30T22:56:16+00:00 c33aef581703cb41e2a4900c258018bc7aa1ff54 In the "vCPU locked or unreachable" lockdep assertion, treat a vCPU as unreachable if its index is invalid, i.e. if the vCPU is in the process of being created. Until the vCPU is inserted into the array of vCPUs, the only way to get at the vCPU is via kvm_vm_ioctl_create_vcpu(). Note, the actual index is set _before_ adding the vCPU to the array, i.e. there's no risk of a false negative on the lockdep assertion. Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com> Link: https://patch.msgid.link/20260630225619.511632-10-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> 
In the "vCPU locked or unreachable" lockdep assertion, treat a vCPU as
unreachable if its index is invalid, i.e. if the vCPU is in the process of
being created.  Until the vCPU is inserted into the array of vCPUs, the
only way to get at the vCPU is via kvm_vm_ioctl_create_vcpu().  Note, the
actual index is set _before_ adding the vCPU to the array, i.e. there's no
risk of a false negative on the lockdep assertion.

Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>