If KVM attempts to translate what it thinks is an L2 GPA with a non-nested MMU, simply WARN and return the GPA, i.e. trust the MMU more than the caller, as there is zero reason to potentially panic the host kernel just because KVM misused an API. Signed-off-by: Sean Christopherson <seanjc@google.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Message-ID: <20260618185746.2023283-1-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The non-MMU changes/preliminary cleanups from the "split kvm_mmu in three" series[1]. The final outcome is to have a single copy of the PDPTRs (in vcpu->arch) instead of two (in root_mmu and nested_mmu). [1] https://lore.kernel.org/kvm/20260603105814.10236-1-pbonzini@redhat.com/T/#t
When L2 runs under nested NPT and uses PAE paging, KVM's cached PDPTRs in mmu->pdptrs[] can hold stale or wrong values after nested transitions and across migration restore, because both nested_svm_load_cr3() and svm_get_nested_state_pages() only refresh PDPTRs on the !nested_npt path. The user-visible bug is on migration restore of an L2 running with nested NPT and 32-bit PAE paging, if userspace uses KVM_SET_SREGS rather than KVM_SET_SREGS2. In that case, load_pdptrs() leaves VCPU_EXREG_PDPTR marked as available, and kvm_pdptr_read() will use a stale translation that used L1 GPAs instead of L2 nGPAs. svm_get_nested_state_pages() runs on first KVM_RUN but skips the refresh because nested_npt_enabled() is true. The CPU itself reads L2's PDPTRs correctly from memory via L1's NPT, but KVM-side walking of guest PAE page tables uses the bogus cached values. Unlike Intel's GUEST_PDPTR0..3 fields in the VMCS, SVM has no VMCB-cached PDPTR state: the in-memory PDPTEs at the current CR3 are the only source of truth, and svm_cache_reg(VCPU_EXREG_PDPTR) simply reloads them from memory via load_pdptrs(). Clearing the avail bit (and the dirty bit because !avail/dirty is invalid) to force a reload when PDPTRs as needed fixes the bug. Do the same for nested_svm_load_cr3()'s nested_npt branch, so that the invariant "PDPTRs need reloading" is handled similarly for both immediate and deferred loading. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-ID: <20260530165545.25599-4-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM SVM changes for 7.2 - Add support for virtualizing gPAT (KVM previously just used L1's PAT when running L2). - Fix goofs where KVM mishandles side effects (e.g. single-step and PMC updates) when emulating VMRUN. - Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most notably where KVM didn't disable interception of IRR, ISR, and TMR regs. - Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
KVM misc x86 changes for 7.2 - Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code gets a chance to handle things like reaping the PML buffer. - Ensure KVM's copy of CR0 and CR3 are up-to-date on SVM prior to invoking fastpath handlers. - Update KVM's view of PV async enabling if and only if the MSR write fully succeeds. - Fix a variety of issues where the emulator doesn't honor guest-debug state, and clean up related code along the way. - Synthesize EPT Violation and #NPF "error code" bits when injecting faults into L1 that didn't originate in hardware (in which case the VMCS/VMCB doesn't hold relevant information). - Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID faulting. - Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a variety of minor bugs along the way. - Fix an OOB memory access due to not checking the VP ID when handling a Hyper-V PV TLB flush for L2. - Fix a bug in the mediated PMU's handling of fixed counters that allowed the guest to bypass the PMU event filter. - Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so the KVM can forward a "retry" status code to the guest, and reserve all unused error codes for future usage. - Misc fixes and cleanups.
Merge the final part of the GHCB 7.2 fixes at https://lore.kernel.org/kvm/20260529183549.1104619-1-pbonzini@redhat.com/. Patches 1-17 have already been included in Linux 7.1; these are minor cleanups, and fixes for behaviors that are suboptimal or contradicting the specification. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that kvm_rax_read() truncates the output value to 32 bits if the vCPU isn't in 64-bit mode, use it instead of the more verbose (and very technically slower) kvm_register_read(). Note! VMLOAD, VMSAVE, and VMRUN emulation are still technically buggy, as they can use EAX (versus RAX) in 64-bit mode via an operand size prefix. Don't bother trying to handle that case, as it would require decoding the code stream, which would open an entirely different can of worms, and in practice no sane guest would shove garbage into RAX[63:32] and then execute VMLOAD/VMSAVE/VMRUN with just EAX. No functional change intended. Cc: Yosry Ahmed <yosry@kernel.org> Reviewed-by: Yosry Ahmed <yosry@kernel.org> Link: https://patch.msgid.link/20260529222223.870923-11-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com>
Make kvm_<reg>_{read,write}() mode-aware (where the value is truncated to
32 bits if the vCPU isn't in 64-bit mode), and convert all the intentional
"raw" accesses to kvm_<reg>_{read,write}_raw() versions. To avoid
confusion and bikeshedding over whether or not explicit 32-bit accesses
should use the "raw" or mode-aware variants, add and use "e" versions, e.g.
for things like RDMSR, WRMSR, and CPUID, where the instruction uses only
bits 31:0, regardless of mode.
No functional change intended (all use of "e" versions is for cases where
the value is already truncated due to bouncing through a u32).
Cc: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260529222223.870923-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reprogram PMU counters on nested transitions for the mediated PMU, to re-evaluate Host-Only and Guest-Only bits and enable/disable the PMU counters accordingly. For example, if Host-Only is set and Guest-Only is cleared, a counter should be disabled when entering guest mode and enabled when exiting guest mode. According to the APM, when EFER.SVME is cleared, setting Host-Only or Guest-Only disables the counter, so also trigger counter reprogramming when EFER.SVME is toggled. Counters setting any of Host-Only and Guest-Only bits are already being tracked in pmc_has_mode_specific_enables, use the bitmap to reprogram these counters. Reprogram the counters synchronously on nested VMRUN/#VMEXIT and EFER.SVME toggling. This is necessary as these instructions are counted based on the new CPU state (after the instruction is retired in hardware). Hence, the PMU needs to be updated before instruction emulation is completed and kvm_pmu_instruction_retired() is called. Defer reprogramming the counters when force leaving guest mode through svm_leave_nested() to avoid potentially reading stale state (e.g. incorrect EFER). All flows force leaving nested are non-architectural, so accuracy is irrelevant. Refactor a helper out of kvm_pmu_request_reprogram_counters() that accepts a boolean allowing synchronous vs deferred reprogramming, and use that from SVM code to support both scenarios. Signed-off-by: Yosry Ahmed <yosry@kernel.org> Link: https://patch.msgid.link/20260527234711.4175166-13-yosry@kernel.org Signed-off-by: Sean Christopherson <seanjc@google.com>
A successful VMRUN retires in guest mode and should be counted by the PMU as a guest instruction. Move the call to kvm_pmu_instruction_retired() after potentially entering guest mode, such that VMRUN is counted correctly. The PMU event will be matched against L2's CPL, but otherwise this does not change the behavior in terms of guest vs. host, because KVM does not virtualize Host-Only/Guest-Only PMC controls yet, so all instructions are counted regardless of the vCPU's host/guest state. But this change is needed for the incoming support for Host-Only/Guest-Only controls to count VMRUN correctly. Signed-off-by: Yosry Ahmed <yosry@kernel.org> Link: https://patch.msgid.link/20260527234711.4175166-5-yosry@kernel.org Signed-off-by: Sean Christopherson <seanjc@google.com>