diff options
| author | Sean Christopherson <seanjc@google.com> | 2026-04-23 09:26:27 -0700 |
|---|---|---|
| committer | Sean Christopherson <seanjc@google.com> | 2026-05-13 09:53:55 -0700 |
| commit | 0ffedf43910e44b76c2c1db4e9fbf12b268190c1 (patch) | |
| tree | 996432dfe2bbb92db780b7f11872d1d75df7d466 | |
| parent | b7fbe9a1bf9ee6c967ef77d366ca58c35fcf1887 (diff) | |
KVM: x86: Ensure vendor's exit handler runs before fastpath userspace exits
Move the handling of fastpath userspace exits into vendor code to ensure
KVM runs vendor specific operations that need to run before userspace gains
control of the vCPU. E.g. for VMX (and soon to be for SVM as well), KVM
needs to flush the PML buffer prior to exiting to userspace, otherwise any
memory written by the final KVM_RUN might never be flagged as dirty.
Note, waiting to snapshot CR0 and CR3 until svm_handle_exit() is flawed in
general, as that risks consuming stale state in a fastpath handler. That
will be addressed in a future change.
Fixes: f7f39c50edb9 ("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip")
Cc: stable@vger.kernel.org
Cc: Nikunj A. Dadhania <nikunj@amd.com>
Reviewed-by: Nikunj A. Dadhania <nikunj@amd.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260423162628.490962-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
| -rw-r--r-- | arch/x86/kvm/svm/svm.c | 3 | ||||
| -rw-r--r-- | arch/x86/kvm/vmx/vmx.c | 3 | ||||
| -rw-r--r-- | arch/x86/kvm/x86.c | 3 |
3 files changed, 6 insertions, 3 deletions
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index b78dd8805ebb..fd0362874756 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3671,6 +3671,9 @@ static int svm_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) vcpu->arch.cr3 = svm->vmcb->save.cr3; } + if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) + return 0; + if (is_guest_mode(vcpu)) { int vmexit; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1701db1b2e18..d81b22359918 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6705,6 +6705,9 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) if (enable_pml && !is_guest_mode(vcpu)) vmx_flush_pml_buffer(vcpu); + if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) + return 0; + /* * KVM should never reach this point with a pending nested VM-Enter. * More specifically, short-circuiting VM-Entry to emulate L2 due to diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 48f259015ce4..810ff08780d1 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11588,9 +11588,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (vcpu->arch.apic_attention) kvm_lapic_sync_from_vapic(vcpu); - if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) - return 0; - r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath); return r; |