Fix a potential race between decrementing an LPI's reference count and
evicting that structure from the LPI xarray.
LPI structures are maintained in the VGIC LPI xarray (dist->lpi_xa).
When the reference count of an LPI structure drops to zero,
vgic_release_lpi_locked() removes the structure from the xarray and
frees it under the xarray lock.
However, the release of an LPI can race with a concurrent LPI
re-registration with the same INTID via vgic_add_lpi() on another CPU,
since the reference count drop and the xarray eviction are not performed
in a single atomic step. This can happen e.g. if the guest issues a
DISCARD while the LPI is still referenced from a vCPU's active-pending
list (ap_list), and the same INTID is re-mapped via MAPTI.
Particularly, vgic_release_lpi_locked() is called from two distinct
paths: direct release via vgic_put_irq(), and deferred release via
vgic_release_deleted_lpis(). During direct release, the issue can result
in deleting a newly registered LPI from the xarray:
CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq()
__vgic_put_irq()
refcount_dec_and_test()
vgic_add_lpi()
xa_lock_irqsave()
old_irq = xa_load(.., intid)
vgic_try_get_irq_ref(old_irq) == false
new IRQ inserted --> __xa_store(.., intid, ..)
xa_unlock_irqrestore()
xa_lock_irqsave();
vgic_release_lpi_locked()
__xa_erase(.., irq->intid) <-- BUG: new IRQ is erased
kfree_rcu(old_irq)
During the deferred release path, the old IRQ can be leaked:
CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq_norelease()
__vgic_put_irq()
refcount_dec_and_test()
irq->pending_release = true
vgic_add_lpi()
xa_lock_irqsave()
old_irq = xa_load(.., intid)
vgic_try_get_irq_ref(oldirq) == false
BUG: old IRQ overwritten --> __xa_store(.., intid, ..)
xa_unlock_irqrestore()
vgic_release_deleted_lpis()
xa_lock_irqsave()
xa_for_each() { .. } <-- old IRQ with pending_release = true
is gone, so it cannot be released
To fix the direct release path, move the reference count drop inside
the xarray lock, making sure that vgic_add_lpi() never encounters the
to-be-released LPI.
In the deferred release path, the refcount drop must happen under a raw
spinlock, so the xarray lock cannot be grabbed, and the same solution
does not work. Instead, update vgic_add_lpi(), so that if it evicts
an LPI from the xarray, it takes on the responsibility of freeing it.
Consequently, an LPI may now be freed concurrently after a deferred
release drops the refcount, so accessing the pending_release field is no
longer safe from use-after-free. Delete all uses of the flag, and update
vgic_release_deleted_lpis() to identify orphaned LPIs purely based on
their refcount.
Reported-by: Claude:claude-opus-4-6
Fixes: 3a08a6ca7c37 ("KVM: arm64: vgic-v3: Use bare refcount for VGIC LPIs")
Fixes: d54594accf73 ("KVM: arm64: vgic-v3: Erase LPIs from xarray outside of raw spinlocks")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260715105137.3973823-4-clopez@suse.de
Signed-off-by: Marc Zyngier <maz@kernel.org>
* kvm-arm64/vgic-v5-PPI-fixes: : . : Substantial cleanup of the vgic-v5 PPI support. From the original : cover letter: : : "With the GICv5 PPi support merged in, it has become obvious that a few : things could be improved, both from the correctness and maintainability : angles." : . KVM: arm64: Fix arch timer interrupts for GICv3-on-GICv5 guests irqchip/gic-v5: Immediately exec priority drop following activate Documentation: KVM: Clarify that PMU_V3_IRQ IntID requirements for GICv5 Documentation: KVM: Fix typos in VGICv5 documentation KVM: arm64: selftests: Improve error handling for GICv5 PPI selftest KVM: arm64: selftests: Cleanup unused vars in GICv5 PPI selftest KVM: arm64: selftests: Add missing GIC CDEN to no-vgic-v5 selftest KVM: arm64: vgic-v5: Atomically assign bits to PPI DVI bitmap KVM: arm64: vgic-v5: Add missing trap handing for NV triage KVM: arm64: vgic-v5: Limit support to 64 PPIs KVM: arm64: vgic: Rationalise per-CPU irq accessor KVM: arm64: vgic-v5: Drop defensive checks from vgic_v5_ppi_queue_irq_unlock() KVM: arm64: vgic: Consolidate vgic_allocate_private_irqs_locked() KVM: arm64: vgic: Constify struct irq_ops usage KVM: arm64: vgic-v5: Drop pointless ARM64_HAS_GICV5_CPUIF check KVM: arm64: vgic-v5: Remove use of __assign_bit() with a constant KVM: arm64: vgic-v5: Move PPI caps into kvm_vgic_global_state KVM: arm64: vgic-v5: Add for_each_visible_v5_ppi() iterator Signed-off-by: Marc Zyngier <maz@kernel.org>
vgic-v5 has introduced much more prevalent usage of the struct irq_ops mechanism. In the process, it becomes evident that suffers from two related problems: - it contains flags, rather than only callbacks - it is mutable, because we need to update the above flags Swap the flags for a helper retrieving the flags, and make all irq_ops const, something that is slightly satisfying. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Link: https://lore.kernel.org/r/20260520091949.542365-6-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
Constant vgic properties are usually kept in kvm_vgic_global_state, but the vgic-v5 code does its own thing. Move the ppi_caps data into the global structure, which has the modest additional advantage of making it ro_after_init. Link: https://lore.kernel.org/r/20260520091949.542365-3-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
Just like the timer, the PMU has an interrupt cache that serves little purpose. Drop it. Reviewed-by: Oliver Upton <oupton@kernel.org> Link: https://patch.msgid.link/20260520100200.543845-5-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
The timer code makes use of a per-timer irq level cache, which looks like a very minor optimisation to avoid taking a lock upon updating the GIC view of the interrupt when it is unchanged from the previous state. This is coming in the way of more important correctness issues, so get rid of the cache, which simplifies a couple of minor things. Reviewed-by: Oliver Upton <oupton@kernel.org> Link: https://patch.msgid.link/20260520100200.543845-4-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
The userspace notification of interrupts is has a few problems: - it is utterly pointless - it is annoyingly split between detecting the need for notification and the population of the interrupts in the run structure We can't do anything about the former (yet), but the latter can be addressed. If we detect that we must notify userspace, we know that we are going to exit, as we populate the exit status. Which means we can also populate the interrupt state at this stage and be done with it. This simplifies the structure of the code. Reviewed-by: Oliver Upton <oupton@kernel.org> Link: https://patch.msgid.link/20260520100200.543845-3-maz@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org>
The newly introduced arch_timer_context::direct field is a bit pointless,
as it is always set on timers that are... err... direct, while
we already have a way to get to that by doing a get_map() operation.
Additionally, this field is:
- only set when get_map() is called
- never cleared
and the single point where it is actually checked doesn't call get_map()
at all.
At this stage, it is probably better to just kill it, and rely on
get_map() to give us the correct information.
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Fixes: 9491c63b6cd7b ("KVM: arm64: gic-v5: Enlighten arch timer for GICv5")
Link: https://sashiko.dev/#/patchset/20260319154937.3619520-1-sascha.bischoff%40arm.com
Link: https://patch.msgid.link/20260401103611.357092-12-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
GICv5 systems will likely not support the full set of PPIs. The presence of any virtual PPI is tied to the presence of the physical PPI. Therefore, the available PPIs will be limited by the physical host. Userspace cannot drive any PPIs that are not implemented. Moreover, it is not desirable to expose all PPIs to the guest in the first place, even if they are supported in hardware. Some devices, such as the arch timer, are implemented in KVM, and hence those PPIs shouldn't be driven by userspace, either. Provided a new UAPI: KVM_DEV_ARM_VGIC_GRP_CTRL => KVM_DEV_ARM_VGIC_USERPSPACE_PPIs This allows userspace to query which PPIs it is able to drive via KVM_IRQ_LINE. Additionally, introduce a check in kvm_vm_ioctl_irq_line() to reject any PPIs not in the userspace mask. Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Link: https://patch.msgid.link/20260319154937.3619520-40-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier <maz@kernel.org>
Make it mandatory to use the architected PPI when running a GICv5 guest. Attempts to set anything other than the architected PPI (23) are rejected. Additionally, KVM_ARM_VCPU_PMU_V3_INIT is relaxed to no longer require KVM_ARM_VCPU_PMU_V3_IRQ to be called for GICv5-based guests. In this case, the architectued PPI is automatically used. Documentation is bumped accordingly. Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Joey Gouly <joey.gouly@arm.com> Link: https://patch.msgid.link/20260319154937.3619520-33-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier <maz@kernel.org>