linux-stable.git/include/rdma/frmr_pools.h, branch linux-rolling-stable Linux kernel stable tree RDMA/core: Add ib_frmr_pool_drop for unrecoverable handles 2026-07-24T14:19:48+00:00 Michael Guralnik michaelgur@nvidia.com 2026-06-10T00:01:44+00:00 7800de33e1cb86aa28fdc6329edd93b4924d4ac3 [ Upstream commit ddbc251be18fb82884ee6e9af634cc9f1171a4d6 ] A driver that has popped a handle from an FRMR pool can hit failures that leave the handle in a state where it can't safely be returned for reuse. The driver destroys the handle itself, but the pool has no way to learn about it, so the in_use counter drifts upward. Add ib_frmr_pool_drop to balance the pool's accounting in this case. Every pop is now balanced by exactly one push or drop. Fixes: 36680ef7bceb ("RDMA/mlx5: Switch from MR cache to FRMR pools") Link: https://patch.msgid.link/r/20260610000145.820592-9-michaelgur@nvidia.com Signed-off-by: Michael Guralnik <michaelgur@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit ddbc251be18fb82884ee6e9af634cc9f1171a4d6 ]

A driver that has popped a handle from an FRMR pool can hit failures
that leave the handle in a state where it can't safely be returned
for reuse. The driver destroys the handle itself, but the pool has
no way to learn about it, so the in_use counter drifts upward.

Add ib_frmr_pool_drop to balance the pool's accounting in this case.
Every pop is now balanced by exactly one push or drop.

Fixes: 36680ef7bceb ("RDMA/mlx5: Switch from MR cache to FRMR pools")
Link: https://patch.msgid.link/r/20260610000145.820592-9-michaelgur@nvidia.com
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
RDMA/core: Fix FRMR handle leak on push failure 2026-07-24T14:19:48+00:00 Michael Guralnik michaelgur@nvidia.com 2026-06-10T00:01:43+00:00 d9e9d10e56ab613fa74c47bd33f45ceed6988cc6 [ Upstream commit 8c76126b866649d8e8acc09a06f2b03b6ff88900 ] Failure to push a handle to the pool, caused by ENOMEM on queue page allocation, will trigger missing in_use counter update, skewing pool state indefinitely. Fix that by moving the handling of handle destruction in such case into the FRMR code, ensuring the handle is either pushed to the pool or destroyed inside the same function. Adjust mlx5_ib call site accordingly. Fixes: ce5df0b891ed ("IB/core: Introduce FRMR pools") Link: https://patch.msgid.link/r/20260610000145.820592-8-michaelgur@nvidia.com Signed-off-by: Michael Guralnik <michaelgur@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 8c76126b866649d8e8acc09a06f2b03b6ff88900 ]

Failure to push a handle to the pool, caused by ENOMEM on queue page
allocation, will trigger missing in_use counter update, skewing pool
state indefinitely.
Fix that by moving the handling of handle destruction in such case
into the FRMR code, ensuring the handle is either pushed to the pool
or destroyed inside the same function.

Adjust mlx5_ib call site accordingly.

Fixes: ce5df0b891ed ("IB/core: Introduce FRMR pools")
Link: https://patch.msgid.link/r/20260610000145.820592-8-michaelgur@nvidia.com
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
RDMA/core: Add pinned handles to FRMR pools 2026-03-02T18:45:13+00:00 Michael Guralnik michaelgur@nvidia.com 2026-02-26T13:52:10+00:00 020d189d16a62ed56115cce7e255459cf0eeb4e6 Add a configuration of pinned handles on a specific FRMR pool. The configured amount of pinned handles will not be aged and will stay available for users to claim. Upon setting the amount of pinned handles to an FRMR pool, we will make sure we have at least the pinned amount of handles associated with the pool and create more, if necessary. The count for pinned handles take into account handles that are used by user MRs and handles in the queue. Introduce a new FRMR operation of build_key that allows drivers to manipulate FRMR keys supplied by the user, allowing failing for unsupported properties and masking of properties that are modifiable. Signed-off-by: Michael Guralnik <michaelgur@nvidia.com> Reviewed-by: Yishai Hadas <yishaih@nvidia.com> Signed-off-by: Edward Srouji <edwards@nvidia.com> Link: https://patch.msgid.link/20260226-frmr_pools-v4-5-95360b54f15e@nvidia.com Signed-off-by: Leon Romanovsky <leon@kernel.org> 
Add a configuration of pinned handles on a specific FRMR pool.
The configured amount of pinned handles will not be aged and will stay
available for users to claim.

Upon setting the amount of pinned handles to an FRMR pool, we will make
sure we have at least the pinned amount of handles associated with the
pool and create more, if necessary.
The count for pinned handles take into account handles that are used by
user MRs and handles in the queue.

Introduce a new FRMR operation of build_key that allows drivers to
manipulate FRMR keys supplied by the user, allowing failing for
unsupported properties and masking of properties that are modifiable.

Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260226-frmr_pools-v4-5-95360b54f15e@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
IB/core: Introduce FRMR pools 2026-03-02T18:44:58+00:00 Michael Guralnik michaelgur@nvidia.com 2026-02-26T13:52:07+00:00 ce5df0b891edfa19620cd7e28bd69246c77ae78c Add a generic Fast Registration Memory Region pools mechanism to allow drivers to optimize memory registration performance. Drivers that have the ability to reuse MRs or their underlying HW objects can take advantage of the mechanism to keep a 'handle' for those objects and use them upon user request. We assume that to achieve this goal a driver and its HW should implement a modify operation for the MRs that is able to at least clear and set the MRs and in more advanced implementations also support changing a subset of the MRs properties. The mechanism is built using an RB-tree consisting of pools, each pool represents a set of MR properties that are shared by all of the MRs residing in the pool and are unmodifiable by the vendor driver or HW. The exposed API from ib_core to the driver has 4 operations: Init and cleanup - handles data structs and locks for the pools. Push and pop - store and retrieve 'handle' for a memory registration or deregistrations request. The FRMR pools mechanism implements the logic to search the RB-tree for a pool with matching properties and create a new one when needed and requires the driver to implement creation and destruction of a 'handle' when pool is empty or a handle is requested or is being destroyed. Later patch will introduce Netlink API to interact with the FRMR pools mechanism to allow users to both configure and track its usage. A vendor wishing to configure FRMR pool without exposing it or without exposing internal MR properties to users, should use the kernel_vendor_key field in the pools key. This can be useful in a few cases, e.g, when the FRMR handle has a vendor-specific un-modifiable property that the user registering the memory might not be aware of. Signed-off-by: Michael Guralnik <michaelgur@nvidia.com> Reviewed-by: Yishai Hadas <yishaih@nvidia.com> Signed-off-by: Edward Srouji <edwards@nvidia.com> Link: https://patch.msgid.link/20260226-frmr_pools-v4-2-95360b54f15e@nvidia.com Signed-off-by: Leon Romanovsky <leon@kernel.org> 
Add a generic Fast Registration Memory Region pools mechanism to allow
drivers to optimize memory registration performance.
Drivers that have the ability to reuse MRs or their underlying HW
objects can take advantage of the mechanism to keep a 'handle' for those
objects and use them upon user request.
We assume that to achieve this goal a driver and its HW should implement
a modify operation for the MRs that is able to at least clear and set the
MRs and in more advanced implementations also support changing a subset
of the MRs properties.

The mechanism is built using an RB-tree consisting of pools, each pool
represents a set of MR properties that are shared by all of the MRs
residing in the pool and are unmodifiable by the vendor driver or HW.

The exposed API from ib_core to the driver has 4 operations:
Init and cleanup - handles data structs and locks for the pools.
Push and pop - store and retrieve 'handle' for a memory registration
or deregistrations request.

The FRMR pools mechanism implements the logic to search the RB-tree for
a pool with matching properties and create a new one when needed and
requires the driver to implement creation and destruction of a 'handle'
when pool is empty or a handle is requested or is being destroyed.

Later patch will introduce Netlink API to interact with the FRMR pools
mechanism to allow users to both configure and track its usage.
A vendor wishing to configure FRMR pool without exposing it or without
exposing internal MR properties to users, should use the
kernel_vendor_key field in the pools key. This can be useful in a few
cases, e.g, when the FRMR handle has a vendor-specific un-modifiable
property that the user registering the memory might not be aware of.

Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260226-frmr_pools-v4-2-95360b54f15e@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>