linux.git/kernel/dma/pool.c, branch v7.3-rc2

dma-pool: track decrypted atomic pools and select them via attrs

2026-07-31T06:40:58+00:00

Teach the atomic DMA pool code to distinguish between encrypted and
unencrypted pools, and make pool allocation select the matching pool based
on DMA attributes.

Introduce a dma_gen_pool wrapper that records whether a pool is
unencrypted, initialize that state when the atomic pools are created, and
use it when expanding and resizing the pools. Update dma_alloc_from_pool()
to take attrs and skip pools whose encrypted state does not match
__DMA_ATTR_ALLOC_CC_SHARED. Update dma_free_from_pool() accordingly.

Also pass __DMA_ATTR_ALLOC_CC_SHARED from the swiotlb atomic allocation
path so decrypted swiotlb allocations are taken from the correct atomic
pool.

Tested-by: Jiri Pirko 
Tested-by: Michael Kelley 
Tested-by: Mostafa Saleh 
Reviewed-by: Mostafa Saleh 
Signed-off-by: Aneesh Kumar K.V (Arm) 
Link: https://lore.kernel.org/r/20260717180442.110954-12-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski

dma: free atomic pool pages by physical address

2026-07-31T06:40:58+00:00

dma_direct_alloc_pages() may satisfy atomic allocations from the coherent
atomic pools. The pool allocation is keyed by the virtual address stored in
the gen_pool, but the pages API returns only the backing struct page.

On architectures with CONFIG_DMA_DIRECT_REMAP, atomic pool chunks are added
to the gen_pool using their remapped virtual address.
dma_direct_free_pages() reconstructs a linear-map address with
page_address(page) and passes that to dma_free_from_pool(). That address
does not match the gen_pool virtual range, so the pool lookup can fail and
the code can fall through to freeing a pool-owned page through the normal
page allocator path.

Add a page-based pool free helper that looks up the owning pool chunk by
physical address, translates it back to the gen_pool virtual address, and
frees that address to the pool. Use it from dma_direct_free_pages() while
keeping the existing virtual-address helper for coherent allocation frees.

Tested-by: Michael Kelley 
Tested-by: Mostafa Saleh 
Signed-off-by: Aneesh Kumar K.V (Arm) 
Link: https://lore.kernel.org/r/20260717180442.110954-5-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski

dma-pool: fix page leak in atomic_pool_expand() cleanup

2026-07-31T06:40:58+00:00

atomic_pool_expand() frees the allocated pages from the remove_mapping
error path only when CONFIG_DMA_DIRECT_REMAP is enabled.

When CONFIG_DMA_DIRECT_REMAP is disabled, failures after page allocation,
such as gen_pool_add_virt(), jump to remove_mapping and return without
freeing the pages.

Move __free_pages(page, order) out of the CONFIG_DMA_DIRECT_REMAP block so
that cleanup paths always release the allocation.

Reviewed-by: Jason Gunthorpe 
Tested-by: Michael Kelley 
Tested-by: Mostafa Saleh 
Signed-off-by: Aneesh Kumar K.V (Arm) 
Link: https://lore.kernel.org/r/20260717180442.110954-3-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski

dma/pool: distinguish between missing and exhausted atomic pools

2026-01-29T09:23:45+00:00

Currently, dma_alloc_from_pool() unconditionally warns and dumps a stack
trace when an allocation fails, with the message "Failed to get suitable
pool".

This conflates two distinct failure modes:
1. Configuration error: No atomic pool is available for the requested
   DMA mask (a fundamental system setup issue)
2. Resource Exhaustion: A suitable pool exists but is currently full (a
   recoverable runtime state)

This lack of distinction prevents drivers from using __GFP_NOWARN to
suppress error messages during temporary pressure spikes, such as when
awaiting synchronous reclaim of descriptors.

Refactor the error handling to distinguish these cases:
- If no suitable pool is found, keep the unconditional WARN regarding
  the missing pool.
- If a pool was found but is exhausted, respect __GFP_NOWARN and update
  the warning message to explicitly state "DMA pool exhausted".

Fixes: 9420139f516d ("dma-pool: fix coherent pool allocations for IOMMU mappings")
Signed-off-by: Sai Sree Kartheek Adivi 
Reviewed-by: Robin Murphy 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/20260128133554.3056582-1-s-adivi@ti.com

dma/pool: Avoid allocating redundant pools

2026-01-14T10:00:00+00:00

On smaller systems, e.g. embedded arm64, it is common for all memory
to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant
to allocate a nominal pool for an empty higher zone that just ends up
coming from a lower zone that should already have its own pool anyway.
We already have logic to skip allocating a ZONE_DMA pool when that is
empty, so generalise that to save memory in the case of other zones too.

Signed-off-by: Robin Murphy 
Tested-by: Vladimir Kondratiev 
Reviewed-by: Baoquan He 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/8ab8d8a620dee0109f33f5cb63d6bfeed35aac37.1768230104.git.robin.murphy@arm.com

dma/pool: Improve pool lookup

2026-01-14T10:00:00+00:00

If CONFIG_ZONE_DMA32 is enabled, but we have not allocated the
corresponding atomic_pool_dma32, dma_guess_pool() may return the NULL
value of that and fail a GFP_DMA32 allocation without trying to fall
back to other pools which may exist. Furthermore, if no GFP_DMA pool
exists, it is preferable to try GFP_DMA32 rather than immediately fall
back to GFP_KERNEL with even less chance of success. Improve matters
by encoding an explicit order of pool preference for each flag.

Signed-off-by: Robin Murphy 
Tested-by: Vladimir Kondratiev 
Reviewed-by: Baoquan He 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/c846b1a2f43295cac926c7af2ce907f62baec518.1768230104.git.robin.murphy@arm.com

dma/pool: eliminate alloc_pages warning in atomic_pool_expand

2025-12-08T08:40:57+00:00

atomic_pool_expand iteratively tries the allocation while decrementing
the page order. There is no need to issue a warning if an attempted
allocation fails.

Signed-off-by: Dave Kleikamp 
Reviewed-by: Robin Murphy 
Fixes: d7e673ec2c8e ("dma-pool: Only allocate from CMA when in same memory zone")
[mszyprow: fixed typo]
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/20251202152810.142370-1-dave.kleikamp@oracle.com

dma/pool: Ensure DMA_DIRECT_REMAP allocations are decrypted

2025-08-13T09:02:10+00:00

When CONFIG_DMA_DIRECT_REMAP is enabled, atomic pool pages are
remapped via dma_common_contiguous_remap() using the supplied
pgprot. Currently, the mapping uses
pgprot_dmacoherent(PAGE_KERNEL), which leaves the memory encrypted
on systems with memory encryption enabled (e.g., ARM CCA Realms).

This can cause the DMA layer to fail or crash when accessing the
memory, as the underlying physical pages are not configured as
expected.

Fix this by requesting a decrypted mapping in the vmap() call:
pgprot_decrypted(pgprot_dmacoherent(PAGE_KERNEL))

This ensures that atomic pool memory is consistently mapped
unencrypted.

Cc: stable@vger.kernel.org
Signed-off-by: Shanker Donthineni 
Reviewed-by: Catalin Marinas 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/20250811181759.998805-1-sdonthineni@nvidia.com

dma-mapping: replace zone_dma_bits by zone_dma_limit

2024-08-22T04:18:00+00:00

The hardware DMA limit might not be power of 2. When RAM range starts
above 0, say 4GB, DMA limit of 30 bits should end at 5GB.  A single high
bit can not encode this limit.

Use a plain  address for the DMA zone limit instead.

Since the DMA zone can now potentially span beyond 4GB physical limit of
DMA32, make sure to use DMA zone for GFP_DMA32 allocations in that case.

Signed-off-by: Catalin Marinas 
Co-developed-by: Baruch Siach 
Signed-off-by: Baruch Siach 
Reviewed-by: Catalin Marinas 
Reviewed-by: Petr Tesarik 
Signed-off-by: Christoph Hellwig

mm, treewide: rename MAX_ORDER to MAX_PAGE_ORDER

2024-01-08T23:27:15+00:00

commit 23baf831a32c ("mm, treewide: redefine MAX_ORDER sanely") has
changed the definition of MAX_ORDER to be inclusive.  This has caused
issues with code that was not yet upstream and depended on the previous
definition.

To draw attention to the altered meaning of the define, rename MAX_ORDER
to MAX_PAGE_ORDER.

Link: https://lkml.kernel.org/r/20231228144704.14033-2-kirill.shutemov@linux.intel.com
Signed-off-by: Kirill A. Shutemov 
Cc: Linus Torvalds 
Signed-off-by: Andrew Morton