linux-stable.git/kernel/dma/direct.c, branch v6.18.37

dma-mapping: direct: fix missing mapping for THRU_HOST_BRIDGE segments

2026-06-19T11:43:57+00:00

[ Upstream commit 560000d619ef162568746ce287f0c725e24ea967 ]

In dma_direct_map_sg(), the case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE
incorrectly used 'break' instead of falling through to MAP_NONE.
As a result, segments traversing the host bridge skipped the required
dma_direct_map_phys() call entirely, leaving sg->dma_address
uninitialized and leading to DMA failures. Fix this by using
'fallthrough;'.

Fixes: a25e7962db0d79 ("PCI/P2PDMA: Refactor the p2pdma mapping helpers")
Reviewed-by: Logan Gunthorpe 
Signed-off-by: Li RongQing 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/20260603013723.2439-1-lirongqing@baidu.com
Signed-off-by: Sasha Levin

dma-direct: Fix missing sg_dma_len assignment in P2PDMA bus mappings

2025-11-26T20:47:13+00:00

Prior to commit a25e7962db0d7 ("PCI/P2PDMA: Refactor the p2pdma mapping
helpers"), P2P segments were mapped using the pci_p2pdma_map_segment()
helper. This helper was responsible for populating sg->dma_address,
marking the bus address, and also setting sg_dma_len(sg).

The refactor[1] removed this helper and moved the mapping logic directly
into the callers. While iommu_dma_map_sg() was correctly updated to set
the length in the new flow, it was missed in dma_direct_map_sg().

Thus, in dma_direct_map_sg(), the PCI_P2PDMA_MAP_BUS_ADDR case sets the
dma_address and marks the segment, but immediately executes 'continue',
which causes the loop to skip the standard assignment logic at the end:

    sg_dma_len(sg) = sg->length;

As a result, when CONFIG_NEED_SG_DMA_LENGTH is enabled, the dma_length
field remains uninitialized (zero) for P2P bus address mappings. This
breaks upper-layer drivers (for e.g. RDMA/IB) that rely on sg_dma_len()
to determine the transfer size.

Fix this by explicitly setting the DMA length in the
PCI_P2PDMA_MAP_BUS_ADDR case before continuing to the next scatterlist
entry.

Fixes: a25e7962db0d7 ("PCI/P2PDMA: Refactor the p2pdma mapping helpers")
Reported-by: Jacob Moroni 
Signed-off-by: Pranjal Shrivastava 

[1]
https://lore.kernel.org/all/ac14a0e94355bf898de65d023ccf8a2ad22a3ece.1746424934.git.leon@kernel.org/

Reviewed-by: Logan Gunthorpe 
Reviewed-by: Leon Romanovsky 
Reviewed-by: Shivaji Kant 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/20251126114112.3694469-1-praan@google.com

dma-mapping: export new dma_*map_phys() interface

2025-09-11T22:18:21+00:00

Introduce new DMA mapping functions dma_map_phys() and dma_unmap_phys()
that operate directly on physical addresses instead of page+offset
parameters. This provides a more efficient interface for drivers that
already have physical addresses available.

The new functions are implemented as the primary mapping layer, with
the existing dma_map_page_attrs()/dma_map_resource() and
dma_unmap_page_attrs()/dma_unmap_resource() functions converted to simple
wrappers around the phys-based implementations.

In case dma_map_page_attrs(), the struct page is converted to physical
address with help of page_to_phys() function and dma_map_resource()
provides physical address as is together with addition of DMA_ATTR_MMIO
attribute.

The old page-based API is preserved in mapping.c to ensure that existing
code won't be affected by changing EXPORT_SYMBOL to EXPORT_SYMBOL_GPL
variant for dma_*map_phys().

Reviewed-by: Jason Gunthorpe 
Reviewed-by: Keith Busch 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/54cc52af91777906bbe4a386113437ba0bcfba9c.1757423202.git.leonro@nvidia.com

dma-mapping: convert dma_direct_*map_page to be phys_addr_t based

2025-09-11T22:18:20+00:00

Convert the DMA direct mapping functions to accept physical addresses
directly instead of page+offset parameters. The functions were already
operating on physical addresses internally, so this change eliminates
the redundant page-to-physical conversion at the API boundary.

The functions dma_direct_map_page() and dma_direct_unmap_page() are
renamed to dma_direct_map_phys() and dma_direct_unmap_phys() respectively,
with their calling convention changed from (struct page *page,
unsigned long offset) to (phys_addr_t phys).

Architecture-specific functions arch_dma_map_page_direct() and
arch_dma_unmap_page_direct() are similarly renamed to
arch_dma_map_phys_direct() and arch_dma_unmap_phys_direct().

The is_pci_p2pdma_page() checks are replaced with DMA_ATTR_MMIO checks
to allow integration with dma_direct_map_resource and dma_direct_map_phys()
is extended to support MMIO path either.

Reviewed-by: Jason Gunthorpe 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/bb15a22f76dc2e26683333ff54e789606cfbfcf0.1757423202.git.leonro@nvidia.com

dma-direct: clean up the logic in __dma_direct_alloc_pages()

2025-08-11T09:28:04+00:00

Convert a goto-based loop to a while() loop. To allow the simplification,
return early when allocation from CMA is successful. As a bonus, this early
return avoids a repeated dma_coherent_ok() check.

No functional change.

Signed-off-by: Petr Tesarik 
Signed-off-by: Marek Szyprowski 
Link: https://lore.kernel.org/r/20250710083829.1853466-1-ptesarik@suse.com

dma-mapping: move the PCI P2PDMA mapping helpers to pci-p2pdma.h

2025-05-06T06:36:53+00:00

To support the upcoming non-scatterlist mapping helpers, we need to go
back to have them called outside of the DMA API.  Thus move them out of
dma-map-ops.h, which is only for DMA API implementations to pci-p2pdma.h,
which is for driver use.

Note that the core helper is still not exported as the mapping is
expected to be done only by very highlevel subsystem code at least for
now.

Signed-off-by: Christoph Hellwig 
Reviewed-by: Logan Gunthorpe 
Acked-by: Bjorn Helgaas 
Tested-by: Jens Axboe 
Reviewed-by: Luis Chamberlain 
Reviewed-by: Lu Baolu 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Marek Szyprowski

PCI/P2PDMA: Refactor the p2pdma mapping helpers

2025-05-06T06:36:53+00:00

The current scheme with a single helper to determine the P2P status
and map a scatterlist segment force users to always use the map_sg
helper to DMA map, which we're trying to get away from because they
are very cache inefficient.

Refactor the code so that there is a single helper that checks the P2P
state for a page, including the result that it is not a P2P page to
simplify the callers, and a second one to perform the address translation
for a bus mapped P2P transfer that does not depend on the scatterlist
structure.

Signed-off-by: Christoph Hellwig 
Reviewed-by: Logan Gunthorpe 
Acked-by: Bjorn Helgaas 
Tested-by: Jens Axboe 
Reviewed-by: Luis Chamberlain 
Reviewed-by: Lu Baolu 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Marek Szyprowski

dma-mapping: fix missing clear bdr in check_ram_in_range_map()

2025-03-12T12:41:44+00:00

As discussed in [1], if 'bdr' is set once, it would never get
cleared, hence 0 is always returned.

Refactor the range check hunk into a new helper dma_find_range(),
which allows 'bdr' to be cleared in each iteration.

Link: https://lore.kernel.org/all/64931fac-085b-4ff3-9314-84bac2fa9bdb@quicinc.com/ # [1]
Fixes: a409d9600959 ("dma-mapping: fix dma_addressing_limited() if dma_range_map can't cover all system RAM")
Suggested-by: Christoph Hellwig 
Signed-off-by: Baochen Qiang 
Link: https://lore.kernel.org/r/20250307030350.69144-1-quic_bqiang@quicinc.com
Signed-off-by: Marek Szyprowski

dma-direct: optimize page freeing when it is not addressable

2024-09-04T04:08:51+00:00

When the CMA allocation succeeds but isn't addressable, its buffer has
already been released and the page is set to NULL.  So later when the
normal page allocation succeeds but isn't addressable, __free_pages()
can be used to free that normal page rather than using
dma_free_contiguous that does extra checks that are not needed.

Signed-off-by: Chen Yu 
Signed-off-by: Christoph Hellwig

dma-mapping: replace zone_dma_bits by zone_dma_limit

2024-08-22T04:18:00+00:00

The hardware DMA limit might not be power of 2. When RAM range starts
above 0, say 4GB, DMA limit of 30 bits should end at 5GB.  A single high
bit can not encode this limit.

Use a plain  address for the DMA zone limit instead.

Since the DMA zone can now potentially span beyond 4GB physical limit of
DMA32, make sure to use DMA zone for GFP_DMA32 allocations in that case.

Signed-off-by: Catalin Marinas 
Co-developed-by: Baruch Siach 
Signed-off-by: Baruch Siach 
Reviewed-by: Catalin Marinas 
Reviewed-by: Petr Tesarik 
Signed-off-by: Christoph Hellwig