linux-stable.git/kernel/dma/direct.c, branch linux-5.4.y

dma-mapping: drop the dev argument to arch_sync_dma_for_*

2023-04-05T09:16:43+00:00

[ Upstream commit 56e35f9c5b87ec1ae93e483284e189c84388de16 ]

These are pure cache maintainance routines, so drop the unused
struct device argument.

Signed-off-by: Christoph Hellwig 
Suggested-by: Daniel Vetter 
Stable-dep-of: ab327f8acdf8 ("mips: bmips: BCM6358: disable RAC flush for TP1")
Signed-off-by: Sasha Levin

dma-direct: avoid redundant memory sync for swiotlb

2022-04-20T07:19:39+00:00

commit 9e02977bfad006af328add9434c8bffa40e053bb upstream.

When we looked into FIO performance with swiotlb enabled in VM, we found
swiotlb_bounce() is always called one more time than expected for each DMA
read request.

It turns out that the bounce buffer is copied to original DMA buffer twice
after the completion of a DMA request (one is done by in
dma_direct_sync_single_for_cpu(), the other by swiotlb_tbl_unmap_single()).
But the content in bounce buffer actually doesn't change between the two
rounds of copy. So, one round of copy is redundant.

Pass DMA_ATTR_SKIP_CPU_SYNC flag to swiotlb_tbl_unmap_single() to
skip the memory copy in it.

This fix increases FIO 64KB sequential read throughput in a guest with
swiotlb=force by 5.6%.

Fixes: 55897af63091 ("dma-direct: merge swiotlb_dma_ops into the dma_direct code")
Reported-by: Wang Zhaoyang1 
Reported-by: Gao Liang 
Signed-off-by: Chao Gao 
Reviewed-by: Kevin Tian 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Greg Kroah-Hartman

dma-direct: fix data truncation in dma_direct_get_required_mask()

2020-04-29T14:33:00+00:00

[ Upstream commit cdcda0d1f8f4ab84efe7cd9921c98364398aefd7 ]

The upper 32-bit physical address gets truncated inadvertently
when dma_direct_get_required_mask() invokes phys_to_dma_direct().
This results in dma_addressing_limited() return incorrect value
when used in platforms with LPAE enabled.
Fix it here by explicitly type casting 'max_pfn' to phys_addr_t
in order to prevent overflow of intermediate value while evaluating
'(max_pfn - 1) << PAGE_SHIFT'.

Signed-off-by: Kishon Vijay Abraham I 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Sasha Levin

dma-direct: don't check swiotlb=force in dma_direct_map_resource

2020-01-26T09:01:08+00:00

[ Upstream commit 4268ac6ae5870af10a7417b22990d615f72f77e2 ]

When mapping resources we can't just use swiotlb ram for bounce
buffering.  Switch to a direct dma_capable check instead.

Fixes: cfced786969c ("dma-mapping: remove the default map_resource implementation")
Reported-by: Robin Murphy 
Signed-off-by: Christoph Hellwig 
Acked-by: Marek Szyprowski 
Tested-by: Marek Szyprowski 
Signed-off-by: Sasha Levin

Merge branches 'arm/omap', 'arm/exynos', 'arm/smmu', 'arm/mediatek', 'arm/qcom', 'arm/renesas', 'x86/amd', 'x86/vt-d' and 'core' into next

2019-09-11T10:39:19+00:00

swiotlb: Split size parameter to map/unmap APIs

2019-09-11T10:34:29+00:00

This splits the size parameter to swiotlb_tbl_map_single() and
swiotlb_tbl_unmap_single() into an alloc_size and a mapping_size
parameter, where the latter one is rounded up to the iommu page
size.

Suggested-by: Christoph Hellwig 
Signed-off-by: Lu Baolu 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Joerg Roedel

dma-direct: fix zone selection after an unaddressable CMA allocation

2019-08-20T22:14:10+00:00

The new dma_alloc_contiguous hides if we allocate CMA or regular
pages, and thus fails to retry a ZONE_NORMAL allocation if the CMA
allocation succeeds but isn't addressable.  That means we either fail
outright or dip into a small zone that might not succeed either.

Thanks to Hillf Danton for debugging this issue.

Fixes: b1d2dc009dec ("dma-contiguous: add dma_{alloc,free}_contiguous() helpers")
Reported-by: Tobias Klausmann 
Signed-off-by: Christoph Hellwig 
Tested-by: Tobias Klausmann

dma-direct: don't truncate dma_required_mask to bus addressing capabilities

2019-08-10T17:52:45+00:00

The dma required_mask needs to reflect the actual addressing capabilities
needed to handle the whole system RAM. When truncated down to the bus
addressing capabilities dma_addressing_limited() will incorrectly signal
no limitations for devices which are restricted by the bus_dma_mask.

Fixes: b4ebe6063204 (dma-direct: implement complete bus_dma_mask handling)
Signed-off-by: Lucas Stach 
Tested-by: Atish Patra 
Signed-off-by: Christoph Hellwig

dma-direct: fix DMA_ATTR_NO_KERNEL_MAPPING

2019-08-10T17:52:45+00:00

The new DMA_ATTR_NO_KERNEL_MAPPING needs to actually assign
a dma_addr to work.  Also skip it if the architecture needs
forced decryption handling, as that needs a kernel virtual
address.

Fixes: d98849aff879 (dma-direct: handle DMA_ATTR_NO_KERNEL_MAPPING in common code)
Signed-off-by: Christoph Hellwig 
Reviewed-by: Lucas Stach

dma-direct: correct the physical addr in dma_direct_sync_sg_for_cpu/device

2019-07-19T12:09:40+00:00

dma_map_sg() may use swiotlb buffer when the kernel command line includes
"swiotlb=force" or the dma_addr is out of dev->dma_mask range.  After
DMA complete the memory moving from device to memory, then user call
dma_sync_sg_for_cpu() to sync with DMA buffer, and copy the original
virtual buffer to other space.

So dma_direct_sync_sg_for_cpu() should use swiotlb physical addr, not
the original physical addr from sg_phys(sg).

dma_direct_sync_sg_for_device() also has the same issue, correct it as
well.

Fixes: 55897af63091("dma-direct: merge swiotlb_dma_ops into the dma_direct code")
Signed-off-by: Fugang Duan 
Reviewed-by: Robin Murphy 
Signed-off-by: Christoph Hellwig