linux-stable.git/kernel/dma, branch v5.10.78

dma-debug: fix sg checks in debug_dma_map_sg()

2021-10-27T07:56:47+00:00

[ Upstream commit 293d92cbbd2418ca2ba43fed07f1b92e884d1c77 ]

The following warning occurred sporadically on s390:
DMA-API: nvme 0006:00:00.0: device driver maps memory from kernel text or rodata [addr=0000000048cc5e2f] [len=131072]
WARNING: CPU: 4 PID: 825 at kernel/dma/debug.c:1083 check_for_illegal_area+0xa8/0x138

It is a false-positive warning, due to broken logic in debug_dma_map_sg().
check_for_illegal_area() checks for overlay of sg elements with kernel text
or rodata. It is called with sg_dma_len(s) instead of s->length as
parameter. After the call to ->map_sg(), sg_dma_len() will contain the
length of possibly combined sg elements in the DMA address space, and not
the individual sg element length, which would be s->length.

The check will then use the physical start address of an sg element, and
add the DMA length for the overlap check, which could result in the false
warning, because the DMA length can be larger than the actual single sg
element length.

In addition, the call to check_for_illegal_area() happens in the iteration
over mapped_ents, which will not include all individual sg elements if
any of them were combined in ->map_sg().

Fix this by using s->length instead of sg_dma_len(s). Also put the call to
check_for_illegal_area() in a separate loop, iterating over all the
individual sg elements ("nents" instead of "mapped_ents").

While at it, as suggested by Robin Murphy, also move check_for_stack()
inside the new loop, as it is similarly concerned with validating the
individual sg elements.

Link: https://lore.kernel.org/lkml/20210705185252.4074653-1-gerald.schaefer@linux.ibm.com
Fixes: 884d05970bfb ("dma-debug: use sg_dma_len accessor")
Signed-off-by: Gerald Schaefer 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Sasha Levin

dma-debug: fix debugfs initialization order

2021-09-18T11:40:13+00:00

[ Upstream commit 173735c346c412d9f084825ecb04f24ada0e2986 ]

Due to link order, dma_debug_init is called before debugfs has a chance
to initialize (via debugfs_init which also happens in the core initcall
stage), so the directories for dma-debug are never created.

Decouple dma_debug_fs_init from dma_debug_init and defer its init until
core_initcall_sync (after debugfs has been initialized) while letting
dma-debug initialization occur as soon as possible to catch any early
mappings, as suggested in [1].

[1] https://lore.kernel.org/linux-iommu/YIgGa6yF%2Fadg8OSN@kroah.com/

Fixes: 15b28bbcd567 ("dma-debug: move initialization to common code")
Signed-off-by: Anthony Iliopoulos 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Sasha Levin

dma-mapping: handle vmalloc addresses in dma_common_{mmap,get_sgtable}

2021-07-28T12:35:38+00:00

[ Upstream commit 40ac971eab89330d6153e7721e88acd2d98833f9 ]

xen-swiotlb can use vmalloc backed addresses for dma coherent allocations
and uses the common helpers.  Properly handle them to unbreak Xen on
ARM platforms.

Fixes: 1b65c4e5a9af ("swiotlb-xen: use xen_alloc/free_coherent_pages")
Signed-off-by: Roman Skakun 
Reviewed-by: Andrii Anisov 
[hch: split the patch, renamed the helpers]
Signed-off-by: Christoph Hellwig 
Signed-off-by: Sasha Levin

swiotlb: manipulate orig_addr when tlb_addr has offset

2021-06-30T12:47:29+00:00

commit 5f89468e2f060031cd89fd4287298e0eaf246bf6 upstream.

in case of driver wants to sync part of ranges with offset,
swiotlb_tbl_sync_single() copies from orig_addr base to tlb_addr with
offset and ends up with data mismatch.

It was removed from
"swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single",
but said logic has to be added back in.

From Linus's email:
"That commit which the removed the offset calculation entirely, because the old

        (unsigned long)tlb_addr & (IO_TLB_SIZE - 1)

was wrong, but instead of removing it, I think it should have just
fixed it to be

        (tlb_addr - mem->start) & (IO_TLB_SIZE - 1);

instead. That way the slot offset always matches the slot index calculation."

(Unfortunatly that broke NVMe).

The use-case that drivers are hitting is as follow:

1. Get dma_addr_t from dma_map_single()

dma_addr_t tlb_addr = dma_map_single(dev, vaddr, vsize, DMA_TO_DEVICE);

    |<---------------vsize------------->|
    +-----------------------------------+
    |                                   | original buffer
    +-----------------------------------+
  vaddr

 swiotlb_align_offset
     |<----->|<---------------vsize------------->|
     +-------+-----------------------------------+
     |       |                                   | swiotlb buffer
     +-------+-----------------------------------+
          tlb_addr

2. Do something
3. Sync dma_addr_t through dma_sync_single_for_device(..)

dma_sync_single_for_device(dev, tlb_addr + offset, size, DMA_TO_DEVICE);

  Error case.
    Copy data to original buffer but it is from base addr (instead of
  base addr + offset) in original buffer:

 swiotlb_align_offset
     |<----->|<- offset ->|<- size ->|
     +-------+-----------------------------------+
     |       |            |##########|           | swiotlb buffer
     +-------+-----------------------------------+
          tlb_addr

    |<- size ->|
    +-----------------------------------+
    |##########|                        | original buffer
    +-----------------------------------+
  vaddr

The fix is to copy the data to the original buffer and take into
account the offset, like so:

 swiotlb_align_offset
     |<----->|<- offset ->|<- size ->|
     +-------+-----------------------------------+
     |       |            |##########|           | swiotlb buffer
     +-------+-----------------------------------+
          tlb_addr

    |<- offset ->|<- size ->|
    +-----------------------------------+
    |            |##########|           | original buffer
    +-----------------------------------+
  vaddr

[One fix which was Linus's that made more sense to as it created a
symmetry would break NVMe. The reason for that is the:
 unsigned int offset = (tlb_addr - mem->start) & (IO_TLB_SIZE - 1);

would come up with the proper offset, but it would lose the
alignment (which this patch contains).]

Fixes: 16fc3cef33a0 ("swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single")
Signed-off-by: Bumyong Lee 
Signed-off-by: Chanho Park 
Reviewed-by: Christoph Hellwig 
Reported-by: Dominique MARTINET 
Reported-by: Horia Geantă 
Tested-by: Horia Geantă 
CC: stable@vger.kernel.org
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Greg Kroah-Hartman

swiotlb: Fix the type of index

2021-05-19T08:13:04+00:00

[ Upstream commit 95b079d8215b83b37fa59341fda92fcb9392f14a ]

Fix the type of index from unsigned int to int since find_slots() might
return -1.

Fixes: 26a7e094783d ("swiotlb: refactor swiotlb_tbl_map_single")
Reviewed-by: Christoph Hellwig 
Signed-off-by: Claire Chang 
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Sasha Levin

swiotlb: respect min_align_mask

2021-05-07T09:04:32+00:00

commit: 1f221a0d0dbf0e48ef3a9c62871281d6a7819f05

swiotlb: respect min_align_mask

Respect the min_align_mask in struct device_dma_parameters in swiotlb.

There are two parts to it:
 1) for the lower bits of the alignment inside the io tlb slot, just
    extent the size of the allocation and leave the start of the slot
     empty
 2) for the high bits ensure we find a slot that matches the high bits
    of the alignment to avoid wasting too much memory

Based on an earlier patch from Jianxiong Gao .

Signed-off-by: Christoph Hellwig 
Acked-by: Jianxiong Gao 
Tested-by: Jianxiong Gao 
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Jianxiong Gao 
Signed-off-by: Greg Kroah-Hartman

swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single

2021-05-07T09:04:32+00:00

commit: 16fc3cef33a04632ab6b31758abdd77563a20759

swiotlb_tbl_map_single currently nevers sets a tlb_addr that is not
aligned to the tlb bucket size.  But we're going to add such a case
soon, for which this adjustment would be bogus.

Signed-off-by: Christoph Hellwig 
Acked-by: Jianxiong Gao 
Tested-by: Jianxiong Gao 
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Jianxiong Gao 
Signed-off-by: Greg Kroah-Hartman

swiotlb: refactor swiotlb_tbl_map_single

2021-05-07T09:04:32+00:00

commit: 26a7e094783d482f3e125f09945a5bb1d867b2e6

Split out a bunch of a self-contained helpers to make the function easier
to follow.

Signed-off-by: Christoph Hellwig 
Acked-by: Jianxiong Gao 
Tested-by: Jianxiong Gao 
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Jianxiong Gao 
Signed-off-by: Greg Kroah-Hartman

swiotlb: clean up swiotlb_tbl_unmap_single

2021-05-07T09:04:32+00:00

commit: ca10d0f8e530600ec63c603dbace2c30927d70b7

swiotlb: clean up swiotlb_tbl_unmap_single

Remove a layer of pointless indentation, replace a hard to follow
ternary expression with a plain if/else.

Signed-off-by: Christoph Hellwig 
Acked-by: Jianxiong Gao 
Tested-by: Jianxiong Gao 
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Jianxiong Gao 
Signed-off-by: Greg Kroah-Hartman

swiotlb: factor out a nr_slots helper

2021-05-07T09:04:32+00:00

commit: c32a77fd18780a5192dfb6eec69f239faebf28fd

Factor out a helper to find the number of slots for a given size.

Signed-off-by: Christoph Hellwig 
Acked-by: Jianxiong Gao 
Tested-by: Jianxiong Gao 
Signed-off-by: Konrad Rzeszutek Wilk 
Signed-off-by: Jianxiong Gao 
Signed-off-by: Greg Kroah-Hartman