linux-stable.git/drivers/dma, branch v5.15.78 Linux kernel stable tree dmaengine: ti: k3-udma: Reset UDMA_CHAN_RT byte counters to prevent overflow 2022-10-26T10:35:46+00:00 Vaishnav Achath vaishnav.a@ti.com 2022-08-02T05:48:35+00:00 d68da10b0cceb4177b653833e794b2923a4ffbd7 [ Upstream commit 7c94dcfa8fcff2dba53915f1dabfee49a3df8b88 ] UDMA_CHAN_RT_*BCNT_REG stores the real-time channel bytecount statistics. These registers are 32-bit hardware counters and the driver uses these counters to monitor the operational progress status for a channel, when transferring more than 4GB of data it was observed that these counters overflow and completion calculation of a operation gets affected and the transfer hangs indefinitely. This commit adds changes to decrease the byte count for every complete transaction so that these registers never overflow and the proper byte count statistics is maintained for ongoing transaction by the RT counters. Earlier uc->bcnt used to maintain a count of the completed bytes at driver side, since the RT counters maintain the statistics of current transaction now, the maintenance of uc->bcnt is not necessary. Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com> Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com> Link: https://lore.kernel.org/r/20220802054835.19482-1-vaishnav.a@ti.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 7c94dcfa8fcff2dba53915f1dabfee49a3df8b88 ]

UDMA_CHAN_RT_*BCNT_REG stores the real-time channel bytecount statistics.
These registers are 32-bit hardware counters and the driver uses these
counters to monitor the operational progress status for a channel, when
transferring more than 4GB of data it was observed that these counters
overflow and completion calculation of a operation gets affected and the
transfer hangs indefinitely.

This commit adds changes to decrease the byte count for every complete
transaction so that these registers never overflow and the proper byte
count statistics is maintained for ongoing transaction by the RT counters.

Earlier uc->bcnt used to maintain a count of the completed bytes at driver
side, since the RT counters maintain the statistics of current transaction
now, the maintenance of uc->bcnt is not necessary.

Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220802054835.19482-1-vaishnav.a@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: ioat: stop mod_timer from resurrecting deleted timer in __cleanup() 2022-10-26T10:35:18+00:00 Dave Jiang dave.jiang@intel.com 2022-09-19T16:58:42+00:00 57f66534a41a75f0ab12250030827777cb38f8aa [ Upstream commit 898ec89dbb55b8294695ad71694a0684e62b2a73 ] User reports observing timer event report channel halted but no error observed in CHANERR register. The driver finished self-test and released channel resources. Debug shows that __cleanup() can call mod_timer() after the timer has been deleted and thus resurrect the timer. While harmless, it causes suprious error message to be emitted. Use mod_timer_pending() call to prevent deleted timer from being resurrected. Fixes: 3372de5813e4 ("dmaengine: ioatdma: removal of dma_v3.c and relevant ioat3 references") Signed-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/166360672197.3851724.17040290563764838369.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 898ec89dbb55b8294695ad71694a0684e62b2a73 ]

User reports observing timer event report channel halted but no error
observed in CHANERR register. The driver finished self-test and released
channel resources. Debug shows that __cleanup() can call
mod_timer() after the timer has been deleted and thus resurrect the
timer. While harmless, it causes suprious error message to be emitted.
Use mod_timer_pending() call to prevent deleted timer from being
resurrected.

Fixes: 3372de5813e4 ("dmaengine: ioatdma: removal of dma_v3.c and relevant ioat3 references")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/166360672197.3851724.17040290563764838369.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: hisilicon: Add multi-thread support for a DMA channel 2022-10-26T10:35:08+00:00 Jie Hai haijie1@huawei.com 2022-08-30T06:22:47+00:00 7cb9b20941e1fb20d22d0a2f460a3d4fa417274c [ Upstream commit 2cbb95883c990d0002a77e13d3278913ab26ad79 ] When we get a DMA channel and try to use it in multiple threads it will cause oops and hanging the system. % echo 100 > /sys/module/dmatest/parameters/threads_per_chan % echo 100 > /sys/module/dmatest/parameters/iterations % echo 1 > /sys/module/dmatest/parameters/run [383493.327077] Unable to handle kernel paging request at virtual address dead000000000108 [383493.335103] Mem abort info: [383493.335103] ESR = 0x96000044 [383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits [383493.335107] SET = 0, FnV = 0 [383493.335108] EA = 0, S1PTW = 0 [383493.335109] FSC = 0x04: level 0 translation fault [383493.335110] Data abort info: [383493.335111] ISV = 0, ISS = 0x00000044 [383493.364739] CM = 0, WnR = 1 [383493.367793] [dead000000000108] address between user and kernel address ranges [383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP [383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump: loaded Tainted: GO 5.17.0-rc4+ #2 [383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [383493.465331] pc : vchan_tx_submit+0x64/0xa0 [383493.469957] lr : vchan_tx_submit+0x34/0xa0 This occurs because the transmission timed out, and that's due to data race. Each thread rewrite channels's descriptor as soon as device_issue_pending is called. It leads to the situation that the driver thinks that it uses the right descriptor in interrupt handler while channels's descriptor has been changed by other thread. The descriptor which in fact reported interrupt will not be handled any more, as well as its tx->callback. That's why timeout reports. With current fixes channels' descriptor changes it's value only when it has been used. A new descriptor is acquired from vc->desc_issued queue that is already filled with descriptors that are ready to be sent. Threads have no direct access to DMA channel descriptor. In case of channel's descriptor is busy, try to submit to HW again when a descriptor is completed. In this case, vc->desc_issued may be empty when hisi_dma_start_transfer is called, so delete error reporting on this. Now it is just possible to queue a descriptor for further processing. Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support") Signed-off-by: Jie Hai <haijie1@huawei.com> Acked-by: Zhou Wang <wangzhou1@hisilicon.com> Link: https://lore.kernel.org/r/20220830062251.52993-4-haijie1@huawei.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 2cbb95883c990d0002a77e13d3278913ab26ad79 ]

When we get a DMA channel and try to use it in multiple threads it
will cause oops and hanging the system.

% echo 100 > /sys/module/dmatest/parameters/threads_per_chan
% echo 100 > /sys/module/dmatest/parameters/iterations
% echo 1 > /sys/module/dmatest/parameters/run
[383493.327077] Unable to handle kernel paging request at virtual
		address dead000000000108
[383493.335103] Mem abort info:
[383493.335103]   ESR = 0x96000044
[383493.335105]   EC = 0x25: DABT (current EL), IL = 32 bits
[383493.335107]   SET = 0, FnV = 0
[383493.335108]   EA = 0, S1PTW = 0
[383493.335109]   FSC = 0x04: level 0 translation fault
[383493.335110] Data abort info:
[383493.335111]   ISV = 0, ISS = 0x00000044
[383493.364739]   CM = 0, WnR = 1
[383493.367793] [dead000000000108] address between user and kernel
		address ranges
[383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
		loaded Tainted: GO 5.17.0-rc4+ #2
[383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
		-SSBS BTYPE=--)
[383493.465331] pc : vchan_tx_submit+0x64/0xa0
[383493.469957] lr : vchan_tx_submit+0x34/0xa0

This occurs because the transmission timed out, and that's due
to data race. Each thread rewrite channels's descriptor as soon as
device_issue_pending is called. It leads to the situation that
the driver thinks that it uses the right descriptor in interrupt
handler while channels's descriptor has been changed by other
thread. The descriptor which in fact reported interrupt will not
be handled any more, as well as its tx->callback.
That's why timeout reports.

With current fixes channels' descriptor changes it's value only
when it has been used. A new descriptor is acquired from
vc->desc_issued queue that is already filled with descriptors
that are ready to be sent. Threads have no direct access to DMA
channel descriptor. In case of channel's descriptor is busy, try
to submit to HW again when a descriptor is completed. In this case,
vc->desc_issued may be empty when hisi_dma_start_transfer is called,
so delete error reporting on this. Now it is just possible to queue
a descriptor for further processing.

Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Link: https://lore.kernel.org/r/20220830062251.52993-4-haijie1@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: hisilicon: Fix CQ head update 2022-10-26T10:35:07+00:00 Jie Hai haijie1@huawei.com 2022-08-30T06:22:46+00:00 b88630d9aac0a748a53041533dcb5dcb455127e7 [ Upstream commit 94477a79cf80e8ab55b68f14bc579a12ddea1e0b ] After completion of data transfer of one or multiple descriptors, the completion status and the current head pointer to submission queue are written into the CQ and interrupt can be generated to inform the software. In interrupt process CQ is read and cq_head is updated. hisi_dma_irq updates cq_head only when the completion status is success. When an abnormal interrupt reports, cq_head will not update which will cause subsequent interrupt processes read the error CQ and never report the correct status. This patch updates cq_head whenever CQ is accessed. Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support") Signed-off-by: Jie Hai <haijie1@huawei.com> Acked-by: Zhou Wang <wangzhou1@hisilicon.com> Link: https://lore.kernel.org/r/20220830062251.52993-3-haijie1@huawei.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 94477a79cf80e8ab55b68f14bc579a12ddea1e0b ]

After completion of data transfer of one or multiple descriptors,
the completion status and the current head pointer to submission
queue are written into the CQ and interrupt can be generated to
inform the software. In interrupt process CQ is read and cq_head
is updated.

hisi_dma_irq updates cq_head only when the completion status is
success. When an abnormal interrupt reports, cq_head will not update
which will cause subsequent interrupt processes read the error CQ
and never report the correct status.

This patch updates cq_head whenever CQ is accessed.

Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Link: https://lore.kernel.org/r/20220830062251.52993-3-haijie1@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: hisilicon: Disable channels when unregister hisi_dma 2022-10-26T10:35:07+00:00 Jie Hai haijie1@huawei.com 2022-08-30T06:22:45+00:00 e84aeeafe8b37ca65b98ab8a9ee5797fa1c03f51 [ Upstream commit e3bdaa04ada31f46d0586df83a2789b8913053c5 ] When hisi_dma is unloaded or unbinded, all of channels should be disabled. This patch disables DMA channels when driver is unloaded or unbinded. Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support") Signed-off-by: Jie Hai <haijie1@huawei.com> Acked-by: Zhou Wang <wangzhou1@hisilicon.com> Link: https://lore.kernel.org/r/20220830062251.52993-2-haijie1@huawei.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit e3bdaa04ada31f46d0586df83a2789b8913053c5 ]

When hisi_dma is unloaded or unbinded, all of channels should be
disabled. This patch disables DMA channels when driver is unloaded
or unbinded.

Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Link: https://lore.kernel.org/r/20220830062251.52993-2-haijie1@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: mxs: use platform_driver_register 2022-10-26T10:34:23+00:00 Dario Binacchi dario.binacchi@amarulasolutions.com 2022-09-21T17:05:56+00:00 a95fb5d55af0bf7efcfe6818a5fef37d21d86d9d commit 26696d4657167112a1079f86cba1739765c1360e upstream. Driver registration fails on SOC imx8mn as its supplier, the clock control module, is probed later than subsys initcall level. This driver uses platform_driver_probe which is not compatible with deferred probing and won't be probed again later if probe function fails due to clock not being available at that time. This patch replaces the use of platform_driver_probe with platform_driver_register which will allow probing the driver later again when the clock control module will be available. The __init annotation has been dropped because it is not compatible with deferred probing. The code is not executed once and its memory cannot be freed. Fixes: a580b8c5429a ("dmaengine: mxs-dma: add dma support for i.MX23/28") Co-developed-by: Michael Trimarchi <michael@amarulasolutions.com> Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com> Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com> Acked-by: Sascha Hauer <s.hauer@pengutronix.de> Cc: stable@vger.kernel.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Link: https://lore.kernel.org/r/20220921170556.1055962-1-dario.binacchi@amarulasolutions.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
commit 26696d4657167112a1079f86cba1739765c1360e upstream.

Driver registration fails on SOC imx8mn as its supplier, the clock
control module, is probed later than subsys initcall level. This driver
uses platform_driver_probe which is not compatible with deferred probing
and won't be probed again later if probe function fails due to clock not
being available at that time.

This patch replaces the use of platform_driver_probe with
platform_driver_register which will allow probing the driver later again
when the clock control module will be available.

The __init annotation has been dropped because it is not compatible with
deferred probing. The code is not executed once and its memory cannot be
freed.

Fixes: a580b8c5429a ("dmaengine: mxs-dma: add dma support for i.MX23/28")
Co-developed-by: Michael Trimarchi <michael@amarulasolutions.com>
Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Cc: stable@vger.kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Link: https://lore.kernel.org/r/20220921170556.1055962-1-dario.binacchi@amarulasolutions.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine: xilinx_dma: Report error in case of dma_set_mask_and_coherent API failure 2022-10-12T07:53:27+00:00 Swati Agarwal swati.agarwal@xilinx.com 2022-08-17T06:11:25+00:00 968299cd58b75bc991401d3fca7f066380cfd6a9 [ Upstream commit 8f2b6bc79c32f0fa60df000ae387a790ec80eae9 ] The driver does not handle the failure case while calling dma_set_mask_and_coherent API. In case of failure, capture the return value of API and then report an error. Addresses-coverity: Unchecked return value (CHECKED_RETURN) Signed-off-by: Swati Agarwal <swati.agarwal@xilinx.com> Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com> Link: https://lore.kernel.org/r/20220817061125.4720-4-swati.agarwal@xilinx.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 8f2b6bc79c32f0fa60df000ae387a790ec80eae9 ]

The driver does not handle the failure case while calling
dma_set_mask_and_coherent API.

In case of failure, capture the return value of API and then report an
error.

Addresses-coverity: Unchecked return value (CHECKED_RETURN)

Signed-off-by: Swati Agarwal <swati.agarwal@xilinx.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/20220817061125.4720-4-swati.agarwal@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: xilinx_dma: cleanup for fetching xlnx,num-fstores property 2022-10-12T07:53:27+00:00 Swati Agarwal swati.agarwal@xilinx.com 2022-08-17T06:11:24+00:00 17f55255af4c7c3783cff65ae97dd35d7bd18ad7 [ Upstream commit 462bce790e6a7e68620a4ce260cc38f7ed0255d5 ] Free the allocated resources for missing xlnx,num-fstores property. Signed-off-by: Swati Agarwal <swati.agarwal@xilinx.com> Link: https://lore.kernel.org/r/20220817061125.4720-3-swati.agarwal@xilinx.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 462bce790e6a7e68620a4ce260cc38f7ed0255d5 ]

Free the allocated resources for missing xlnx,num-fstores property.

Signed-off-by: Swati Agarwal <swati.agarwal@xilinx.com>
Link: https://lore.kernel.org/r/20220817061125.4720-3-swati.agarwal@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: xilinx_dma: Fix devm_platform_ioremap_resource error handling 2022-10-12T07:53:27+00:00 Swati Agarwal swati.agarwal@xilinx.com 2022-08-17T06:11:23+00:00 b2f2755501366ece756241721f7487cf7e37ac35 [ Upstream commit 91df7751eb890e970afc08f50b8f0fa5ea39e03d ] Add missing cleanup in devm_platform_ioremap_resource(). When probe fails remove dma channel resources and disable clocks in accordance with the order of resources allocated . Signed-off-by: Swati Agarwal <swati.agarwal@xilinx.com> Link: https://lore.kernel.org/r/20220817061125.4720-2-swati.agarwal@xilinx.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 91df7751eb890e970afc08f50b8f0fa5ea39e03d ]

Add missing cleanup in devm_platform_ioremap_resource().
When probe fails remove dma channel resources and disable clocks in
accordance with the order of resources allocated .

Signed-off-by: Swati Agarwal <swati.agarwal@xilinx.com>
Link: https://lore.kernel.org/r/20220817061125.4720-2-swati.agarwal@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: ti: k3-udma-private: Fix refcount leak bug in of_xudma_dev_get() 2022-09-28T09:11:46+00:00 Liang He windhl@126.com 2022-07-20T07:32:34+00:00 dd5a6c5a08752b613e83ad2cb5133e72a64b876d [ Upstream commit f9fdb0b86f087c2b7f6c6168dd0985a3c1eda87e ] We should call of_node_put() for the reference returned by of_parse_phandle() in fail path or when it is not used anymore. Here we only need to move the of_node_put() before the check. Fixes: d70241913413 ("dmaengine: ti: k3-udma: Add glue layer for non DMAengine users") Signed-off-by: Liang He <windhl@126.com> Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com> Link: https://lore.kernel.org/r/20220720073234.1255474-1-windhl@126.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit f9fdb0b86f087c2b7f6c6168dd0985a3c1eda87e ]

We should call of_node_put() for the reference returned by
of_parse_phandle() in fail path or when it is not used anymore.
Here we only need to move the of_node_put() before the check.

Fixes: d70241913413 ("dmaengine: ti: k3-udma: Add glue layer for non DMAengine users")
Signed-off-by: Liang He <windhl@126.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220720073234.1255474-1-windhl@126.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>