linux-stable.git/drivers/dma, branch v5.4.78 Linux kernel stable tree dmaengine: dma-jz4780: Fix race in jz4780_dma_tx_status 2020-11-05T10:43:29+00:00 Paul Cercueil paul@crapouillou.net 2020-10-04T14:03:07+00:00 a4f02a81c7e6ce74be47ff7ba45b60c39e82eb5b commit baf6fd97b16ea8f981b8a8b04039596f32fc2972 upstream. The jz4780_dma_tx_status() function would check if a channel's cookie state was set to 'completed', and if not, it would enter the critical section. However, in that time frame, the jz4780_dma_chan_irq() function was able to set the cookie to 'completed', and clear the jzchan->vchan pointer, which was deferenced in the critical section of the first function. Fix this race by checking the channel's cookie state after entering the critical function and not before. Fixes: d894fc6046fe ("dmaengine: jz4780: add driver for the Ingenic JZ4780 DMA controller") Cc: stable@vger.kernel.org # v4.0 Signed-off-by: Paul Cercueil <paul@crapouillou.net> Reported-by: Artur Rojek <contact@artur-rojek.eu> Tested-by: Artur Rojek <contact@artur-rojek.eu> Link: https://lore.kernel.org/r/20201004140307.885556-1-paul@crapouillou.net Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit baf6fd97b16ea8f981b8a8b04039596f32fc2972 upstream.

The jz4780_dma_tx_status() function would check if a channel's cookie
state was set to 'completed', and if not, it would enter the critical
section. However, in that time frame, the jz4780_dma_chan_irq() function
was able to set the cookie to 'completed', and clear the jzchan->vchan
pointer, which was deferenced in the critical section of the first
function.

Fix this race by checking the channel's cookie state after entering the
critical function and not before.

Fixes: d894fc6046fe ("dmaengine: jz4780: add driver for the Ingenic JZ4780 DMA controller")
Cc: stable@vger.kernel.org # v4.0
Signed-off-by: Paul Cercueil <paul@crapouillou.net>
Reported-by: Artur Rojek <contact@artur-rojek.eu>
Tested-by: Artur Rojek <contact@artur-rojek.eu>
Link: https://lore.kernel.org/r/20201004140307.885556-1-paul@crapouillou.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
dmaengine: dw: Activate FIFO-mode for memory peripherals only 2020-10-29T08:58:09+00:00 Serge Semin Sergey.Semin@baikalelectronics.ru 2020-07-31T20:08:23+00:00 18ec92b1ce2975f8d27af868084c70835b86ace4 [ Upstream commit 6d9459d04081c796fc67c2bb771f4e4ebb5744c4 ] CFGx.FIFO_MODE field controls a DMA-controller "FIFO readiness" criterion. In other words it determines when to start pushing data out of a DW DMAC channel FIFO to a destination peripheral or from a source peripheral to the DW DMAC channel FIFO. Currently FIFO-mode is set to one for all DW DMAC channels. It means they are tuned to flush data out of FIFO (to a memory peripheral or by accepting the burst transaction requests) when FIFO is at least half-full (except at the end of the block transfer, when FIFO-flush mode is activated) and are configured to get data to the FIFO when it's at least half-empty. Such configuration is a good choice when there is no slave device involved in the DMA transfers. In that case the number of bursts per block is less than when CFGx.FIFO_MODE = 0 and, hence, the bus utilization will improve. But the latency of DMA transfers may increase when CFGx.FIFO_MODE = 1, since DW DMAC will wait for the channel FIFO contents to be either half-full or half-empty depending on having the destination or the source transfers. Such latencies might be dangerous in case if the DMA transfers are expected to be performed from/to a slave device. Since normally peripheral devices keep data in internal FIFOs, any latency at some critical moment may cause one being overflown and consequently losing data. This especially concerns a case when either a peripheral device is relatively fast or the DW DMAC engine is relatively slow with respect to the incoming data pace. In order to solve problems, which might be caused by the latencies described above, let's enable the FIFO half-full/half-empty "FIFO readiness" criterion only for DMA transfers with no slave device involved. Thanks to the commit 99ba8b9b0d97 ("dmaengine: dw: Initialize channel before each transfer") we can freely do that in the generic dw_dma_initialize_chan() method. Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Link: https://lore.kernel.org/r/20200731200826.9292-3-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 6d9459d04081c796fc67c2bb771f4e4ebb5744c4 ]

CFGx.FIFO_MODE field controls a DMA-controller "FIFO readiness" criterion.
In other words it determines when to start pushing data out of a DW
DMAC channel FIFO to a destination peripheral or from a source
peripheral to the DW DMAC channel FIFO. Currently FIFO-mode is set to one
for all DW DMAC channels. It means they are tuned to flush data out of
FIFO (to a memory peripheral or by accepting the burst transaction
requests) when FIFO is at least half-full (except at the end of the block
transfer, when FIFO-flush mode is activated) and are configured to get
data to the FIFO when it's at least half-empty.

Such configuration is a good choice when there is no slave device involved
in the DMA transfers. In that case the number of bursts per block is less
than when CFGx.FIFO_MODE = 0 and, hence, the bus utilization will improve.
But the latency of DMA transfers may increase when CFGx.FIFO_MODE = 1,
since DW DMAC will wait for the channel FIFO contents to be either
half-full or half-empty depending on having the destination or the source
transfers. Such latencies might be dangerous in case if the DMA transfers
are expected to be performed from/to a slave device. Since normally
peripheral devices keep data in internal FIFOs, any latency at some
critical moment may cause one being overflown and consequently losing
data. This especially concerns a case when either a peripheral device is
relatively fast or the DW DMAC engine is relatively slow with respect to
the incoming data pace.

In order to solve problems, which might be caused by the latencies
described above, let's enable the FIFO half-full/half-empty "FIFO
readiness" criterion only for DMA transfers with no slave device involved.
Thanks to the commit 99ba8b9b0d97 ("dmaengine: dw: Initialize channel
before each transfer") we can freely do that in the generic
dw_dma_initialize_chan() method.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://lore.kernel.org/r/20200731200826.9292-3-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: dw: Add DMA-channels mask cell support 2020-10-29T08:58:09+00:00 Serge Semin Sergey.Semin@baikalelectronics.ru 2020-07-31T20:08:26+00:00 190bce292b734de5f164515b587d352b9e26441f [ Upstream commit e8ee6c8cb61b676f1a2d6b942329e98224bd8ee9 ] DW DMA IP-core provides a way to synthesize the DMA controller with channels having different parameters like maximum burst-length, multi-block support, maximum data width, etc. Those parameters both explicitly and implicitly affect the channels performance. Since DMA slave devices might be very demanding to the DMA performance, let's provide a functionality for the slaves to be assigned with DW DMA channels, which performance according to the platform engineer fulfill their requirements. After this patch is applied it can be done by passing the mask of suitable DMA-channels either directly in the dw_dma_slave structure instance or as a fifth cell of the DMA DT-property. If mask is zero or not provided, then there is no limitation on the channels allocation. For instance Baikal-T1 SoC is equipped with a DW DMAC engine, which first two channels are synthesized with max burst length of 16, while the rest of the channels have been created with max-burst-len=4. It would seem that the first two channels must be faster than the others and should be more preferable for the time-critical DMA slave devices. In practice it turned out that the situation is quite the opposite. The channels with max-burst-len=4 demonstrated a better performance than the channels with max-burst-len=16 even when they both had been initialized with the same settings. The performance drop of the first two DMA-channels made them unsuitable for the DW APB SSI slave device. No matter what settings they are configured with, full-duplex SPI transfers occasionally experience the Rx FIFO overflow. It means that the DMA-engine doesn't keep up with incoming data pace even though the SPI-bus is enabled with speed of 25MHz while the DW DMA controller is clocked with 50MHz signal. There is no such problem has been noticed for the channels synthesized with max-burst-len=4. Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru> Link: https://lore.kernel.org/r/20200731200826.9292-6-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit e8ee6c8cb61b676f1a2d6b942329e98224bd8ee9 ]

DW DMA IP-core provides a way to synthesize the DMA controller with
channels having different parameters like maximum burst-length,
multi-block support, maximum data width, etc. Those parameters both
explicitly and implicitly affect the channels performance. Since DMA slave
devices might be very demanding to the DMA performance, let's provide a
functionality for the slaves to be assigned with DW DMA channels, which
performance according to the platform engineer fulfill their requirements.
After this patch is applied it can be done by passing the mask of suitable
DMA-channels either directly in the dw_dma_slave structure instance or as
a fifth cell of the DMA DT-property. If mask is zero or not provided, then
there is no limitation on the channels allocation.

For instance Baikal-T1 SoC is equipped with a DW DMAC engine, which first
two channels are synthesized with max burst length of 16, while the rest
of the channels have been created with max-burst-len=4. It would seem that
the first two channels must be faster than the others and should be more
preferable for the time-critical DMA slave devices. In practice it turned
out that the situation is quite the opposite. The channels with
max-burst-len=4 demonstrated a better performance than the channels with
max-burst-len=16 even when they both had been initialized with the same
settings. The performance drop of the first two DMA-channels made them
unsuitable for the DW APB SSI slave device. No matter what settings they
are configured with, full-duplex SPI transfers occasionally experience the
Rx FIFO overflow. It means that the DMA-engine doesn't keep up with
incoming data pace even though the SPI-bus is enabled with speed of 25MHz
while the DW DMA controller is clocked with 50MHz signal. There is no such
problem has been noticed for the channels synthesized with
max-burst-len=4.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Link: https://lore.kernel.org/r/20200731200826.9292-6-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: dmatest: Check list for emptiness before access its last entry 2020-10-29T08:57:41+00:00 Andy Shevchenko andriy.shevchenko@linux.intel.com 2020-09-22T11:58:46+00:00 a7bf4cf31f57665265cb5102e80c6997efcc2f0b [ Upstream commit b28de385b71abf31ce68ec0387638bee26ae9024 ] After writing a garbage to the channel we get an Oops in dmatest_chan_set() due to access to last entry in the empty list. [ 212.670672] BUG: unable to handle page fault for address: fffffff000000020 [ 212.677562] #PF: supervisor read access in kernel mode [ 212.682702] #PF: error_code(0x0000) - not-present page ... [ 212.710074] RIP: 0010:dmatest_chan_set+0x149/0x2d0 [dmatest] [ 212.715739] Code: e8 cc f9 ff ff 48 8b 1d 0d 55 00 00 48 83 7b 10 00 0f 84 63 01 00 00 48 c7 c7 d0 65 4d c0 e8 ee 4a f5 e1 48 89 c6 48 8b 43 10 <48> 8b 40 20 48 8b 78 58 48 85 ff 0f 84 f5 00 00 00 e8 b1 41 f5 e1 Fix this by checking list for emptiness before accessing its last entry. Fixes: d53513d5dc28 ("dmaengine: dmatest: Add support for multi channel testing") Cc: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Tested-by: Peter Ujfalusi <peter.ujfalusi@ti.com> Link: https://lore.kernel.org/r/20200922115847.30100-2-andriy.shevchenko@linux.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit b28de385b71abf31ce68ec0387638bee26ae9024 ]

After writing a garbage to the channel we get an Oops in dmatest_chan_set()
due to access to last entry in the empty list.

[  212.670672] BUG: unable to handle page fault for address: fffffff000000020
[  212.677562] #PF: supervisor read access in kernel mode
[  212.682702] #PF: error_code(0x0000) - not-present page
...
[  212.710074] RIP: 0010:dmatest_chan_set+0x149/0x2d0 [dmatest]
[  212.715739] Code: e8 cc f9 ff ff 48 8b 1d 0d 55 00 00 48 83 7b 10 00 0f 84 63 01 00 00 48 c7 c7 d0 65 4d c0 e8 ee 4a f5 e1 48 89 c6 48 8b 43 10 <48> 8b 40 20 48 8b 78 58 48 85 ff 0f 84 f5 00 00 00 e8 b1 41 f5 e1

Fix this by checking list for emptiness before accessing its last entry.

Fixes: d53513d5dc28 ("dmaengine: dmatest: Add support for multi channel testing")
Cc: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200922115847.30100-2-andriy.shevchenko@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: tegra-apb: Prevent race conditions on channel's freeing 2020-10-01T11:17:36+00:00 Dmitry Osipenko digetx@gmail.com 2020-02-09T16:33:41+00:00 c4b62a7e52ee6b00bc52fe8beeb512f3eca6b144 [ Upstream commit 8e84172e372bdca20c305d92d51d33640d2da431 ] It's incorrect to check the channel's "busy" state without taking a lock. That shouldn't cause any real troubles, nevertheless it's always better not to have any race conditions in the code. Signed-off-by: Dmitry Osipenko <digetx@gmail.com> Acked-by: Jon Hunter <jonathanh@nvidia.com> Link: https://lore.kernel.org/r/20200209163356.6439-5-digetx@gmail.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 8e84172e372bdca20c305d92d51d33640d2da431 ]

It's incorrect to check the channel's "busy" state without taking a lock.
That shouldn't cause any real troubles, nevertheless it's always better
not to have any race conditions in the code.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://lore.kernel.org/r/20200209163356.6439-5-digetx@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: stm32-dma: use vchan_terminate_vdesc() in .terminate_all 2020-10-01T11:17:36+00:00 Amelie Delaunay amelie.delaunay@st.com 2020-01-29T15:36:28+00:00 cdf63c29af1e7eb1f53820c1b8adca8adc663537 [ Upstream commit d80cbef35bf89b763f06e03bb4ff8f933bf012c5 ] To avoid race with vchan_complete, use the race free way to terminate running transfer. Move vdesc->node list_del in stm32_dma_start_transfer instead of in stm32_mdma_chan_complete to avoid another race in vchan_dma_desc_free_list. Signed-off-by: Amelie Delaunay <amelie.delaunay@st.com> Link: https://lore.kernel.org/r/20200129153628.29329-9-amelie.delaunay@st.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit d80cbef35bf89b763f06e03bb4ff8f933bf012c5 ]

To avoid race with vchan_complete, use the race free way to terminate
running transfer.

Move vdesc->node list_del in stm32_dma_start_transfer instead of in
stm32_mdma_chan_complete to avoid another race in vchan_dma_desc_free_list.

Signed-off-by: Amelie Delaunay <amelie.delaunay@st.com>
Link: https://lore.kernel.org/r/20200129153628.29329-9-amelie.delaunay@st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: stm32-mdma: use vchan_terminate_vdesc() in .terminate_all 2020-10-01T11:17:35+00:00 Amelie Delaunay amelie.delaunay@st.com 2020-01-27T08:53:34+00:00 5132f9f3d3e2dff61a42043f00c93e7bf7b959f2 [ Upstream commit dfc708812a2acfc0ca56f56233b3c3e7b0d4ffe7 ] To avoid race with vchan_complete, use the race free way to terminate running transfer. Move vdesc->node list_del in stm32_mdma_start_transfer instead of in stm32_mdma_xfer_end to avoid another race in vchan_dma_desc_free_list. Signed-off-by: Amelie Delaunay <amelie.delaunay@st.com> Link: https://lore.kernel.org/r/20200127085334.13163-7-amelie.delaunay@st.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit dfc708812a2acfc0ca56f56233b3c3e7b0d4ffe7 ]

To avoid race with vchan_complete, use the race free way to terminate
running transfer.

Move vdesc->node list_del in stm32_mdma_start_transfer instead of in
stm32_mdma_xfer_end to avoid another race in vchan_dma_desc_free_list.

Signed-off-by: Amelie Delaunay <amelie.delaunay@st.com>
Link: https://lore.kernel.org/r/20200127085334.13163-7-amelie.delaunay@st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: zynqmp_dma: fix burst length configuration 2020-10-01T11:17:28+00:00 Matthias Fend matthias.fend@wolfvision.net 2020-01-15T10:22:49+00:00 26966e89ca14ba279eb44c6f12c3f4545a653fa6 [ Upstream commit cc88525ebffc757e00cc5a5d61da6271646c7f5f ] Since the dma engine expects the burst length register content as power of 2 value, the burst length needs to be converted first. Additionally add a burst length range check to avoid corrupting unrelated register bits. Signed-off-by: Matthias Fend <matthias.fend@wolfvision.net> Link: https://lore.kernel.org/r/20200115102249.24398-1-matthias.fend@wolfvision.net Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit cc88525ebffc757e00cc5a5d61da6271646c7f5f ]

Since the dma engine expects the burst length register content as
power of 2 value, the burst length needs to be converted first.
Additionally add a burst length range check to avoid corrupting unrelated
register bits.

Signed-off-by: Matthias Fend <matthias.fend@wolfvision.net>
Link: https://lore.kernel.org/r/20200115102249.24398-1-matthias.fend@wolfvision.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
dmaengine: mediatek: hsdma_probe: fixed a memory leak when devm_request_irq fails 2020-10-01T11:17:18+00:00 Satendra Singh Thakur sst2005@gmail.com 2019-11-09T11:35:23+00:00 21c82e8d42016a23c7ec92f0f9013678ae8d07ba [ Upstream commit 1ff95243257fad07290dcbc5f7a6ad79d6e703e2 ] When devm_request_irq fails, currently, the function dma_async_device_unregister gets called. This doesn't free the resources allocated by of_dma_controller_register. Therefore, we have called of_dma_controller_free for this purpose. Signed-off-by: Satendra Singh Thakur <sst2005@gmail.com> Link: https://lore.kernel.org/r/20191109113523.6067-1-sst2005@gmail.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 1ff95243257fad07290dcbc5f7a6ad79d6e703e2 ]

When devm_request_irq fails, currently, the function
dma_async_device_unregister gets called. This doesn't free
the resources allocated by of_dma_controller_register.
Therefore, we have called of_dma_controller_free for this purpose.

Signed-off-by: Satendra Singh Thakur <sst2005@gmail.com>
Link: https://lore.kernel.org/r/20191109113523.6067-1-sst2005@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
drivers/dma/dma-jz4780: Fix race condition between probe and irq handler 2020-09-17T11:47:46+00:00 Madhuparna Bhowmik madhuparnabhowmik10@gmail.com 2020-08-21T03:44:23+00:00 cb54984b79892b015ad766d62af0e5cfa3747fe8 [ Upstream commit 6d6018fc30bee67290dbed2fa51123f7c6f3d691 ] In probe, IRQ is requested before zchan->id is initialized which can be read in the irq handler. Hence, shift request irq after other initializations complete. Found by Linux Driver Verification project (linuxtesting.org). Signed-off-by: Madhuparna Bhowmik <madhuparnabhowmik10@gmail.com> Reviewed-by: Paul Cercueil <paul@crapouillou.net> Link: https://lore.kernel.org/r/20200821034423.12713-1-madhuparnabhowmik10@gmail.com Signed-off-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 6d6018fc30bee67290dbed2fa51123f7c6f3d691 ]

In probe, IRQ is requested before zchan->id is initialized which can be
read in the irq handler. Hence, shift request irq after other initializations
complete.

Found by Linux Driver Verification project (linuxtesting.org).

Signed-off-by: Madhuparna Bhowmik <madhuparnabhowmik10@gmail.com>
Reviewed-by: Paul Cercueil <paul@crapouillou.net>
Link: https://lore.kernel.org/r/20200821034423.12713-1-madhuparnabhowmik10@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>