linux.git/drivers/dma/idxd, branch v5.14 Linux kernel source tree dmaengine: idxd: fix submission race window 2021-07-20T17:25:08+00:00 Dave Jiang dave.jiang@intel.com 2021-07-14T18:50:06+00:00 6b4b87f2c31ac1af4f244990a7cbfb50d3f3e33f Konstantin observed that when descriptors are submitted, the descriptor is added to the pending list after the submission. This creates a race window with the slight possibility that the descriptor can complete before it gets added to the pending list and this window would cause the completion handler to miss processing the descriptor. To address the issue, the addition of the descriptor to the pending list must be done before it gets submitted to the hardware. However, submitting to swq with ENQCMDS instruction can cause a failure with the condition of either wq is full or wq is not "active". With the descriptor allocation being the gate to the wq capacity, it is not possible to hit a retry with ENQCMDS submission to the swq. The only possible failure can happen is when wq is no longer "active" due to hw error and therefore we are moving towards taking down the portal. Given this is a rare condition and there's no longer concern over I/O performance, the driver can walk the completion lists in order to retrieve and abort the descriptor. The error path will set the descriptor to aborted status. It will take the work list lock to prevent further processing of worklist. It will do a delete_all on the pending llist to retrieve all descriptors on the pending llist. The delete_all action does not require a lock. It will walk through the acquired llist to find the aborted descriptor while add all remaining descriptors to the work list since it holds the lock. If it does not find the aborted descriptor on the llist, it will walk through the work list. And if it still does not find the descriptor, then it means the interrupt handler has removed the desc from the llist but is pending on the work list lock and will process it once the error path releases the lock. Fixes: eb15e7154fbf ("dmaengine: idxd: add interrupt handle request and release support") Reported-by: Konstantin Ananyev <konstantin.ananyev@intel.com> Signed-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/162628855747.360485.10101925573082466530.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
Konstantin observed that when descriptors are submitted, the descriptor is
added to the pending list after the submission. This creates a race window
with the slight possibility that the descriptor can complete before it
gets added to the pending list and this window would cause the completion
handler to miss processing the descriptor.

To address the issue, the addition of the descriptor to the pending list
must be done before it gets submitted to the hardware. However, submitting
to swq with ENQCMDS instruction can cause a failure with the condition of
either wq is full or wq is not "active".

With the descriptor allocation being the gate to the wq capacity, it is not
possible to hit a retry with ENQCMDS submission to the swq. The only
possible failure can happen is when wq is no longer "active" due to hw
error and therefore we are moving towards taking down the portal. Given
this is a rare condition and there's no longer concern over I/O
performance, the driver can walk the completion lists in order to retrieve
and abort the descriptor.

The error path will set the descriptor to aborted status. It will take the
work list lock to prevent further processing of worklist. It will do a
delete_all on the pending llist to retrieve all descriptors on the pending
llist. The delete_all action does not require a lock. It will walk through
the acquired llist to find the aborted descriptor while add all remaining
descriptors to the work list since it holds the lock. If it does not find
the aborted descriptor on the llist, it will walk through the work
list. And if it still does not find the descriptor, then it means the
interrupt handler has removed the desc from the llist but is pending on
the work list lock and will process it once the error path releases the
lock.

Fixes: eb15e7154fbf ("dmaengine: idxd: add interrupt handle request and release support")
Reported-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162628855747.360485.10101925573082466530.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine: idxd: fix sequence for pci driver remove() and shutdown() 2021-07-20T17:24:20+00:00 Dave Jiang dave.jiang@intel.com 2021-07-14T21:57:19+00:00 7eb25da161befbc9a80e94e1bd90d6c06aa645cf ->shutdown() call should only be responsible for quiescing the device. Currently it is doing PCI device tear down. This causes issue when things like MMIO mapping is removed while idxd_unregister_devices() will trigger removal of idxd device sub-driver and still initiates MMIO writes to the device. Another issue is with the unregistering of idxd 'struct device', the memory context gets freed. So the teardown calls are accessing freed memory and can cause kernel oops. Move all the teardown bits that doesn't belong in shutdown to ->remove() call. Move unregistering of the idxd conf_dev 'struct device' to after doing all the teardown to free all the memory that's no longer needed. Fixes: 47c16ac27d4c ("dmaengine: idxd: fix idxd conf_dev 'struct device' lifetime") Signed-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/162629983901.395844.17964803190905549615.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
->shutdown() call should only be responsible for quiescing the device.
Currently it is doing PCI device tear down. This causes issue when things
like MMIO mapping is removed while idxd_unregister_devices() will trigger
removal of idxd device sub-driver and still initiates MMIO writes to the
device. Another issue is with the unregistering of idxd 'struct device',
the memory context gets freed. So the teardown calls are accessing freed
memory and can cause kernel oops. Move all the teardown bits that doesn't
belong in shutdown to ->remove() call. Move unregistering of the idxd
conf_dev 'struct device' to after doing all the teardown to free all
the memory that's no longer needed.

Fixes: 47c16ac27d4c ("dmaengine: idxd: fix idxd conf_dev 'struct device' lifetime")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162629983901.395844.17964803190905549615.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine: idxd: fix desc->vector that isn't being updated 2021-07-20T17:23:56+00:00 Dave Jiang dave.jiang@intel.com 2021-07-14T18:38:41+00:00 8ba89a3c7967808f33478a8573277cf6a7412c4c Missing update for desc->vector when the wq vector gets updated. This causes the desc->vector to always be at 0. Fixes: da435aedb00a ("dmaengine: idxd: fix array index when int_handles are being used") Signed-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/162628784374.353761.4736602409627820431.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
Missing update for desc->vector when the wq vector gets updated. This
causes the desc->vector to always be at 0.

Fixes: da435aedb00a ("dmaengine: idxd: fix array index when int_handles are being used")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162628784374.353761.4736602409627820431.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine: idxd: fix setup sequence for MSIXPERM table 2021-07-14T06:57:28+00:00 Dave Jiang dave.jiang@intel.com 2021-06-24T20:43:32+00:00 d5c10e0fc8645342fe5c9796b00c84ab078cd713 The MSIX permission table should be programmed BEFORE request_irq() happens. This prevents any possibility of an interrupt happening before the MSIX perm table is setup, however slight. Fixes: 6df0e6c57dfc ("dmaengine: idxd: clear MSIX permission entry on shutdown") Sign-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/162456741222.1138073.1298447364671237896.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
The MSIX permission table should be programmed BEFORE request_irq()
happens. This prevents any possibility of an interrupt happening before the
MSIX perm table is setup, however slight.

Fixes: 6df0e6c57dfc ("dmaengine: idxd: clear MSIX permission entry on shutdown")
Sign-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162456741222.1138073.1298447364671237896.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine: idxd: fix array index when int_handles are being used 2021-07-14T06:57:28+00:00 Dave Jiang dave.jiang@intel.com 2021-06-24T19:09:29+00:00 da435aedb00a4ef61019ff11ae0c08ffb9b1fb18 The index to the irq vector should be local and has no relation to the assigned interrupt handle. Assign the MSIX interrupt index that is programmed for the descriptor. The interrupt handle only matters when it comes to hardware descriptor programming. Fixes: eb15e7154fbf ("dmaengine: idxd: add interrupt handle request and release support") Signed-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/162456176939.1121476.3366256009925001897.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
The index to the irq vector should be local and has no relation to
the assigned interrupt handle. Assign the MSIX interrupt index that is
programmed for the descriptor. The interrupt handle only matters when it
comes to hardware descriptor programming.

Fixes: eb15e7154fbf ("dmaengine: idxd: add interrupt handle request and release support")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162456176939.1121476.3366256009925001897.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Merge tag 'dmaengine-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine 2021-07-05T19:05:13+00:00 Linus Torvalds torvalds@linux-foundation.org 2021-07-05T19:05:13+00:00 18ef082713ad1104c32cd17a15abdc3f43c9b28a Pull dmaengine updates from Vinod Koul: "This time around we have a smaller pull request than usual and this includes code removal, so should be good! New drivers/devices - Support for QCOM SM8250 GPI DMA - removal of shdma-of driver and binding Updates: - arm-pl08x yaml binding move - altera-msgdma gained DT support - removal of imx-sdma platform data support - idxd and xilinx driver updates" * tag 'dmaengine-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (22 commits) dmaengine: imx-sdma: Remove platform data header dmaengine: xilinx: dpdma: Fix spacing around addr[i-1] dmaengine: xilinx: dpdma: Use kernel type u32 over uint32_t dmaengine: altera-msgdma: add OF support MAINTAINERS: add entry for Altera mSGDMA dt-bindings: dma: add schema for altera-msgdma dmaengine: xilinx: dpdma: fix kernel-doc dmaengine: sf-pdma: apply proper spinlock flags in sf_pdma_prep_dma_memcpy() dmaengine: sh: Remove unused shdma-of driver dt-bindings: dmaengine: Remove SHDMA Device Tree bindings dmaengine: qcom: gpi: Add SM8250 compatible dt-bindings: dmaengine: qcom: gpi: add compatible for sm8250 dmaengine: sun4i: Use list_move_tail instead of list_del/list_add_tail dmaengine: ti: omap-dma: Skip pointless cpu_pm context restore on errors dmaengine: hsu: Account transferred bytes dmaengine: Move kdoc description of struct dma_chan_percpu closer to it dmaengine: xilinx: dpdma: Print debug message when losing vsync race dmaengine: xilinx: dpdma: Print channel number in kernel log messages dt-bindings: dma: convert arm-pl08x to yaml dmaengine: idxd: remove devm allocation for idxd->int_handles ... 
Pull dmaengine updates from Vinod Koul:
 "This time around we have a smaller pull request than usual and this
  includes code removal, so should be good!

  New drivers/devices
   - Support for QCOM SM8250 GPI DMA
   - removal of shdma-of driver and binding

  Updates:
   - arm-pl08x yaml binding move
   - altera-msgdma gained DT support
   - removal of imx-sdma platform data support
   - idxd and xilinx driver updates"

* tag 'dmaengine-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (22 commits)
  dmaengine: imx-sdma: Remove platform data header
  dmaengine: xilinx: dpdma: Fix spacing around addr[i-1]
  dmaengine: xilinx: dpdma: Use kernel type u32 over uint32_t
  dmaengine: altera-msgdma: add OF support
  MAINTAINERS: add entry for Altera mSGDMA
  dt-bindings: dma: add schema for altera-msgdma
  dmaengine: xilinx: dpdma: fix kernel-doc
  dmaengine: sf-pdma: apply proper spinlock flags in sf_pdma_prep_dma_memcpy()
  dmaengine: sh: Remove unused shdma-of driver
  dt-bindings: dmaengine: Remove SHDMA Device Tree bindings
  dmaengine: qcom: gpi: Add SM8250 compatible
  dt-bindings: dmaengine: qcom: gpi: add compatible for sm8250
  dmaengine: sun4i: Use list_move_tail instead of list_del/list_add_tail
  dmaengine: ti: omap-dma: Skip pointless cpu_pm context restore on errors
  dmaengine: hsu: Account transferred bytes
  dmaengine: Move kdoc description of struct dma_chan_percpu closer to it
  dmaengine: xilinx: dpdma: Print debug message when losing vsync race
  dmaengine: xilinx: dpdma: Print channel number in kernel log messages
  dt-bindings: dma: convert arm-pl08x to yaml
  dmaengine: idxd: remove devm allocation for idxd->int_handles
  ...
Merge tag 'dmaengine-fix-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine 2021-06-16T16:03:52+00:00 Linus Torvalds torvalds@linux-foundation.org 2021-06-16T16:03:52+00:00 6b00bc639f1f2beeff3595e1bab9faaa51d23b01 Pull dmaengine fixes from Vinod Koul: "A bunch of driver fixes, notably: - More idxd fixes for driver unregister, error handling and bus assignment - HAS_IOMEM depends fix for few drivers - lock fix in pl330 driver - xilinx drivers fixes for initialize registers, missing dependencies and limiting descriptor IDs - mediatek descriptor management fixes" * tag 'dmaengine-fix-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: dmaengine: mediatek: use GFP_NOWAIT instead of GFP_ATOMIC in prep_dma dmaengine: mediatek: do not issue a new desc if one is still current dmaengine: mediatek: free the proper desc in desc_free handler dmaengine: ipu: fix doc warning in ipu_irq.c dmaengine: rcar-dmac: Fix PM reference leak in rcar_dmac_probe() dmaengine: idxd: Fix missing error code in idxd_cdev_open() dmaengine: stedma40: add missing iounmap() on error in d40_probe() dmaengine: SF_PDMA depends on HAS_IOMEM dmaengine: QCOM_HIDMA_MGMT depends on HAS_IOMEM dmaengine: ALTERA_MSGDMA depends on HAS_IOMEM dmaengine: idxd: Add missing cleanup for early error out in probe call dmaengine: xilinx: dpdma: Limit descriptor IDs to 16 bits dmaengine: xilinx: dpdma: Add missing dependencies to Kconfig dmaengine: stm32-mdma: fix PM reference leak in stm32_mdma_alloc_chan_resourc() dmaengine: zynqmp_dma: Fix PM reference leak in zynqmp_dma_alloc_chan_resourc() dmaengine: xilinx: dpdma: initialize registers before request_irq dmaengine: pl330: fix wrong usage of spinlock flags in dma_cyclc dmaengine: fsl-dpaa2-qdma: Fix error return code in two functions dmaengine: idxd: add missing dsa driver unregister dmaengine: idxd: add engine 'struct device' missing bus type assignment 
Pull dmaengine fixes from Vinod Koul:
 "A bunch of driver fixes, notably:

   - More idxd fixes for driver unregister, error handling and bus
     assignment

   - HAS_IOMEM depends fix for few drivers

   - lock fix in pl330 driver

   - xilinx drivers fixes for initialize registers, missing dependencies
     and limiting descriptor IDs

   - mediatek descriptor management fixes"

* tag 'dmaengine-fix-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine:
  dmaengine: mediatek: use GFP_NOWAIT instead of GFP_ATOMIC in prep_dma
  dmaengine: mediatek: do not issue a new desc if one is still current
  dmaengine: mediatek: free the proper desc in desc_free handler
  dmaengine: ipu: fix doc warning in ipu_irq.c
  dmaengine: rcar-dmac: Fix PM reference leak in rcar_dmac_probe()
  dmaengine: idxd: Fix missing error code in idxd_cdev_open()
  dmaengine: stedma40: add missing iounmap() on error in d40_probe()
  dmaengine: SF_PDMA depends on HAS_IOMEM
  dmaengine: QCOM_HIDMA_MGMT depends on HAS_IOMEM
  dmaengine: ALTERA_MSGDMA depends on HAS_IOMEM
  dmaengine: idxd: Add missing cleanup for early error out in probe call
  dmaengine: xilinx: dpdma: Limit descriptor IDs to 16 bits
  dmaengine: xilinx: dpdma: Add missing dependencies to Kconfig
  dmaengine: stm32-mdma: fix PM reference leak in stm32_mdma_alloc_chan_resourc()
  dmaengine: zynqmp_dma: Fix PM reference leak in zynqmp_dma_alloc_chan_resourc()
  dmaengine: xilinx: dpdma: initialize registers before request_irq
  dmaengine: pl330: fix wrong usage of spinlock flags in dma_cyclc
  dmaengine: fsl-dpaa2-qdma: Fix error return code in two functions
  dmaengine: idxd: add missing dsa driver unregister
  dmaengine: idxd: add engine 'struct device' missing bus type assignment
dmaengine: idxd: Use cpu_feature_enabled() 2021-06-03T14:32:59+00:00 Borislav Petkov bp@suse.de 2021-06-02T10:07:52+00:00 74b2fc882d380d8fafc2a26f01d401c2a7beeadb When testing x86 feature bits, use cpu_feature_enabled() so that build-disabled features can remain off, regardless of what CPUID says. Fixes: 8e50d392652f ("dmaengine: idxd: Add shared workqueue support") Signed-off-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Acked-By: Vinod Koul <vkoul@kernel.org> Cc: <stable@vger.kernel.org> 
When testing x86 feature bits, use cpu_feature_enabled() so that
build-disabled features can remain off, regardless of what CPUID says.

Fixes: 8e50d392652f ("dmaengine: idxd: Add shared workqueue support")
Signed-off-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Acked-By: Vinod Koul <vkoul@kernel.org>
Cc: <stable@vger.kernel.org>
dmaengine: idxd: Fix missing error code in idxd_cdev_open() 2021-06-03T06:58:37+00:00 Jiapeng Chong jiapeng.chong@linux.alibaba.com 2021-06-02T10:07:26+00:00 99b18e88a1cf737ae924123d63b46d9a3d17b1af The error code is missing in this code scenario, add the error code '-EINVAL' to the return value 'rc'. Eliminate the follow smatch warning: drivers/dma/idxd/cdev.c:113 idxd_cdev_open() warn: missing error code 'rc'. Reported-by: Abaci Robot <abaci@linux.alibaba.com> Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com> Acked-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/1622628446-87909-1-git-send-email-jiapeng.chong@linux.alibaba.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
The error code is missing in this code scenario, add the error code
'-EINVAL' to the return value 'rc'.

Eliminate the follow smatch warning:

drivers/dma/idxd/cdev.c:113 idxd_cdev_open() warn: missing error code
'rc'.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/1622628446-87909-1-git-send-email-jiapeng.chong@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine: idxd: Add missing cleanup for early error out in probe call 2021-05-31T04:09:14+00:00 Dave Jiang dave.jiang@intel.com 2021-05-25T19:23:37+00:00 ddf742d4f3f12a6ba1b8e6ecbbf3ae736942f970 The probe call stack is missing some cleanup when things fail in the middle. Add the appropriate cleanup routines to make sure we exit gracefully. Fixes: a39c7cd0438e ("dmaengine: idxd: removal of pcim managed mmio mapping") Reported-by: Nikhil Rao <nikhil.rao@intel.com> Signed-off-by: Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/162197061707.392656.15760573520817310791.stgit@djiang5-desk3.ch.intel.com Signed-off-by: Vinod Koul <vkoul@kernel.org> 
The probe call stack is missing some cleanup when things fail in the
middle. Add the appropriate cleanup routines to make sure we exit
gracefully.

Fixes: a39c7cd0438e ("dmaengine: idxd: removal of pcim managed mmio mapping")
Reported-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162197061707.392656.15760573520817310791.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>