linux.git/include/linux/dma/edma.h, branch v7.3-rc2

dmaengine: dw-edma: Program endpoint function numbers

2026-07-21T10:30:36+00:00

The eDMA/HDMA transfers the driver issues carry a requester function
number in their TLPs, but nothing ever programs it: eDMA v0 leaves the
FUNC_NUM field of the channel control word zero and HDMA leaves the
per-channel func_num register at its reset value, so every transfer is
attributed to function 0. That is invisible in single-function setups,
but once the DMA block serves a non-zero endpoint function, its
requests must carry that function's number for the host to attribute
and translate them correctly.

Record the function number in the chip data (PCI_FUNC() of the probing
device for dw-edma-pcie) and program it per channel.

Endpoint-local chip instances keep func_no at 0, so transfers issued by
the endpoint-side driver remain PF0-attributed. Delegated channels are
programmed by the host-side dw-edma-pcie instance when it takes over the
channel, using that instance's PCI_FUNC().

Reviewed-by: Frank Li 
Signed-off-by: Koichiro Den 
Link: https://patch.msgid.link/20260721062815.4117887-15-den@valinux.co.jp
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Add partial channel ownership mode

2026-07-21T10:30:36+00:00

A DesignWare eDMA instance may represent only a subset of channels that
is also initialized by another OS instance, such as an endpoint-side OS.
Add a partial ownership flag for instances that must preserve
controller-wide state owned by that peer.

In partial ownership mode, dw-edma skips the initial core reset and uses
the limited quiesce path in probe() and remove() instead of the full
core-off path. The flag also makes the driver validate the ownership
granularity required by each register layout before registering
channels.

Probe fails if the limited quiesce cannot stop its resources; remove
reports the error after completing the remaining driver teardown.

Partial instances also skip interrupt-emulation doorbell allocation: the
emulated doorbell is a controller-level resource, and a partial owner
must not claim it on behalf of the whole block.

For EDMA_MF_EDMA_UNROLL and EDMA_MF_HDMA_COMPAT, the driver programs
per-direction registers, such as DMA_{WRITE,READ}_INT_MASK_OFF and
DMA_{WRITE,READ}_INT_CLEAR_OFF. These register layouts have at most
EDMA_MAX_{WR,RD}_CH channels per direction, so the capped hardware
channel count still represents the whole direction. A partial instance
can therefore expose write or read channels only if it owns every
channel in that direction; otherwise two OS instances could update the
same direction-wide registers without a shared locking protocol.

In contrast, HDMA native uses per-channel registers, so it can be owned
at channel granularity.

Signed-off-by: Koichiro Den 
Link: https://patch.msgid.link/20260721062815.4117887-6-den@valinux.co.jp
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Add per-channel interrupt routing control

2026-07-21T10:30:35+00:00

DesignWare eDMA can signal completion locally through edma_int[] and
remotely through IMWr/MSI. When channels are delegated to a remote
frontend, the local endpoint side and the remote host side must not both
service the same DONE/ABORT status.

Add channel interrupt routing state and initialize it from the controller
instance configuration. Update the eDMA and HDMA native paths so
linked-list interrupt generation, HDMA non-linked-list interrupt enables,
and DONE/ABORT masking follow the selected mode. For HDMA native
non-linked-list channels, keep the local stop/abort enables set so status
is latched. In remote mode, also enable remote signaling and mask the local
interrupt pins.

Keep the existing dw-edma-pcie host-side instances in remote interrupt
routing mode so their IMWr/MSI completion model remains unchanged after
local routing becomes the zero value.

Note:
- The routing mode describes where a channel should report completion.
  It does not by itself say whether this dw-edma instance owns the
  interrupt status. A local instance must ignore remote-only channels,
  and a remote instance must ignore local-only channels, even if such
  interrupts are unexpectedly delivered. Otherwise the non-owner side
  could steal the interrupt from the owner by clearing shared DONE/ABORT
  status.

Cc: Devendra K Verma 
Suggested-by: Frank Li 
Signed-off-by: Koichiro Den 
Link: https://patch.msgid.link/20260721062815.4117887-3-den@valinux.co.jp
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Enable HDMA 64R/W Channels

2026-07-17T12:02:22+00:00

As per 'Designware Cores PCI Express Controller Databook',
Section 7.1 - Overview, HDMA supports 64 Read and 64 Write
channels. Current controller driver supports up to 8 read and
write channels only. In order to utilize all the channels the
controller driver need to have the channel related structs
and variables as per the number of channels supported by IP.
Following changes are made to enable 64 Read / 64 Write
channel support:

 o Defined HDMA specific macros to reflect the channel count.
 o The count of ll_regions and dt_regions in dw_edma_chip and
   dw_edma_pcie_data shall be in accordance to number of read
   and write channels.
 o In dw_edma_probe() configure the channels as per the channels
   of the IP used.
 o Changed mask types to u64 for higher channel counts.

Signed-off-by: Devendra K Verma 
Reviewed-by: Frank Li 
Link: https://patch.msgid.link/20260713064854.4065262-1-devverma@amd.com
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Add non-LL mode

2026-03-18T09:41:40+00:00

AMD MDB IP supports Linked List (LL) mode as well as non-LL mode.
The current code does not have the mechanisms to enable the
DMA transactions using the non-LL mode. The following two cases
are added with this patch:
- For the AMD (Xilinx) only, when a valid physical base address of
  the device side DDR is not configured, then the IP can still be
  used in non-LL mode. For all the channels DMA transactions will
  be using the non-LL mode only. This, the default non-LL mode,
  is not applicable for Synopsys IP with the current code addition.

- If the default mode is LL-mode, for both AMD (Xilinx) and Synosys,
  and if user wants to use non-LL mode then user can do so via
  configuring the peripheral_config param of dma_slave_config.

Signed-off-by: Devendra K Verma 
Reviewed-by: Frank Li 
Link: https://patch.msgid.link/20260318070403.1634706-3-devendra.verma@amd.com
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Add virtual IRQ for interrupt-emulation doorbells

2026-02-25T09:54:19+00:00

Interrupt emulation can assert the dw-edma IRQ line without updating the
DONE/ABORT bits. With the shared read/write/common IRQ handlers, the
driver cannot reliably distinguish such an emulated interrupt from a
real one and leaving a level IRQ asserted may wedge the line.

Allocate a dedicated, requestable Linux virtual IRQ (db_irq) for
interrupt emulation and attach an irq_chip whose .irq_ack runs the
core-specific deassert sequence (.ack_emulated_irq()). The physical
dw-edma interrupt handlers raise this virtual IRQ via
generic_handle_irq(), ensuring emulated IRQs are always deasserted.

Export the virtual IRQ number (db_irq) and the doorbell register offset
(db_offset) via struct dw_edma_chip so platform users can expose
interrupt emulation as a doorbell.

Without this, a single interrupt-emulation write can leave the level IRQ
line asserted and cause the generic IRQ layer to disable it.

Signed-off-by: Koichiro Den 
Reviewed-by: Frank Li 
Link: https://patch.msgid.link/20260215152216.3393561-3-den@valinux.co.jp
Signed-off-by: Vinod Koul

dmaengine: dw_edma: correct kernel-doc warnings in

2025-12-16T16:53:35+00:00

Use the correct enum name in its kernel-doc heading.
Add ending ':' to struct member names.
Drop the @id: kernel-doc entry since there is no struct member named 'id'.

edma.h:46: warning: expecting prototype for struct dw_edma_core_ops.
 Prototype was for struct dw_edma_plat_ops instead
Warning: edma.h:101 struct member 'ops' not described in 'dw_edma_chip'
Warning: edma.h:101 struct member 'flags' not described in 'dw_edma_chip'
Warning: edma.h:101 struct member 'reg_base' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'll_wr_cnt' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'll_rd_cnt' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'll_region_wr' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'll_region_rd' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'dt_region_wr' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'dt_region_rd' not described
 in 'dw_edma_chip'
Warning: edma.h:101 struct member 'mf' not described in 'dw_edma_chip'

Signed-off-by: Randy Dunlap 
Link: https://patch.msgid.link/20251101191524.1991135-1-rdunlap@infradead.org
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Add support for native HDMA

2023-05-24T06:50:45+00:00

Add support for HDMA NATIVE, as long the IP design has set
the compatible register map parameter-HDMA_NATIVE,
which allows compatibility for native HDMA register configuration.

The HDMA Hyper-DMA IP is an enhancement of the eDMA embedded-DMA IP.
And the native HDMA registers are different from eDMA, so this patch
add support for HDMA NATIVE mode.

HDMA write and read channels operate independently to maximize
the performance of the HDMA read and write data transfer over
the link When you configure the HDMA with multiple read channels,
then it uses a round robin (RR) arbitration scheme to select
the next read channel to be serviced.The same applies when you
have multiple write channels.

The native HDMA driver also supports a maximum of 16 independent
channels (8 write + 8 read), which can run simultaneously.
Both SAR (Source Address Register) and DAR (Destination Address Register)
are aligned to byte.

Signed-off-by: Cai Huoqing 
Reviewed-by: Serge Semin 
Reviewed-by: Manivannan Sadhasivam 
Tested-by: Serge Semin 
Link: https://lore.kernel.org/r/20230520050854.73160-4-cai.huoqing@linux.dev
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Rename dw_edma_core_ops structure to dw_edma_plat_ops

2023-05-24T06:50:45+00:00

The dw_edma_core_ops structure contains a set of the operations:
device IRQ numbers getter, CPU/PCI address translation. Based on the
functions semantics the structure name "dw_edma_plat_ops" looks more
descriptive since indeed the operations are platform-specific. The
"dw_edma_core_ops" name shall be used for a structure with the IP-core
specific set of callbacks in order to abstract out DW eDMA and DW HDMA
setups. Such structure will be added in one of the next commit in the
framework of the set of changes adding the DW HDMA device support.

Anyway the renaming was necessary to distinguish two types of
the implementation callbacks:
1. DW eDMA/hDMA IP-core specific operations: device-specific CSR
setups in one or another aspect of the DMA-engine initialization.
2. DW eDMA/hDMA platform specific operations: the DMA device
environment configs like IRQs, address translation, etc.

Signed-off-by: Cai Huoqing 
Reviewed-by: Serge Semin 
Reviewed-by: Manivannan Sadhasivam 
Tested-by: Serge Semin 
Link: https://lore.kernel.org/r/20230520050854.73160-2-cai.huoqing@linux.dev
Signed-off-by: Vinod Koul

dmaengine: dw-edma: Prepare dw_edma_probe() for builtin callers

2023-02-22T19:46:09+00:00

When CONFIG_DW_EDMA=m, dw_edma_probe() is built as a module.  Previously
edma.h declared it as extern, but the implementation isn't available for
builtin callers.  A subsequent commit will add calls from
dw_pcie_host_init() and dw_pcie_ep_init(), which can only be built-in.

Make it safe for such builtin callers to call dw_edma_probe() by using
IS_REACHABLE() to define a stub when CONFIG_DW_EDMA=m.

When CONFIG_DW_EDMA=m, these builtin callers will fail to detect and
register eDMA devices, so eDMA won't be usable even if the dw-edma module
is loaded.

[bhelgaas: split to separate patch, commit log]
Link: https://lore.kernel.org/r/20230113171409.30470-25-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin 
Signed-off-by: Lorenzo Pieralisi 
Signed-off-by: Bjorn Helgaas 
Acked-by: Vinod Koul