[ Upstream commit 5f5ac460dfe7f4e11f99de9870f240e39189cf72 ]

commit bb38919ec56e ("PCI: imx6: Add support for i.MX6 PCIe controller")
added a fault hook to this driver in the probe function. So it was only
installed if needed.

commit bde4a5a00e76 ("PCI: imx6: Allow probe deferral by reset GPIO")
moved it from probe to driver init which installs the hook unconditionally
as soon as the driver is compiled into a kernel.

When this driver is compiled as a module, the hook is not registered
until after the driver has been matched with a .compatible and
loaded.

commit 415b6185c541 ("PCI: imx6: Fix config read timeout handling")
extended the fault handling code.

commit 2d8ed461dbc9 ("PCI: imx6: Add support for i.MX8MQ")
added some protection for non-ARM architectures, but this does not
protect non-i.MX ARM architectures.

Since fault handlers can be triggered on any architecture for different
reasons, there is no guarantee that they will be triggered only for the
assumed situation, leading to improper error handling (i.MX6-specific
imx6q_pcie_abort_handler) on foreign systems.

I had seen strange L3 imprecise external abort messages several times on
OMAP4 and OMAP5 devices and couldn't make sense of them until I realized
they were related to this unused imx6q driver because I had
CONFIG_PCI_IMX6=y.

Note that CONFIG_PCI_IMX6=y is useful for kernel binaries that are designed
to run on different ARM SoC and be differentiated only by device tree
binaries. So turning off CONFIG_PCI_IMX6 is not a solution.

Therefore we check the compatible in the init function before registering
the fault handler.

Link: https://lore.kernel.org/r/e1bcfc3078c82b53aa9b78077a89955abe4ea009.1678380991.git.hns@goldelico.com
Fixes: bde4a5a00e76 ("PCI: imx6: Allow probe deferral by reset GPIO")
Fixes: 415b6185c541 ("PCI: imx6: Fix config read timeout handling")
Fixes: 2d8ed461dbc9 ("PCI: imx6: Add support for i.MX8MQ")
Signed-off-by: H. Nikolaus Schaller <hns@goldelico.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Richard Zhu <hongxing.zhu@nxp.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

commit 2542e16c392508800f1d9037feee881a9c444951 upstream.

Qcom PCIe IP version v2.7.0 and its derivatives don't contain the
PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT register. Instead, they have the new
PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT_V2 register. So fix the incorrect
register usage which is modifying a different register.

Also in this IP version, this register change doesn't depend on MSI
being enabled. So remove that check also.

Link: https://lore.kernel.org/r/20230316081117.14288-2-manivannan.sadhasivam@linaro.org
Fixes: ed8cc3b1fc84 ("PCI: qcom: Add support for SDM845 PCIe controller")
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Cc: <stable@vger.kernel.org> # 5.6+
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 3a2776e8a0e156a61f5b59ae341d8fffc730b962 upstream.

pcie-kirin uses regmaps, and needs to pull them in; otherwise, with
CONFIG_PCIE_KIRIN=y and without CONFIG_REGMAP_MMIO pcie-kirin produces
a linker failure looking for __devm_regmap_init_mmio_clk().

Fixes: d19afe7be126 ("PCI: kirin: Use regmap for APB registers")
Link: https://lore.kernel.org/r/04636141da1d6d592174eefb56760511468d035d.1668410580.git.josh@joshtriplett.org
Signed-off-by: Josh Triplett <josh@joshtriplett.org>
[lpieralisi@kernel.org: commit log and removed REGMAP select]
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Cc: stable@vger.kernel.org # 5.16+
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

If CDM_CHECK is enabled (by the DT "snps,enable-cdm-check" property), 'val'
is overwritten by PCIE_PL_CHK_REG_CONTROL_STATUS initialization.  Commit
ec7b952f453c ("PCI: dwc: Always enable CDM check if "snps,enable-cdm-check"
exists") did not account for further usage of 'val', so we wrote improper
values to PCIE_PORT_LINK_CONTROL when the CDM check is enabled.

Move the PCIE_PORT_LINK_CONTROL update to be completely after the
PCIE_PL_CHK_REG_CONTROL_STATUS register initialization.

[bhelgaas: commit log adapted from Serge's version]
Fixes: ec7b952f453c ("PCI: dwc: Always enable CDM check if "snps,enable-cdm-check" exists")
Link: https://lore.kernel.org/r/20230310123510.675685-2-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>

Pull PCI updates from Bjorn Helgaas:
 "Enumeration:

   - Rework portdrv shutdown so it disables interrupts but doesn't
     disable bus mastering, which leads to hangs on Loongson LS7A

   - Add mechanism to prevent Max_Read_Request_Size (MRRS) increases,
     again to avoid hardware issues on Loongson LS7A (and likely other
     devices based on DesignWare IP)

   - Ignore devices with a firmware (DT or ACPI) node that says the
     device is disabled

  Resource management:

   - Distribute spare resources to unconfigured hotplug bridges at
     boot-time (not just when hot-adding such a bridge), which makes
     hot-adding devices to docks work better. Tried this in v6.1 but had
     to revert for regressions, so try again

   - Fix root bus issue that dropped resources that happened to end
     at 0, e.g., [bus 00]

  PCI device hotplug:

   - Remove device locking when marking device as disconnected so this
     doesn't have to wait for concurrent driver bind/unbind to complete

   - Quirk more Qualcomm bridges that don't fully implement the PCIe
     Slot Status 'Command Completed' bit

  Power management:

   - Account for _S0W of the target bridge in acpi_pci_bridge_d3() so we
     don't miss hot-add notifications for USB4 docks, Thunderbolt, etc

  Reset:

   - Observe delay after reset, e.g., resuming from system sleep,
     regardless of whether a bridge can suspend to D3cold at runtime

   - Wait for secondary bus to become ready after a bridge reset

  Virtualization:

   - Avoid FLR on some AMD FCH AHCI adapters where it doesn't work

   - Allow independent IOMMU groups for some Wangxun NICs that prevent
     peer-to-peer transactions but don't advertise an ACS Capability

  Error handling:

   - Configure End-to-End-CRC (ECRC) only if Linux owns the AER
     Capability

   - Remove redundant Device Control Error Reporting Enable in the AER
     service driver since this is already done for all devices during
     enumeration

  ASPM:

   - Add pci_enable_link_state() interface to allow drivers to enable
     ASPM link state

  Endpoint framework:

   - Move dra7xx and tegra194 linkup processing from hard IRQ to
     threaded IRQ handler

   - Add a separate lock for endpoint controller list of endpoint
     function drivers to prevent deadlock in callbacks

   - Pass events from endpoint controller to endpoint function drivers
     via callbacks instead of notifiers

  Synopsys DesignWare eDMA controller driver (acked by Vinod):

   - Fix CPU vs PCI address issues

   - Fix source vs destination address issues

   - Fix issues with interleaved transfer semantics

   - Fix channel count initialization issue (issue still exists in
     several other drivers)

   - Clean up and improve debugfs usage so it will work on platforms
     with several eDMA devices

  Baikal T-1 PCIe controller driver:

   - Set a 64-bit DMA mask

  Freescale i.MX6 PCIe controller driver:

   - Add i.MX8MM, i.MX8MQ, i.MX8MP endpoint mode DT binding and driver
     support

  Intel VMD host bridge driver:

   - Add quirk to configure PCIe ASPM and LTR. This is normally done by
     BIOS, and will be for future products

  Marvell MVEBU PCIe controller driver:

   - Mark this driver as broken in Kconfig since bugs prevent its daily
     usage

  MediaTek MT7621 PCIe controller driver:

   - Delay PHY port initialization to improve boot reliability for ZBT
     WE1326, ZBT WF3526-P, and some Netgear models

  Qualcomm PCIe controller driver:

   - Add MSM8998 DT compatible string

   - Unify MSM8996 and MSM8998 clock orderings

   - Add SM8350 DT binding and driver support

   - Add IPQ8074 Gen3 DT binding and driver support

   - Correct qcom,perst-regs in DT binding

   - Add qcom_pcie_host_deinit() so the PHY is powered off and
     regulators and clocks are disabled on late host-init errors

  Socionext UniPhier Pro5 controller driver:

   - Clean up uniphier-ep reg, clocks, resets, and their names in DT
     binding

  Synopsys DesignWare PCIe controller driver:

   - Restrict coherent DMA mask to 32 bits for MSI, but allow controller
     drivers to set 64-bit streaming DMA mask

   - Add eDMA engine support in both Root Port and Endpoint controllers

  Miscellaneous:

   - Remove MODULE_LICENSE from boolean drivers so they don't look like
     modules so modprobe can complain about them"

* tag 'pci-v6.3-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci: (86 commits)
  PCI: dwc: Add Root Port and Endpoint controller eDMA engine support
  PCI: bt1: Set 64-bit DMA mask
  PCI: dwc: Restrict only coherent DMA mask for MSI address allocation
  dmaengine: dw-edma: Prepare dw_edma_probe() for builtin callers
  dmaengine: dw-edma: Depend on DW_EDMA instead of selecting it
  dmaengine: dw-edma: Add mem-mapped LL-entries support
  PCI: Remove MODULE_LICENSE so boolean drivers don't look like modules
  PCI: hv: Drop duplicate PCI_MSI dependency
  PCI/P2PDMA: Annotate RCU dereference
  PCI/sysfs: Constify struct kobj_type pci_slot_ktype
  PCI: hotplug: Allow marking devices as disconnected during bind/unbind
  PCI: pciehp: Add Qualcomm quirk for Command Completed erratum
  PCI: qcom: Add IPQ8074 Gen3 port support
  dt-bindings: PCI: qcom: Add IPQ8074 Gen3 port
  dt-bindings: PCI: qcom: Sort compatibles alphabetically
  PCI: qcom: Fix host-init error handling
  PCI: qcom: Add SM8350 support
  dt-bindings: PCI: qcom: Add SM8350
  dt-bindings: PCI: qcom-ep: Correct qcom,perst-regs
  dt-bindings: PCI: qcom: Unify MSM8996 and MSM8998 clock order
  ...

- Add DT compatible for qcom MSM8998 (Krzysztof Kozlowski)

- Unify qcom MSM8996 and MSM8998 clock orderings (Krzysztof Kozlowski)

- Correct qcom,perst-regs (Krzysztof Kozlowski)

- Add qcom SM8350 DT binding and driver support (Dmitry Baryshkov)

- Add qcom_pcie_host_deinit() so the PHY is powered off and regulators and
  clocks are disabled on late host-init errors (Johan Hovold)

- Add IPQ8074 Gen3 port DT binding and driver support (the Gen2 port was
  already supported) (Robert Marko)

* pci/controller/qcom:
  PCI: qcom: Add IPQ8074 Gen3 port support
  dt-bindings: PCI: qcom: Add IPQ8074 Gen3 port
  dt-bindings: PCI: qcom: Sort compatibles alphabetically
  PCI: qcom: Fix host-init error handling
  PCI: qcom: Add SM8350 support
  dt-bindings: PCI: qcom: Add SM8350
  dt-bindings: PCI: qcom-ep: Correct qcom,perst-regs
  dt-bindings: PCI: qcom: Unify MSM8996 and MSM8998 clock order
  dt-bindings: PCI: qcom: Add MSM8998 specific compatible
  dt-bindings: PCI: qcom: Add oneOf to compatible match

- Add i.MX8MM, i.MX8MQ, i.MX8MP endpoint mode DT binding and driver support
  (Richard Zhu)

* pci/controller/imx6:
  PCI: imx6: Add i.MX8MP PCIe EP support
  PCI: imx6: Add i.MX8MM PCIe EP support
  PCI: imx6: Add i.MX8MQ PCIe EP support
  PCI: imx6: Add i.MX PCIe EP mode support
  misc: pci_endpoint_test: Add i.MX8 PCIe EP device support
  dt-bindings: imx6q-pcie: Add i.MX8MP PCIe EP mode compatible string
  dt-bindings: imx6q-pcie: Add i.MX8MQ PCIe EP mode compatible string
  dt-bindings: imx6q-pcie: Add i.MX8MM PCIe EP mode compatible string

- Release previously-requested DW eDMA IRQs if request_irq() fails (Serge
  Semin)

- Convert DW eDMA linked-list (ll) and data target (dt) from CPU-relative
  addresses to PCI bus addresses (Serge Semin)

- Fix missing src/dst address for interleaved transfers (Serge Semin)

- Enforce the DW eDMA restriction that interleaved transfers must increment
  src and dst addresses (Serge Semin)

- Fix some invalid interleaved transfer semantics (Serge Semin)

- Convert CPU-relative addresses to PCI bus addresses for eDMA engine
  (Serge Semin)

- Drop chancnt initialization from dw-edma-core, since it is managed by the
  dmaengine core, e.g., in dma_async_device_channel_register() (Serge Semin)

- Clean up bogus casting of debugfs_entries.reg addresses (Serge Semin)

- Ignore debugfs file/directory creation errors (Serge Semin)

- Allocate debugfs entries from the heap to prepare for multi-eDMA
  platforms (Serge Semin)

- Simplify and rework register accessors to remove another obstacle to
  multi-eDMA platforms (Serge Semin)

- Consolidate eDMA read/write channels in a single dma_device to simplify,
  better reflect the hardware design, and avoid a debugfs complaint (Serge
  Semin)

- Move eDMA-specific debugfs nodes into existing dmaengine subdirectory
  (Serge Semin)

- Fix a readq_ch() truncation from 64 to 32 bits (Serge Semin)

- Use existing readq()/writeq rather than hand-coding new ones (Serge
  Semin)

- Drop unnecessary data target region allocation in favor of existing
  dw_edma_chip members (Serge Semin)

- Use parent device in eDMA controller name to prepare for multi-eDMA
  platforms (Serge Semin)

- In addition to the existing MMIO accessors for linked list entries, add
  support for ioremapped entries for use by eDMA in Root Ports or local
  Endpoints (Serge Semin)

- Convert DW_EDMA_PCIE so it depends on DW_EDMA instead of selecting it
  (Serge Semin)

- Allow DWC drivers to set streaming DMA masks larger than 32 bits;
  previously both streaming and coherent DMA were limited to 32 bits
  because some PCI devices only support coherent 32-bit DMA for MSI (Serge
  Semin)

- Set 64-bit streaming and coherent DMA mask for the bt1 driver (Serge
  Semin)

- Add DW Root Port and Endpoint controller support for eDMA (Serge Semin)

* pci/controller/dwc:
  PCI: dwc: Add Root Port and Endpoint controller eDMA engine support
  PCI: bt1: Set 64-bit DMA mask
  PCI: dwc: Restrict only coherent DMA mask for MSI address allocation
  dmaengine: dw-edma: Prepare dw_edma_probe() for builtin callers
  dmaengine: dw-edma: Depend on DW_EDMA instead of selecting it
  dmaengine: dw-edma: Add mem-mapped LL-entries support
  dmaengine: dw-edma: Skip cleanup procedure if no private data found
  dmaengine: dw-edma: Replace chip ID number with device name
  dmaengine: dw-edma: Drop DT-region allocation
  dmaengine: dw-edma: Use non-atomic io-64 methods
  dmaengine: dw-edma: Fix readq_ch() return value truncation
  dmaengine: dw-edma: Use DMA engine device debugfs subdirectory
  dmaengine: dw-edma: Join read/write channels into a single device
  dmaengine: dw-edma: Move eDMA data pointer to debugfs node descriptor
  dmaengine: dw-edma: Simplify debugfs context CSRs init procedure
  dmaengine: dw-edma: Rename debugfs dentry variables to 'dent'
  dmaengine: dw-edma: Convert debugfs descs to being heap-allocated
  dmaengine: dw-edma: Add dw_edma prefix to debugfs nodes descriptor
  dmaengine: dw-edma: Stop checking debugfs_create_*() return value
  dmaengine: dw-edma: Drop unnecessary debugfs reg casts
  dmaengine: dw-edma: Drop chancnt initialization
  dmaengine: dw-edma: Add PCI bus address getter to the remote EP glue driver
  dmaengine: dw-edma: Add CPU to PCI bus address translation
  dmaengine: dw-edma: Fix invalid interleaved xfers semantics
  dmaengine: dw-edma: Don't permit non-inc interleaved xfers
  dmaengine: dw-edma: Fix missing src/dst address of interleaved xfers
  dmaengine: dw-edma: Convert ll/dt phys address to PCI bus/DMA address
  dmaengine: dw-edma: Release requested IRQs on failure
  dmaengine: Fix dma_slave_config.dst_addr description

- Convert dra7xx to threaded IRQ handler (Manivannan Sadhasivam)

- Move tegra194 dw_pcie_ep_linkup() to threaded IRQ handler (Manivannan
  Sadhasivam)

- Add a separate lock for the endpoint pci_epf list to avoid deadlock
  while running callbacks (Manivannan Sadhasivam)

- Use callbacks instead of notifier chains to signal events from EPC to EPF
  drivers (Manivannan Sadhasivam)

- Use link_up() callback in place of LINK_UP notifier (Manivannan
  Sadhasivam)

* pci/endpoint:
  PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
  PCI: endpoint: Use callback mechanism for passing events from EPC to EPF
  PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
  PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
  PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ

Since the DW eDMA core now supports eDMA controllers embedded in locally
accessible DW PCIe Root Ports and Endpoints, register these controllers
when possible.

To do that the DW PCIe core driver needs to perform some preparations
first. First of all, it needs to find the eDMA controller CSRs base
address, whether they are accessible over the Port Logic or iATU unrolled
space.  Afterwards it can try to auto-detect the eDMA controller
availability and number of read/write channels.  If none are found the
procedure silently returns without error.

Secondly, the platform is supposed to provide either combined or
per-channel IRQ signals.  If no valid IRQs set is found, the procedure
returns without error to be backward compatible with platforms where DW
PCIe controllers have eDMA but lack the IRQ description.

Finally, before actually probing the eDMA device we need to allocate LLP
items buffers. After that the DW eDMA can be registered. If registration is
successful, a message regarding the number of detected Read/Write eDMA
channels will be printed to the system as is done for the iATU settings.

Note: the DW PCI controller driver (either host or endpoint mode) is
currently always built-in, so if the DW eDMA core is built as a module
(CONFIG_DW_EDMA=m), eDMA controllers will not be registered even if the
dw-edma module is later loaded.

Link: https://lore.kernel.org/r/20230113171409.30470-28-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>