This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci

This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.

Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.

Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>

A GPU reset can race with BO teardown after the BO's GTT resource has
been marked for deletion but before its drm_mm node is removed. In this
window, amdgpu_gtt_mgr_recover() can treat the node as a live BO and try
to restore its GART mapping while its TT backing is being destroyed.

Recolor the GTT node from amdgpu_bo_delete_mem_notify() so that recovery
skips it, reusing the existing color for ranges without a BO. The range
stays allocated until the resource is freed.

This prevents reset recovery from accessing a BO whose backing storage
is no longer valid.

v2: refine commit message. (David Francis)
v3: Remove new BO color. (Christian)

Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Perry Yuan <perry.yuan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm-misc-next for v7.3:

UAPI Changes:
- Remove the default udmabuf size limit of 64MB.

Cross-subsystem Changes:
- Add dmemcg support for eviction, and hook it up for amdgpu and xe.

Core Changes:
- Changes to TTM to be more aggressive when allocating below protection limit!
- Improve dt binding documentation for renesas.
- Add helper to convert physical address back to buddy block,
  add that to and improve its kunit test.

Driver Changes:
- Assorted small fixes to ti-sn65dsi86, panthor, imagination, omapdrm,
  bridge/synopsys, panel-edp, ssd130x, panel/tdo-tl070wsh30.
- Add Sharp LQ120P1JX51 panel.
- Add dmemcg support to nouveau.
- Various updates and improvements to sun4i, among which YUV and 4k support.

Signed-off-by: Dave Airlie <airlied@redhat.com>

From: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patch.msgid.link/917d462a-8976-4a15-bec4-4513ec51c5c0@linux.intel.com

Register the VRAM manager with the dmem cgroup reclaim infrastructure
so that lowering dmem.max below current VRAM usage triggers TTM
eviction rather than failing with -EBUSY.

Guard place->flags in amdgpu_ttm_bo_eviction_valuable() against NULL,
as the TTM reclaim path passes a NULL place in cgroup drain mode.

Use drmm_cgroup_register_region() so that the region is automatically
unregistered at DRM device release, after drm_dev_unplug() has already
made drm_dev_enter() return false.  The drm_dev_enter/exit guard in the
reclaim callback ensures no reclaim work touches the TTM manager after
driver unbind, closing the window between vram_mgr_fini() (called from
drm_driver.release) and the drmm cleanup that unregisters the region.

v3:
- Rebased on fix for uninitialized list and buddy allocator on the
  drmm_cgroup_register_region() error path.

v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Clear the reclaim callback in amdgpu_vram_mgr_fini() to prevent
  use-after-free if cgroup reclaim is triggered after driver unbind
  while userspace holds an open DRM file descriptor. (Sashiko-bot)
- Switch from drmm_cgroup_register_region() to the raw
  dmem_cgroup_register_region() and store the region in
  amdgpu_vram_mgr.cg_region. Call dmem_cgroup_unregister_region()
  in amdgpu_vram_mgr_fini() after ttm_resource_manager_evict_all()
  to drain in-flight reclaim callbacks, and clear man->cg afterwards.
  This is required because amdgpu's vram manager fini is called
  explicitly during driver unbind, which may precede the DRM device
  release and thus precede any drmm-based cleanup. (Sashiko-bot)

v6:
- Fix mgr->cg_region never being assigned, so
  dmem_cgroup_unregister_region() in fini silently no-ops on NULL
  and leaks the region. (Sashiko-bot)
- Reorder fini to call set_used(false) and evict_all() before
  dmem_cgroup_unregister_region(), so ttm_resource_free() can
  uncharge via man->cg during eviction; clear man->cg after
  unregister. (Sashiko-bot)

v7:
- Move dmem_cgroup_unregister_region() before the early return on
  evict_all() failure; not doing so leaves a dangling reclaim callback
  pointing to the partially-torn-down VRAM manager, causing a
  use-after-free when the cgroup later triggers reclaim. (Sashiko-bot)
- Switch back to drmm_cgroup_register_region() with a drm_dev_enter/
  exit guard in the reclaim callback (matching xe), rather than manual
  register/unregister.  drm_dev_unplug() fires before vram_mgr_fini(),
  so drm_dev_enter() returning false prevents any reclaim from touching
  the manager during teardown.  This also fixes the "vram" name
  collision on multi-GPU systems, since drmm_cgroup_register_region()
  automatically prefixes with "drm/<pci-addr>/". (Sashiko-bot)

v8:
- Move the dmem cgroup region teardown back into
  amdgpu_vram_mgr_fini(): register the region with
  dmem_cgroup_register_region() (keeping the "drm/<unique>/vram"
  prefix), store it in amdgpu_vram_mgr.cg_region, and unregister it
  explicitly after ttm_resource_manager_evict_all(). The v7
  drmm-only approach left a use-after-free on the probe-error
  teardown path: drm_dev_unplug() is not called there, so the
  drm_dev_enter() guard stays open while drm_driver.release destroys
  the manager before the drmm action unregisters the region. The
  explicit unregister drains in-flight reclaim on every teardown
  path; the drm_dev_enter() guard is kept as defense against reclaim
  after unplug. (Sashiko-bot)

v9:
- Don't leak a gpu buddy manager on cgroup init failure. (Sashiko-bot)
- Acquire a runtime PM reference (pm_runtime_get_sync/put_autosuspend)
  around the TTM reclaim call in amdgpu_vram_mgr_dmem_reclaim().
  (Sashiko-bot)

Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-7-thomas.hellstrom@linux.intel.com
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>

amdgpu_pci_remove() calls drm_dev_unplug() before invoking the fini
routines. After drm_dev_unplug() the drm_dev_enter() guard in
amdgpu_ttm_fini() always returns false, so iounmap() for
aper_base_kaddr is silently skipped. On connected_to_cpu hardware
ioremap_cache() maps the aperture as WB; when iounmap() is skipped the
stale WB PAT entry persists. On reload IP discovery's
memremap(MEMREMAP_WC) on the same aperture range hits a WB/WC conflict,
producing an ioremap error and failing re-probe.

Remove the drm_dev_enter() guard and call iounmap() unconditionally.
The aperture mapping is plain MMIO and does not require device-presence
protection. Surprise-removal cleanup of aper_base_kaddr is already
handled unconditionally by amdgpu_device_unmap_mmio().

Fixes: 62d5f9f7110a ("drm/amdgpu: Unmap MMIO mappings when device is not unplugged")
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

devres teardown is LIFO. The aperture devres node was registered after
the DRM device node, so devres_release_all() unmaps the aperture before
the DRM device release callback fires amdgpu_device_fini_sw(). IP
sw_fini callbacks (e.g. vcn_v4_0_sw_fini) write to fw_shared through a
pointer derived from aper_base_kaddr, causing a kernel page fault on
probe failure / rollback:

  BUG: unable to handle page fault ... PMD 0
  RIP: vcn_v4_0_sw_fini+0x7b/0x170 [amdgpu]
  Call Trace:
    amdgpu_device_fini_sw
    amdgpu_driver_release_kms
    devm_drm_dev_init_release
    devres_release_all

This reverts commit d871e99879cb5fd1fa798b006b4888887e63a17a.

Fixes: d871e99879cb ("drm/amdgpu: fix aperture mapping leak")
Reported-by: Yuansheng Mao <yuansheng.mao@amd.com>
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

SVM BOs are now migrated back to system memory synchronously from the TTM
eviction path (svm_range_evict_svm_bo), so the per-svm_bo eviction fence
is no longer used.

Remove the eviction fence from svm_range_bo, drop the
amdgpu_amdkfd_fence->svm_bo back pointer and the
amdgpu_amdkfd_evict_svm_bo() helper, and stop special-casing svm_bo
fences in the KFD fence enable_signaling and check_mm paths. Embed struct
amdgpu_bo directly in svm_range_bo with a dedicated svm_range_bo_destroy()
callback, and keep the owning mm via mmgrab()/mmdrop() instead of through
the fence.

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

svm_range_evict_svm_bo_worker() migrated an SVM BO's pages back to system
memory from a work item that took mmap_read_lock. When an mmap writer was
pending, that read lock blocked behind the writer while the thread
allocating a new migration VRAM BO waited on this BO's eviction fence - a
circular wait that hung the SVM workers.

Evict the SVM BO synchronously from the TTM eviction path
(amdgpu_ttm_bo_eviction_valuable) instead of deferring to a work item.
The BO is already reserved and the lock order is mmap_lock -> BO
reservation, so only trylock the owning process's mmap lock; on
contention return -EBUSY so TTM skips this BO. This removes the eviction
work item and the enable_signaling path, so no worker can block on
mmap_read_lock.

The SVM BO uses AMDGPU_GEM_CREATE_DISCARDABLE, so ttm_bo_evict takes the
pipeline_gutting path and skips allocating a system memory placement.
That would be wasted work, since svm_migrate_vram_to_ram allocates the
system pages and copies the data back itself.

Eviction now migrates ranges directly, so it must serialize with the
owning process: it trylocks migrate_mutex under svm_bo->list_lock before
unlinking the range, and svm_range_free() unlinks the range then waits on
migrate_mutex, so a concurrent eviction cannot free a range under it.

Drop the mm reference with mmput_async so exit_mmap() does not run under
the BO reservation.

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

On APUs, the GTT pool is backed by system RAM, but its size is not bound
to the non-carveout memory that actually backs it. A user can end up
with GTT + VRAM exceeding total physical memory through the following
sequence:

 - Have a large non-carveout memory space (~128GB) and accordingly set a
   large GTT (~100GB) via the ttm module parameter.
 - Lower the non-carveout memory space in BIOS by increasing the UMA
   Frame Buffer Size (VRAM) to 64GB.
 - The previously set GTT value (~100GB) persists, even though the new
   non-carveout space (64GB) can no longer back it.

This leads to a case where kernel reports GTT (100GB) + VRAM (64GB)
despite the sum being greater than total physical memory (128GB).

Cap the GTT size to totalram_pages() on APUs. totalram_pages() already
excludes the VRAM carveout, so the resulting GTT can never exceed the
system RAM that actually backs it.

Signed-off-by: Harkirat Gill <harkirat.gill@amd.com>
Reviewed-by: David Francis <David.Francis@amd.com>
Assisted-by: Claude:claude-opus-4
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Currently when amdgpu copies or fills a buffer,
it uses the maximum byte count supported by the
copy engine (SDMA). This is problematic when the
maximum byte count is not aligned to 256 bytes
because it then can't use all memory channels
optimally and can cause the SDMA to operate in its
slower byte mode (as opposed to the faster dword mode).

For example, when copying a 10 MiB buffer on SDMA v2.4,
we get 5 packets copying 2097151 bytes and 1 packet copying
the remaining 5 bytes. All 6 packets are misaligned and operate
in byte mode.
For this example, the optimal solution would be to have
5 packets each copying 2096896 bytes and 1 last packet to
copy the remaining 1280 bytes, in which case all 6 packets
are aligned to 256 bytes and operate in dword mode.

Let's use the following scheme from now on:

When byte count is dword-aligned and fits a single packet,
just emit a single packet.

Otherwise, align the copy packet size down to 256 bytes
for optimal use of memory channels and to ensure the HW
can use the dword mode.

This assumes that the starting addresses of BOs are always
dword aligned, which should be the case for every copy
operation in the kernel, because the kernel always copies
pages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>