| Age | Commit message (Collapse) | Author |
|---|
|
Allocate doorbells from dma pools instead of the pgdir allocator.
Doorbell records remain cache-line sized coherent DMA allocations, but
their sub-allocation is now handled by the common mlx5 DMA pool
infrastructure.
This also makes doorbell allocation honor the requested NUMA node when
reusing existing backing pages. The old pgdir allocator used the
requested node only when allocating a new pgdir page; later
allocations scanned one global pgdir list and could take any pgdir with
a free entry, even if that page had been allocated for a different
NUMA node. Selecting the per-node DMA pool before sub-allocation keeps
reused doorbell records on pages allocated for the requested node.
Signed-off-by: Nimrod Oren <noren@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Link: https://patch.msgid.link/20260803132520.2891860-3-tariqt@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add per-node doorbell dma pool creation and cleanup to mdev lifecycle.
Signed-off-by: Nimrod Oren <noren@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Link: https://patch.msgid.link/20260803132520.2891860-2-tariqt@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Some newer AMD/Pensando cards have minimal memory and there are cases
where components, specifically in the control plane, need more memory.
This series adds support for host backed DMA memory that can be used
by the firmware for the previously mentioned cases.
Host memory allocation is best-effort: if some allocations fail, the
driver continues with whatever succeeded. Firmware gracefully degrades
when less memory is available than requested.
Signed-off-by: Vamsi Atluri <Vamsi.Atluri@amd.com>
Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
Link: https://patch.msgid.link/20260730-upstream_v8-v12-5-136cd174ee85@amd.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Implement PLDM FW Update in the pds_core driver using the upstream
pldmfw API. This allows updating an entire PLDM FW package at once
or updating specific firmware components by name.
Flash the entire image:
devlink dev flash pci/0000:b5:00.0 file firmware.pldmfw
Flash a specific component from the PLDM FW package:
devlink dev flash pci/0000:b5:00.0 \
file firmware.pldmfw component fw.cpld
Per-component update uses driver-defined component names (fw, fw.cpld,
etc.). Not all components support per-component update - devlink will
reject the request if the specified component cannot be updated.
Signed-off-by: Brett Creeley <brett.creeley@amd.com>
Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
Link: https://patch.msgid.link/20260730-upstream_v8-v12-3-136cd174ee85@amd.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add a new capabilities field in struct pds_core_dev_identity,
which requires bumping the identity version to 2, i.e.
PDS_CORE_IDENTITY_VERSION_2. If version 2 negotiation fails,
then quietly fall back to version 1. If version 1 negotiation
fails, then driver load will fail.
Another patch in the series will make use of the capabilities
field.
Signed-off-by: Brett Creeley <brett.creeley@amd.com>
Link: https://patch.msgid.link/20260730-upstream_v8-v12-2-136cd174ee85@amd.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
dibs->lock is initialised by dibs_dev_add(), but a dibs device can
already take interrupts before that call: ism_probe() runs
ism_dev_init(), and hence request_irq(), before it calls
dibs_dev_add(). No client can have registered a dmb at that point, so
no dmb interrupt can occur, but a GID event interrupt can, and
ism_handle_irq() takes dibs->lock unconditionally on entry, before it
inspects anything else.
Initialise the lock in dibs_dev_alloc() instead, so that it is valid as
soon as a driver can publish the device to its interrupt handler.
Fixes: cc21191b584c ("dibs: Move data path to dibs layer")
Cc: stable@vger.kernel.org
Reviewed-by: Alexandra Winter <wintera@linux.ibm.com>
Signed-off-by: Hidayath Khan <hidayath@linux.ibm.com>
Link: https://patch.msgid.link/20260730124227.167829-1-hidayath@linux.ibm.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next
Pablo Neira Ayuso says:
====================
Netfilter updates for net-next
The following patchset contains Netfilter updates for net-next:
1) Update conncount to use the original tuple after ct lookup to ensure
consistent counting, from Fernando F. Mancera.
2) Remove redundant net_device field in info structure that helps
parse the flowtable path discovery.
3) Move net_device to flowtable check to the flowtable discovery
path parser. This is preparation work to pass the tunnel dst_entry
via .fill_forward_path.
4) Update DSA .fill_forward_path to break at the user DSA, since
the conduit DSA is not used in the datapath. This slighly simplifies
the flowtable path discovery parser.
5) Do not advance index in the path stack prematurely, otherwise
it points to uninitialized slots on error. Not an issue currently
but it could be once tunnel dst_entry is passed via .fill_forward_path.
6) Pass the tunnel dst_entry via dev_fill_forward_path().
7) Update ipip and ip6ip6 tunnels to pass the dst_entry through
dev_fill_forward_path().
8) Call skb_valid_dst() before accessing skb_dst() to ensure dst_entry
is not a template.
9) Use UNACK timeout when RST packet does not match the expected
window while in ESTABLISHED state, the existing approach the CLOSE
state timeout which is only 10 seconds. Adopt a more conservative
timeout by default for this case.
* tag 'nf-next-26-07-31' of git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next:
netfilter: conntrack: tcp: use UNACK timeout for non-closing RST packets
netfilter: nf_tables: call skb_valid_dst() before skb_dst()
netfilter: flowtable: release tunnel route on error when building forward path
net: pass dst via net_device_path in dev_fill_forward_path()
net: do not advance stack index from dev_fwd_path()
net: dsa: stop at the user device in .fill_forward_path
netfilter: flowtable: consolidate flowtable device check
netfilter: flowtable: consolidate net_device field in nft_forward_info struct
netfilter: conncount: normalize tuple and zone on successful ct lookup
====================
Link: https://patch.msgid.link/20260731153402.851224-1-pablo@netfilter.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Partial pull of the nf-26-07-31 tag
Pablo says:
====================
The following patchset contains Netfilter/IPVS fixes net, this
includes fixes for ebtables nflog target, ipset hash type,
IPVS kthread estimator
1) Prevent IPVS kthread estimator from draining the est_temp_list
when netns is being dismantled. From Zhiling Zou.
2) Missing module nflog refcount bump from ebtables nflog target from
.checkentry path. Similar dependency exists already in xt_NFLOG and
nft_log. From Chengfeng Ye.
3) Use RCU to fix ipset bookkeeping of cidr values on weakly-ordered
architectures. From Jozsef Kadlecsik.
4) Use atomic64_t for set->ext_size in ipset to fix parallel inserts
and deletes racing on updating it. From Jozsef Kadlecsik.
5) Add small wrappers for hash and bucket size to prepare the update
of ipset hash set types to rhashtable, from Florian Westphal.
6) Add mtype_del_cidr_all() and use it to prepare the migration of
ipset hash types to rhashtable. From Florian Westphal.
7) Replace existing ipset call_rcu() based destruction with rcu_work
api also to ease the transition to rhashtable. Also from Florian.
8) Avoid reading the IPv4 ihl field multiple times to prevent local
attacker to cause out-of-bounds write in ip_vs_nat_icmp(), from
Julian Anastasov.
9) Restore the checksum validations that could be needed by the IPVS
FORWARD hook. Also from Julian.
====================
Link: https://patch.msgid.link/20260731151806.849724-1-pablo@netfilter.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
cxl: Rename find_cxl_port() to find_cxl_port_by_dport()
cxl: Tighten CPER kfifo registration API and symbol visibility
acpi/apei/ghes: Use raw_spinlock_t for CXL CPER work locks
cxl/ras: Fix cxl_rch_get_aer_severity() wrong severity register
cxl/ras: Fix cxl_rch_get_aer_info() out-of-bounds AER register read
|
|
Tighten the CPER protocol error kfifo registration API and symbol
visibility.
Use EXPORT_SYMBOL_FOR_MODULES() instead of EXPORT_SYMBOL_NS_GPL() for
the CPER kfifo registration symbols. This names the consuming module
explicitly and gives compile-time enforcement.
Drop the work_struct argument from the unregister path. Change the
WARN_ONCE condition to a NULL check since there is no caller pointer
to compare against anymore.
Return void from the registration helpers whose result callers ignore.
cxl_cper_register_work() keeps its int return as the only one consumed by
a caller. Flag double registration with WARN_ONCE() inside the lock
instead of returning an error.
Change cxl_ras_init() to void because there is one consumer and one
producer so the error return was unnecessary. Remove the now-dead error
check in cxl_core_init().
Co-developed-by: Terry Bowman <terry.bowman@amd.com>
Signed-off-by: Terry Bowman <terry.bowman@amd.com>
Signed-off-by: Dan Williams <djbw@kernel.org>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Alison Schofield <alison.schofield@intel.com>
Link: https://patch.msgid.link/20260803221810.3685703-5-terry.bowman@amd.com
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
|
|
Add the clock binding header and schema for the Rockchip RV1106 clock
and reset unit. The clock IDs are numbered continuously from 0, in the
same way as the RV1103B. The header also covers the MMC drive and
sample phase clocks, which the CRU provides using control registers in
the GRF region.
The RV1103 is a package variant of the RV1106 and uses the same CRU.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
Link: https://patch.msgid.link/20260729133609.3465563-2-sjg@chromium.org
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
|
|
syzbot is reporting KCOV state corruption on PREEMPT_RT kernels, for the
temporary storage used for saving/restoring remote KCOV state is currently
allocated as the per-CPU area.
On PREEMPT_RT kernels, softirq handlers run as preemptible task threads
(e.g., ksoftirqd). If a softirq context preempts a task running a remote
KCOV session, it safely saves the task's state into the per-CPU area.
However, if that softirq thread is subsequently preempted by a higher-
priority softirq thread on the same CPU, the second softirq will overwrite
the same per-CPU area, permanently destroying the original task's KCOV
state.
Fix this data corruption by moving the temporary storage from the per-CPU
area to the per-thread area. Since each softirq thread now owns its own
task context, nested softirq preemption no longer causes data overwrites.
Note that while the temporary storage is now on a per-thread basis, the
per-CPU kcov_percpu_data.lock must be retained, for we need to ensure that
kcov_remote_start() and kcov_remote_stop() operate atomically without
racing against asynchronous interrupts that manipulate the current task's
KCOV state.
It is likely that GFP_KERNEL allocation by vmalloc_node() in kcov_init()
has already called panic() before returning NULL, for there will be no
OOM-killable userspace processes when __init function of built-in module
runs. But this patch also fixes crashing the kernel when vmalloc_node()
in kcov_init() returned NULL, for kcov_init() left per-CPU irq_area == NULL
but kcov_remote_start() depends on per-CPU irq_area != NULL, resulting in
(1) doing vmalloc() in kcov_remote_start() despite !in_task() context
(2) out-of-array-bounds access if (1) succeeded but
kcov->remote_size < CONFIG_KCOV_IRQ_AREA_SIZE
(3) always leak memory allocated by (1), eventually killing all
OOM-killable userspace processes
problems.
Link: https://lore.kernel.org/43552d09-2ce2-4b19-b0d3-a2d1ab952145@I-love.SAKURA.ne.jp
Reported-by: syzbot+3f51ad7ac3ae57a6fdcc@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=3f51ad7ac3ae57a6fdcc
Reported-by: syzbot+47cf95ca1f9dcca872c8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=47cf95ca1f9dcca872c8
Reported-by: syzbot+8a173e13208949931dc7@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=8a173e13208949931dc7
Reported-by: syzbot+90984d3713722683112e@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=90984d3713722683112e
Analyzed-by: AI Mode in Google Search (no mail address)
Fixes: 5ff3b30ab57d ("kcov: collect coverage from interrupts")
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Clark Williams <williams@redhat.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Marco Elver <elver@google.com>
Cc: Mark Brown <broonie@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
- use the "typedef" keyword when describing a typedef
- add a Returns: section
to prevent these kernel-doc warnings:
Warning: include/linux/stacktrace.h:20 function parameter
'stack_trace_consume_fn' not described in 'bool'
Warning: include/linux/stacktrace.h:20 expecting prototype for
stack_trace_consume_fn(). Prototype was for bool() instead
Warning: include/linux/stacktrace.h:58 No description found for
return value of 'arch_stack_walk_reliable'
Link: https://lore.kernel.org/20260723161831.138237-1-rdunlap@infradead.org
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Marc Rutland <mark.rutland@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Correct the function parameter name to avoid kernel-doc warnings:
Warning: ./include/linux/llist.h:71 function parameter 'list' not described in 'init_llist_head'
Warning: ./include/linux/llist.h:71 Excess function parameter 'head' description in 'init_llist_head'
Link: https://lore.kernel.org/20260723165113.225098-1-rdunlap@infradead.org
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Use the correct function parameter name in rb_next_match(). Use the
struct keyword when describing struct latch_tree_ops.
Prevents the following kernel-doc warnings:
Warning: ./include/linux/rbtree.h:510 function parameter 'node' not
described in 'rb_next_match'
Warning: ./include/linux/rbtree.h:510 Excess function parameter 'tree'
description in 'rb_next_match'
Warning: ./include/linux/rbtree_latch.h:63 cannot understand function
prototype: 'struct latch_tree_ops'
Link: https://lore.kernel.org/20260722050759.3892887-1-rdunlap@infradead.org
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Renesas R-Car X5H CPG DT Binding Definitions
DT bindings and binding definitions for the Renesas R-Car X5H (R8A78000)
Clock Pulse Generator (CPG), shared by driver and DT source files.
|
|
Document support for the Renesas R-Car X5H Clock Pulse Generator, and
add binding definitions for a limited initial set of clocks.
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Tested-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://patch.msgid.link/7a33c99816c850faeec5c17532811faf78959a1d.1785941595.git.geert+renesas@glider.be
|
|
Implement UC QP creation in the RNIC HW for user API. An UC QP is exposed
as three work queues: send, receive, and memory management. The latter is
used for bind and invalidate WQEs to support memory windows.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260723115955.1859519-3-kotaranov@linux.microsoft.com
Reviewed-by: Long Li <longli@microsoft.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
CONFIG_SYSCTL used to make sense as a separate hidden bool before commit
61a47c1ad3a4 ("sysctl: Remove the sysctl system call"); it was selected
by both CONFIG_SYSCTL_SYSCALL and CONFIG_PROC_SYSCTL.
Today CONFIG_PROC_SYSCTL is the only selector, so the two are always
equal. Kill the hidden bool, rename the PROC_SYSCTL prompt to SYSCTL,
and s/CONFIG_PROC_SYSCTL/CONFIG_SYSCTL/ tree-wide.
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Convert a non-kernel-comment to use "/*" instead.
Don't use kernel-doc for the nested @type enum values since they
aren't part of the struct.
Warning: ./include/linux/sysctl.h:62 Cannot find identifier on line:
*
Warning: ./include/linux/sysctl.h:63 Cannot find identifier on line:
* "dir" originates from read_iter (dir = 0) or write_iter (dir = 1)
Warning: ./include/linux/sysctl.h:64 This comment starts with '/**', but isn't a kernel-doc comment.
* in the file_operations struct at proc/proc_sysctl.c. Its value means
Warning: ./include/linux/sysctl.h:274 Excess struct member
'type.SYSCTL_TABLE_TYPE_DEFAULT' description in 'ctl_table_header'
Warning: ./include/linux/sysctl.h:274 Excess struct member
'type.SYSCTL_TABLE_TYPE_PERMANENTLY_EMPTY' description in 'ctl_table_header'
Note: This still leaves 7 struct members in ctl_table_header that
are not described. E.g.:
Warning: include/linux/sysctl.h:274 struct member 'unregistering' not described in 'ctl_table_header'
Warning: include/linux/sysctl.h:274 struct member 'ctl_table_arg' not described in 'ctl_table_header'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Remove "_minmax" from proc_doulongvec_minmax_conv as it does not enforce
min/max limits but serves as a generic converter for unsigned long
vectors. Update function declaration in sysctl.h, definition in
sysctl.c, and caller in jiffies.c accordingly.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Make four groups in the sysctl header and document each group with an
example of how to use them.
1. proc_handler : All functions that can be passed to the proc_handler
pointer in ctl_table
2. proc handler aggregators: Functions to create proc handlers with
custom converters
3. bi-directional converters: Functions to create read/write custom
converters. Can be passed to proc handler aggregators
4. uni-directional converters: Functions to create read or write custom
converters. Can be passed as args to bi-directional converters
Use just one naming convention in the declarations: 'write' becomes
'dir' and 'buffer' becomes 'buf'.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Make do_proc_vec static and parametrize by proc_vec_type enum which
defines the type being processed and selects which converter is "live".
Signed-ness and size are calculated based on proc_vec_type and
table->data is now walked as raw bytes and advanced by the element size;
the converter still performs the actual typed load/store. Pass converter
as a union to avoid a cast from void*. The public
proc_do{int,uint,ulong}vec_conv() prototypes and all converter
signatures in kernel/, fs/ and the header are therefore unchanged.
Remove do_proc_doulongvec_minmax. proc_doulongvec_minmax_conv uses a
converter callback passed by the caller instead of conversions based on
conv{mul,div}. Create uni and bi-direction converters for milliseconds
to jiffies in proc_doulongvec_ms_jiffies_minmax; which is the only user
of proc_doulongvec_minmax_conv.
Replace do_proc_douintvec{,_w,_r} functions with a call to do_proc_vec.
Disallow vectors for uint by returning -EINVAL when more than one
element is detected.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Updates all douintvec converter function signatures to include a bool
*negp parameter. This is a preparation commit required to eventually run
all converters under the same function. The negp argument will be
ignored as it is not relevant for the uint type. Note that
do_proc_uint_conv_pipe_maxsz in pipe.c is also modified.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
cpc_write() serializes every SystemMemory write with the per-CPU
rmw_lock. The lock is required for read-modify-write fields and for
registers whose access units overlap, but not for a full-width register
in a standalone access unit.
The _CPC layout is immutable after it has been parsed. Classify each
SystemMemory register at probe time and retain locking for partial
fields, invalid access widths, and overlapping access units. Allow
standalone full-width registers to bypass the descriptor lookup and
spinlock.
Store the classification in existing structure padding so that struct
cpc_register_resource does not grow.
Signed-off-by: Christian Loehle <christian.loehle@arm.com>
Link: https://patch.msgid.link/20260803210527.1285229-3-christian.loehle@arm.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
|
|
Peter Ujfalusi <peter.ujfalusi@linux.intel.com> says:
reg_defaults must be sorted by ascending register address as
regcache_lookup_reg() locates the entries in it with bsearch(), see commit
fd80df352ba1 ("regcache: Add support for sorting defaults arrays").
These three tables have entries which are out of order, so the binary search
does not find part of them. For those registers regcache_reg_needs_sync()
cannot compare the cached value against the default and reports that a sync
is needed, so they are written to the device on every regcache_sync() even
when they were never touched.
The patches only reorder the existing entries, the text of every entry is
kept verbatim and no default value is changed. Each table was verified by
evaluating the register addresses and replaying lib/bsearch.c on them.
Entries not reachable by the binary search, per table:
cs35l41_reg 2 (of 47)
cs35l45_defaults 36 (of 73)
cs4265_reg_defaults 3 (of 16)
For cs35l45 this is nearly half of the table: the DSP1_RX*_RATE and
DSP1_TX*_RATE registers sit in the middle of it while their addresses are
far above everything else, which cuts the search off from the whole
0x4c40 - 0xf010 range.
Found by an audit of all reg_defaults tables under sound/, the SoundWire
codec drivers are fixed by a separate series.
Link: https://patch.msgid.link/20260805082413.26174-1-peter.ujfalusi@linux.intel.com
|
|
Both target Intel's skylake-driver, previously found in sound/soc/intel
which has been removed few years ago and succeeded by the avs-driver.
snd_sst_tokens.h is succeeded by uapi/sound/intel/avs/tokens.h
skl-tplg-interface.h to the best of my knowledge has no users. Its types
are reflected in sound/soc/intel/avs/messages.h and are not intended for
public use. Closest public equivalent would be the firmware's
processing modules but in such case a user has to compile against
Intel's AudioDSP headers, not Linux ones.
Signed-off-by: Cezary Rojewski <cezary.rojewski@intel.com>
Link: https://patch.msgid.link/20260805064435.429654-1-cezary.rojewski@intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
|
|
Coupling resource allocation and cgroup charging is racy when charging
succeeds, but subsequent resource allocation fails. Certain eviction
decisions are made on the basis of whether the allocating cgroup is
protected, i.e. within its min/low limits, but with the charge being
tied to resource allocation (and uncharged when the resource allocation
fails), this check is done at a point where the allocation is not actually
charged to the cgroup.
This is subtly wrong if the allocation were to cause the cgroup to exceed
the min/low protection, but it's even more wrong if the same cgroup tries
allocating multiple buffers concurrently: In this case, the min/low
protection may pass for all allocation attempts when the real min/low
protection covers only some, or potentially none of the allocated
buffers.
Instead, charge the allocation to the cgroup once and keep the charge
for as long as we try to allocate a ttm_resource, and only undo the charge
if allocating the resource is ultimately unsuccessful and we move on to
a different ttm_place.
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-4-07af96681bf8@gmx.de
|
|
This helps to find a common subtree of two resources, which is important
when determining whether it's helpful to evict one resource in favor of
another.
To facilitate this, add a common helper to find the ancestor of two
cgroups using each cgroup's ancestor array.
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-2-07af96681bf8@gmx.de
|
|
Callers can use this feedback to be more aggressive in making space for
allocations of a cgroup if they know it is protected.
These are counterparts to memcg's mem_cgroup_below_{min,low}.
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-1-07af96681bf8@gmx.de
|
|
The fixed-size components field in SNDRV_CTL_IOCTL_CARD_INFO can be too
small on systems with many audio devices.
Keep the existing struct snd_ctl_card_info ABI intact and add a new
ioctl SNDRV_CTL_IOCTL_CARD_BYTES that carries a variable-length payload
selected by a type discriminator. The first defined type
SND_CTL_CARD_BTYPE_COMPONENTS returns the full components string. The
ioctl is designed to be reused for other variable-length card payloads
in the future.
The user-space caller may set data_allocated == 0 (or data == NULL) to
query the required length; otherwise the kernel copies the payload into
the user buffer and writes back the actual length in data_len.
When the legacy components field in struct snd_ctl_card_info is
truncated, '>' is written just before the NUL terminator to signal to
user-space that the full string is available via the new ioctl.
card->components is now dynamically allocated and grown in 32 byte
increments via krealloc(), capped at 512 bytes.
Link: https://github.com/alsa-project/alsa-lib/pull/494
Suggested-by: Jaroslav Kysela <perex@perex.cz>
Suggested-by: Takashi Iwai <tiwai@suse.com>
Signed-off-by: Maciej Strozek <mstrozek@opensource.cirrus.com>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260720103505.1860399-2-mstrozek@opensource.cirrus.com
|
|
For some use-cases it does make sense to poll the rings directly instead
of relying on the interrupt. For this reason add a new flag RING_FLAG_NO_INTERRUPT
that can be used to allocate ring in polled mode.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
|
|
Patch series "mm: fix UAF caused by race between ptdump and vmap pgtable
freeing", v6.
Kernel page table walkers fall into two broad categories - those ranges
where no exclusion is required via walk_kernel_page_table_range_lockless()
and those where exclusion is required via walk_kernel_page_table_range()
or walk_page_range_debug().
The former category is used only by arm64 arch code operating on ranges it
both wholly owns and does not concurrently write.
The latter category consists of kernel page table walkers operating on
ranges that are wholly owned (but which need exclusion against concurrent
writers).
The lock used for exclusion is the mmap lock, and for kernel ranges this
is the mmap lock on init_mm.
ptdump is a special case being both the only user of
walk_page_range_debug(), and the only case in which it walks ranges it
does not own.
This presents a problem, as page tables may be freed under ptdump. And
indeed there is a use-after-free bug in the kernel as a result, which this
series addresses.
vmap promotes page tables to huge leaf entries where possible, freeing the
lower page table when it does. It does this with no meaningful locks held
against concurrent ptdump walks.
As a result, use-after-free can currently occur. This series addresses
the issue by having the vmap huge promotion logic acquire the mmap read
lock while both setting the huge page table entry and freeing the prior
leaf page table.
The ptdump code already acquires the mmap write lock, so by doing so we
ensure that the ptdump walker only ever observes either the huge page
table entry or the existing page table entry, and nothing is freed
underneath it.
A mitigation for this issue was already applied for arm64 in commit
fa93b45fd397 ("arm64: Enable vmalloc-huge with ptdump"), which this series
has to deal with carefully.
This mitigation resolves the issue by acquiring the mmap read lock on
init_mm on vmap page table free if a ptdump is in progress.
However the fix in this series would cause a deadlock if we were to simply
apply it for arm64 without also reverting the change.
This is because vmap may acquire the read lock before ptdump attempts to
acquire the write lock, which then gets queued, and rwsem starvation rules
mean that the (unacknowledged) nested mmap read lock in the arm64 code
would also block, meaning the original read lock is never released and
thus deadlock.
This series works around this by #ifndef CONFIG_ARM64'ing the mmap read
lock in vmap logic, then partially reverting commit fa93b45fd397 ("arm64:
Enable vmalloc-huge with ptdump"), keeping the enablement of huge vmap
support, and removing the ifdeffery with the partial revert patch.
There are related issues that are also addressed in this series:
* x86 page attribute logic, specifically Change Page Attributes (CPA),
implements a feature whereby huge ranges can be collapsed into huge leaf
entries. This can similarly cause a UAF when done in parallel with a
ptdump walk, so similarly acquire the init_mm mmap lock to avoid this.
* The CPA logic allows concurrent page table manipulation and CPA
collapse, meaning the former risks accessing a page table the latter
frees. Fix this by acquiring mmap write lock on init_mm across the
whole CPA collapse operation and read lock on the page table
manipulation.
* x86 and arm64 permit walks of non-kernel mm's (both allowing efi mm
walks, and in x86's case arbitrary mm's), so we ensure kernel mappings
remain stable by locking the init_mm as well as the mm being walked.
The ordering of patches is established for both strict dependencies (the
arm64 partial revert in particular has to be done after the vmap changes)
and logical ones (the non-kernel mm fix only makes sense once the vmap/CPA
fixes are in place).
This patch (of 3):
Currently there is a nasty race between ptdump and vmap when attempting to
map a huge P4D, PUD or PMD entry:
* ptdump walks kernel page table ranges it doesn't own.
* When vmap maps ranges it tries to promotes existing ones to huge page
tables in vmap_try_huge_[p4d,pud,pmd]() at P4D, PUD and PMD level,
freeing the lower page table in [p4d,pud,pmd]_free_[pud,pmd,pte]_page()
when it succeeds.
Both of these things can happen at the same time and as a result ptdump
can access a freed page table, resulting in a use-after-free and memory
corruption.
This is possible because while ptdump_walk_pgd() holds both the mem
hotplug lock and the mmap write lock before invoking
walk_page_range_debug(), vmap takes no relevant locks at all.
Fix this by holding the mmap read lock in vmap_try_huge_*() when freeing
page tables.
The read lock is sufficient: ptdump is the only walker that must be
excluded and it holds the mmap write lock. Other holders of the read lock
may run concurrently, but each exclusively owns the range it operates on
and cannot reach the page tables freed here.
We also hold the lock while assigning the huge page table entry, which
means page table walkers observe only the huge or non-huge page table
entry.
We use a trylock to prevent ptdump from blocking vmap making forward
progress. This is fine because it's an optimisation in any case, and thus
the vmap can safely proceed regardless.
All other kernel page table walkers that touch vmalloc ranges either
exclusively own the memory walked or acquire the mmap lock, so this
correctly excludes those walkers.
One wrinkle here is commit fa93b45fd397 ("arm64: Enable vmalloc-huge with
ptdump"), which addresses the issue for arm64 only by explicitly acquiring
the mmap read lock on kernel page table freeing should a concurrent ptdump
be in progress.
This is problematic as vmap may acquire the mmap read lock prior to ptdump
attempting to acquire an mmap write lock, leading to a deadlock when the
mmap read lock is slept upon on page table freeing due to rwsem
anti-starvation.
We work around this by predicating the mmap lock being taken on
!CONFIG_ARM64 for the time being.
With this patch applied, a follow up will partially revert commit
fa93b45fd397 ("arm64: Enable vmalloc-huge with ptdump") and at that stage
remove the arm64 ifdeffery.
We also update walk_page_range_debug() to assert the mmap write lock
unconditionally and update the comment here to reflect this change.
The issue has existed as long as ptdump was available and vmap freed page
tables when promoting to a huge leaf entry, that is, since commit
b6bdb7517c3d ("mm/vmalloc: add interfaces to free unmapped page table")
for huge ioremap, and commit 121e6f3258fe ("mm/vmalloc: hugepage vmalloc
mappings") for huge vmalloc.
Since the former is the earlier of the two we choose that for our Fixes
tag.
We also define a guard class for mmap_read_trylock() so we can use
cleanup.h to make the scope handling cleaner in the implementation.
This patch is based on work by David Carlier (linked), with gratitude!
Link: https://lore.kernel.org/20260723-series-vmap-race-fix-v6-0-8cc77dcc0018@kernel.org
Link: https://lore.kernel.org/20260723-series-vmap-race-fix-v6-1-8cc77dcc0018@kernel.org
Fixes: b6bdb7517c3d ("mm/vmalloc: add interfaces to free unmapped page table")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reported-by: syzbot+fd95a72470f5a44e464c@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a287988.39669fcc.33b062.00a0.GAE@google.com/T/
Link: https://lore.kernel.org/linux-mm/20260706203128.162335-1-devnexen@gmail.com/
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Kiryl Shutsemau <kas@kernel.org>
Cc: <stable@vger.kernel.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: "Borah, Chaitanya Kumar" <chaitanya.kumar.borah@intel.com>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Toshi Kani <toshi.kani@hpe.com>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Simplify try_to_unmap_one() by separating the hugetlb parts into
try_to_unmap_poisoned_hugetlb_one().
To understand the correctness of the refactoring, the following points
are noted:
1. try_to_unmap() is called for hugetlb folios only when they are
hwpoisoned.
2. A hugetlb VMA cannot be mlocked.
3. page_vma_mapped_walk() returns at most one hugetlb mapping in a VMA,
and that mapping points at the head PFN.
4. We won't ever process a softleaf entry that encodes a hugetlb folio;
hugetlb folios are never swapped out, migration entries will be
skipped (PVMW_MIGRATION not passed), and device-exclusive does not
work for hugetlb.
5. The hwpoison entry is constructed from the poisoned folio, just as in
the pre-refactor code. Any previous uffd-wp state is deliberately not
preserved for the hwpoison entry.
6. TTU_HWPOISON is always present; for it to not be present, either the
folio has to be in swapcache, or mapping_can_writeback() is true (see
unmap_poisoned_folio), none of which is true for hugetlb folios.
7. Hugetlb uses separate counters from normal rss counters, therefore
update_highwater_rss() need not be called.
While at it:
- Change VM_BUG_* to VM_WARN_*.
- Do not declare variables which are only used once.
- Constify some variables.
- Add some more VM_WARN_* to assert some invariants.
Except the above 4 points, no functional change intended.
Link: https://lore.kernel.org/20260730094559.418003-3-dev.jain@arm.com
Signed-off-by: Dev Jain <dev.jain@arm.com>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Harry Yoo <harry@kernel.org>
Cc: Jann Horn <jannh@google.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
With __install_special_mapping() moved to vma.c, vma_set_range() can be
made into a static function there and is now completely isolated from the
rest of mm.
While we're here, we can also remove the insert_vm_struct() declaration
from mm.h - the function is implemented in vma.c and already declared in
vma.h, and has no users outside of mm.
Also update the VMA userland tests to reflect this change.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-25-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
vma_assert_write_locked() and vma_assert_attached() are useful for their
own purposes, however VMA code absolutely does allow the modification of
non-write locked VMAs if they are at that point detached (i.e. unreachable
from anywhere).
It's therefore useful to be able to assert that a VMA is either
detached (modification doesn't matter) or write locked (you're explicitly
locked for modification).
Therefore introduce vma_assert_can_modify() for this purpose.
While we're here, make vma_is_attached() available generally - if
!CONFIG_PER_VMA_LOCK, then there's no sense in which a VMA is
detached (vma_mark_detached() is a noop), so have this default to true in
this case.
Also update VMA userland tests to reflect this change, correcting the
previously open-coded vma_assert_[attached,detached]() there.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-22-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
There are many instances in which linear_page_index() (as well as
linear_page_delta()) is open-coded, which is confusing and inconsistent.
Additionally, vma->vm_pgoff doesn't necessarily make it clear that this is
the page offset of the start of the VMA range.
Doing so also aids greppability.
So use vma_start_pgoff() in favour of directly accessing vma->vm_pgoff, and
linear_page_index() where we can.
This also lays the ground for future changes which will add an anonymous
page offset in order to be able to index MAP_PRIVATE-file backed anon
folios in terms of their virtual page offset.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-18-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: SJ Park <sj@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
It's often useful to obtain the number of pages a given address lies at
within a VMA.
Add linear_page_delta() to determine this and update linear_page_index() to
make use of it.
Add comments to describe both linear_page_delta() and linear_page_index().
We refer the reader to the comment for vma_start_pgoff() so we don't
duplicate information about the meaning of a VMA's page offset, which is
also less likely to result in bitrot.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-17-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
To be consistent with the newly renamed mapping_rmap_tree_*(), rename the
anon_vma_interval_tree_*() helpers to anon_rmap_tree_*().
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-11-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Rename parameters used by anon_vma_interval_tree_*() functions: 'node' to
'avc', 'start/first' to 'pgoff_start', and 'last' to 'pgoff_last' to make
clear what is being passed.
Also, express page offsets in terms of pgoff_t to be consistent.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-10-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Similar to what we did with mapping_rmap_tree*(), let's declare
anon_vma_interval_tree*() in terms of anon_vma rather than rb_root_cached.
In each case the rb tree referenced is &anon_vma->rb_root, so just pass
anon_vma and the functions can figure this out themselves.
Also update the VMA userland tests to reflect the change.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-9-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The family of vma_interval_tree_() functions manipulate the
address_space (which, of course, is generally referred to as 'mapping')
reverse mapping, but are named the 'VMA' interval tree.
VMAs may be mapped by an anon_vma, an address_space, or both. Therefore
calling the mapping interval tree a 'VMA' interval tree is rather
confusing.
This is also inconsistent with the anon_vma_interval_tree_*() functions
which explicitly reference the rmap object to which they pertain.
Rename the vma_interval_tree_*() functions to mapping_rmap_tree_*() to
correct this.
We will rename the anon rmap functions similarly in a subsequent patch.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-8-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The file-backed mapping interval tree functions vma_interval_tree_*()
accept a raw rb_root_cached pointer to determine the tree in which they are
operating.
However, in each case, this is always associated with an address_space data
type.
So simply pass a pointer to that instead to simplify the code, and more
clearly differentiate between these operations and those concerning
anonymous mappings.
While we're here, make the generated interval tree functions static as they
do not need to be used externally (any previously existing external users
have now been removed).
We also rename VMA parameters from 'node' to 'vma' as calling this a node
is simply confusing, update the input index types to pgoff_t since they
reference page offsets and rename the parameters to pgoff_start and
pgoff_last.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-6-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
We already have vma_last_pgoff() which retrieves the last page offset
within a VMA.
However, code often wishes to span a page offset range, which requires the
exclusive end of this range.
So provide this in vma_end_pgoff() and update vma_last_pgoff() to use this
function.
Also update the VMA userland tests to reflect the change.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-4-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Describe what vma_start_pgoff() and vma_last_pgoff() actually provide in
detail.
This is in order that we can differentiate this between functions that will
be added in a subsequent patch which provide a different page offset.
We go to lengths to describe the edge cases that can be run into here.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-2-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "mm: make VMA page offset handling more consistent", v2.
This series performs a series of cleanups and improvements around how the
vma->vm_pgoff field is used.
Folios belonging to file-backed mappings are simply indexed by the page
offset within the file they map (excepting pfnmap and driver mappings
which sometimes do things different).
However, anonymous folios belonging to pure anonymous mappings are indexed
by their "virtual" page offset, which is equal to addr >> PAGE_SHIFT at
the time at which the VMA was first faulted in.
The page offset of a VMA is stored in vma->vm_pgoff and indicates the page
offset of the start of the VMA range, whether it be file-backed or
anonymous.
The work here both cleans up how we reference this field, as well as
laying the foundations for a future series which addresses the
inconsistency of CoW'd folios in MAP_PRIVATE-file backed mappings, which
are indexed as if they were file-backed but behave as if they were
anonymous.
This future series will make it such that all anonymous folios are indexed
by virtual page offset whether belonging to VMAs who satisfy
vma_is_anonymous() or MAP_PRIVATE-mapped file-backed mappings.
This series:
* Exposes vma_start_pgoff() and updates the kernel to use it consistently.
* Adds and uses the useful vma_end_pgoff() helper.
* Parameterises the file-backed mapping helpers vma_interval_tree_*()
by address_space rather than rb_root_cached.
* Renames: the misleadingly-named vma_interval_tree_*() helpers to
mapping_rmap_tree_*() to be consistent with the renamed
anon_rmap_tree_*().
* Parameterises anon_rmap_tree_*() by anon_vma.
* Moves mm/interval_tree.c to the rmap section.
* Adds vmg_*() helpers for page offset.
* Clarifies the confusing vmg_adjust_set_range() function.
* Introduces linear_page_delta() to provide relative pgoff within a VMA.
* Replaces open-coded versions of linear_page_delta() and
linear_page_index() with invocations of these functions.
* Introduces and uses vma_assert_can_modify() to account for whether a VMA
can be modified (detached or write locked).
* Adds and uses vma_[add,sub]_pgoff() to adjust VMA page offset.
* Moves __install_special_mapping() to vma.c.
* Makes vma_set_range() static and internal to vma.c.
* Introduces and makes use of vma_set_pgoff().
* Fixes incorrect vma.h header inclusion.
* Defaults VMA userland tests to 64-bit vma flags size.
* Updates VMA userland tests to give better output on failure.
* Various smaller cleanups.
* Updated the vma_set_pgoff() assert to account for MAP_PRIVATE /dev/zero
correctly.
This patch (of 33):
vma_last_pgoff() already lives there, so it's a bit odd to keep
vma_start_pgoff() in mm/interval_tree.c. Move them together.
These each return unsigned long, which pgoff_t is typedef'd to. Make this
consistent and have these functions return pgoff_t instead.
Additionally, express vma_last_pgoff() in terms of vma_start_pgoff(), since
we wrap the vma->vm_pgoff access, we may as well use it here.
Also while we're here, const-ify the VMA and cleanup a bit.
Also update the VMA userland tests to reflect the change.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-0-2a5aa403d977@kernel.org
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-1-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
hmm_range_fault() requires the caller to hold the mmap read lock for the
duration of the call. This is incompatible with mappings whose fault
handler may release the mmap lock, notably userfaultfd-managed regions,
where handle_mm_fault() can return VM_FAULT_RETRY or VM_FAULT_COMPLETED
after dropping the lock. Drivers that need to populate device page tables
for such mappings have no way to do so today.
Add hmm_range_fault_unlocked_timeout() for callers that do not need to
hold mmap_lock across any work outside the HMM fault itself. The helper
takes mmap_read_lock_killable() internally, calls the common HMM fault
implementation, and releases the lock before returning if it is still
held. The timeout is specified in jiffies; passing 0 retries
indefinitely, while a non-zero timeout makes the helper return -EBUSY when
the retry budget expires. The retry deadline is set before refreshing the
notifier sequence and acquiring mmap_lock, so contended mmap_lock
acquisition is included in the retry budget. After acquiring mmap_lock,
the helper also rejects unstable address spaces before walking page
tables.
When handle_mm_fault() drops mmap_lock, or when the range is invalidated,
hmm_range_fault_unlocked_timeout() refreshes range->notifier_seq and
retries the walk internally. If the lock was dropped, the retry deadline
is also restarted because a lock-dropping fault handler made progress.
Ordinary -EBUSY retries keep the existing deadline, preserving the
caller's timeout policy for repeated mmu-notifier invalidations.
The caller only needs to perform the usual post-success
mmu_interval_read_retry() check while holding its update lock before
consuming the pfns. If mmap_lock acquisition is interrupted or a fatal
signal is pending during retry handling, -EINTR is returned instead.
The common implementation conditionally sets FAULT_FLAG_ALLOW_RETRY and
FAULT_FLAG_KILLABLE only for hmm_range_fault_unlocked_timeout(). The
existing hmm_range_fault() path still passes no locked state, does not
allow handle_mm_fault() to drop mmap_lock, and remains a thin wrapper
preserving the existing API contract for current callers.
The previous refactor that moved page fault handling out of the page-table
walk callbacks is what makes this change small. Faults now run after
walk_page_range() has unwound, with only mmap_lock held, so dropping it
does not interact with the walker's pte spinlock or hugetlb_vma_lock.
Hugetlb regions therefore participate in the unlocked path uniformly with
PTE- and PMD-level mappings; no special case is required.
Documentation/mm/hmm.rst is updated with a description of the new API and
the recommended caller pattern.
Link: https://lore.kernel.org/20260723-hmm-v10-v11-2-c55b003a4b61@gmail.com
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Dave Airlie <airlied@gmail.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lizhi Hou <lizhi.hou@amd.com>
Cc: Long Li <longli@microsoft.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Lyude <lyude@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Oded Gabbay <ogabbay@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Thomas Zimemrmann <tzimmermann@suse.de>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Liu <wei.liu@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
When damon_probe->weight is set, the weighted sum of probe hits will be
useful. It will be useful for not only the users but also DAMON internal
logics like regions merging. Implement a function for calculating it.
Link: https://lore.kernel.org/20260710134651.18084-6-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
check_accesses() DAMON ops callback returns the maximum nr_accesses of
regions. DAMON core uses it to calculate a reasonable region merge
threshold. The core will need to adjust regions for not nr_accesses but
probe hits weighted sum in future. For that, the core needs to know the
maximum weighted sum of the regions. Update the protocol for the task.
Link: https://lore.kernel.org/20260710134651.18084-5-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
prepare_access_checks() DAMON ops callback sets the monitoring sampling
address per region. In future, DAMON will be able to call only
apply_probes(). In this case, applyy_probes() may need to do the sampling
address setup, to minimize unnecessary regions iteration. Update the
protocol for the request.
Link: https://lore.kernel.org/20260710134651.18084-3-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
