| Age | Commit message (Collapse) | Author |
|---|
|
Disabling preemption or local IRQs keeps the current CPU in an RCU
read-side critical section, but in_rcu_cs() does not account for either
state. The verifier therefore rejects safe kptr accesses and invalidates
pointers when another RCU source ends.
Include preemption-disabled and IRQ-disabled state in in_rcu_cs().
Invalidate RCU-protected pointers on RCU unlock, preempt enable, or IRQ
restore only after the final protection ends.
Signed-off-by: Ning Ding <dingning04@gmail.com>
Link: https://lore.kernel.org/bpf/20260805233940.3966981-2-dingning04@gmail.com
[ kkd: Simplify was_in_rcu_cs on spin unlock and adjust the selftest. ]
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The kernel/module/dups.c file relies on the following definitions and
associated functions:
* module_param() -> linux/moduleparam.h,
* DEFINE_MUTEX() -> linux/mutex.h,
* LIST_HEAD(), list_for_each_entry(), ... -> linux/list.h,
* refcount_t, refcount_inc(), ... -> linux/refcount.h,
* MODULE_NAME_LEN -> linux/module.h,
* completion, complete_all(), ... -> linux/completion.h,
* delayed_work, work_struct, ... -> linux/workqueue.h,
* lockdep_assert_held() -> linux/lockdep.h,
* strcmp(), memcpy() -> linux/string.h,
* container_of() -> linux/container_of.h,
* DEFINE_FREE(), __free(), scoped_guard() -> linux/cleanup.h,
* kzalloc_obj(), kfree() -> linux/slab.h,
* pr_debug(), pr_warn() -> linux/printk.h,
* WARN() -> linux/bug.h,
* TASK_KILLABLE -> linux/sched.h,
* HZ -> linux/param.h.
Update the file's include list accordingly.
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
Use strcmp() instead of strlen()+memcmp() to compare module names in
kmod_dup_request_lookup(), since all strings are NUL-terminated.
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
Use scope-based cleanup helpers for kmod_dup_mutex and kmod_req to shorten
the code and to clarify where the lock is taken in
kmod_dup_request_exists_wait().
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
The kmod dups code preallocates kmod_dup_req before taking kmod_dup_mutex
to avoid allocating memory while holding the lock. This provides little
benefit, since the allocation is fast and can safely be done under the
lock. On the other hand, it leads to unnecessary allocations when the
request turns out to be a duplicate and slightly complicates the code.
Allocate kmod_dup_req only when needed and introduce a helper function
alloc_kmod_req() to initialize the structure.
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
The kmod dups code uses RCU to ensure that a kmod_dup_req instance is freed
only after it is no longer referenced. When releasing an instance, the
kmod_dup_request_delete() function removes the kmod_dup_req from the
dup_kmod_reqs list, waits via synchronize_rcu() and finally frees it.
However, this doesn't work correctly because parallel users referencing the
instance in kmod_dup_request_exists_wait() don't enter an RCU read-side
critical section. This can result in a use-after-free.
The kmod_dup_request_exists_wait() function may need to hold a valid
reference to a kmod_dup_req instance across a blocking wait until the
corresponding modprobe command completes. This makes it unsuitable for RCU.
Fix the issue by changing the lifecycle management of kmod_dup_req to use
reference counting.
Fixes: 8660484ed1cf ("module: add debugging auto-load duplicate module support")
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
When kmod_dup_request_announce() announces the completion of
a request_module() call to duplicate waiters, it queues a work item to
invoke kmod_dup_request_complete(), and only that function calls
complete_all().
This adds an arbitrary delay that is unnecessary and provides little
benefit. Call complete_all() directly from kmod_dup_request_announce()
instead.
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
Historically, various parameter-handling code kept pointers into
module::args, most notably the charp support. However, in 2009,
commit e180a6b7759a ("param: fix charp parameters set via sysfs") changed
charp parameters to kstrdup() the input string as well. As a result,
module::args now mostly wastes memory.
The last users that still pointed into module::args have now been cleaned
up, so remove this data.
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
module_total_size() returns unsigned int but uses a signed int
accumulator. While the result is numerically correct, the type
mismatch is misleading.
Change the accumulator to unsigned int to match the return type.
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
[ppavlu: correct the commit title]
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
Both try_add_failed_module() and kmod_dup_request_exists_wait() use
memcpy() with strlen() to copy module names into fixed-size
char[MODULE_NAME_LEN] buffers. Neither performs a bounds check on the
copy. Current callers always pass names originating from
mod->name (itself char[MODULE_NAME_LEN]), so this is not exploitable
today. However both functions accept a plain const char * with no
documented length contract, making them latent buffer overflows if a
future caller passes a longer string.
Replace memcpy() with strscpy() in both sites, which bounds the copy
to MODULE_NAME_LEN and always NUL-terminates.
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
The comment wrongly references to /sys/modules/XYZ/parameters/ directory
instead of /sys/module/XYZ/parameters/. Fix it.
Signed-off-by: Zenghui Yu <zenghui.yu@linux.dev>
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
The symbol lookup code assumes the queried address resides in either
MOD_TEXT or MOD_INIT_TEXT. This breaks for addresses in other module
memory regions (e.g. rodata or data), resulting in incorrect upper
bounds and wrong symbol size.
Select the module memory region the address belongs to instead of
hardcoding text sections. Also initialize the lower bound to the start
of that region, as searching from address 0 is unnecessary.
Cc: stable@vger.kernel.org
Signed-off-by: Stanislaw Gruszka <stf_xl@wp.pl>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
The verifier currently keeps a hard-coded list of kfuncs that may be
called while holding a bpf_spin_lock. With KF_SPINLOCK_SAFE available,
retaining this list creates two sources of truth and requires verifier
changes whenever another lock-safe kfunc is added.
Mark every kfunc currently accepted by kfunc_spin_allowed() with
KF_SPINLOCK_SAFE. This covers the graph, numeric iterator, resource
spin lock, arena, and stream kfuncs.
Remove the obsolete category checks and make kfunc_spin_allowed() rely
solely on the kfunc registration metadata. This preserves the behavior
of existing kfuncs while using the same mechanism for built-in and
module kfuncs.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260805153340.34776-3-kaitao.cheng@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Introduce the KF_SPINLOCK_SAFE kfunc metadata flag in BTF so kfuncs may
be explicitly marked as safe to call while holding bpf_spin_lock.
Allow kfuncs defined in kernel modules to be marked with KF_SPINLOCK_SAFE.
Example: BTF_ID_FLAGS(func, $kfunc_name, KF_SPINLOCK_SAFE)
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260805153340.34776-2-kaitao.cheng@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
DECLARE_EVENT_CLASS emitted __bpf_trace_btf_ids_<call> through
BTF_ID_LIST_GLOBAL, i.e. a global symbol named after the event class.
The class name is not unique across the kernel, so the symbol
multiply-defines whenever two translation units instantiate the same
class.
Switch to the file-local BTF_ID_LIST: the list is reached
only through the event_class_<call>.btf_ids pointer, initialised in the
same unit, so tracefs readers never reference the symbol by name and
resolve_btfids still fills the now-local .BTF_ids entries.
The handcrafted syscall classes are the one cross-unit consumer: give
them their own local BTF_ID_LIST rather than importing the generated
sys_{enter,exit} lists.
Link: https://patch.msgid.link/20260730-b4-fix_btf_tracefs-v2-1-6b66da8dc103@meta.com
Fixes: eadc0725ab8d3 ("tracing: Expose tracepoint BTF ids via tracefs")
Reported-by: Mark Brown <broonie@kernel.org>
Closes: https://lore.kernel.org/all/ff58b01c-3f5e-4d55-be82-609d2faaf12e@sirena.org.uk/
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
bloom_map_alloc() has two 32-bit-specific problems when the computed
bitmap reaches the U32_MAX fallback case.
First, BITS_TO_BYTES(U32_MAX) is evaluated with 32-bit arithmetic. The
addition performed by DIV_ROUND_UP wraps, so the map allocates only the
fixed-size bloom filter object while keeping bitset_mask == U32_MAX.
Subsequent updates can then write past the allocated object.
Second, fixing only the allocation size is not sufficient. The bloom hash
is a u32, but set_bit() takes a signed long bit number and x86 test_bit()
eventually feeds the index to variable_test_bit(long, ...). On 32-bit
kernels, hashes in [0x80000000, U32_MAX] therefore become negative bit
offsets. x86 bt/bts with a memory operand interpret those offsets relative
to the supplied base, so a map with bitset_mask == U32_MAX can read or
write before bloom->bitset even after allocating the full 512 MiB bitmap.
Keep the U32_MAX fallback, but split each hash into a word pointer and an
in-word bit number before calling test_bit() or set_bit(). The bitops
argument is then always in [0, BITS_PER_LONG - 1], while BIT_WORD(h) still
selects the intended word in the full bitmap.
Compute the bitset size from (u64)bitset_mask + 1 before passing the final
size to bpf_map_area_alloc(). This fixes the original under-allocation and
keeps the allocated storage consistent with the addressable bitset.
Exploitation note: local privilege escalation is possible on a 32-bit x86
kernel using the under-allocation bug from a binary with CAP_BPF.
Fixes: 9330986c0300 ("bpf: Add bloom filter map implementation")
Signed-off-by: Jérémy Jean <Jeremy.Jean@oss.cyber.gouv.fr>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260805060228.2703051-1-Jeremy.Jean@oss.cyber.gouv.fr
Assisted-by: Codex:gpt-5
|
|
When CONFIG_FUNCTION_ERROR_INJECTION is disabled, a sleepable tracing prog
is allowed to attach to '__x64_'-alike prefix symbols.
It is because the verifier does not verify whether the symbol is a kernel
function or a bpf prog. That said, a sleepable tracing prog is allowed to
attach to a bpf prog target whose name has '__x64_'-alike prefix.
For example, a sleepable fentry prog attaches to a '__x64_sys_nop' XDP
prog, and copies buffer from a user pointer with bpf_copy_from_user()
helper. After attaching the XDP prog to lo interface, the kernel BUG
could be triggered by 'ping -c 1 -W 1 127.0.0.1':
[ 3.460756] BUG: sleeping function called from invalid context at kernel/bpf/trampoline.c:1324
Fix it by disallowing sleepable prog always when its target
btf is not a kernel's btf.
Fixes: 16d9c5660692 ("bpf: Always allow sleepable programs on syscalls")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Viktor Malik <vmalik@redhat.com>
Link: https://lore.kernel.org/bpf/20260805150810.34907-2-leon.hwang@linux.dev
|
|
There's no need to modify the trace object bpf_get_stackid_pe, we just
need to pass the needed callchain length in separate argument.
This way we can have callchain pointers const and remove the trace->nr
modification and restoration.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-13-jolsa@kernel.org
|
|
There's no need to modify the trace object bpf_get_stack_pe, we just
need to pass the needed callchain length in separate argument.
This way we can have callchain pointers const and remove the trace->nr
modification and restoration.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-12-jolsa@kernel.org
|
|
get_perf_callchain() returns a per-CPU perf_callchain_entry buffer and
releases its recursion slot via put_callchain_entry() before returning,
so nothing keeps the entry reserved while __bpf_get_stack() consumes
it below.
A preemptible BPF program (e.g. a non-sleepable raw tracepoint program
on a PREEMPT kernel, which runs under migrate_disable() but not
preempt_disable()) can be scheduled out between obtaining the entry
and the copy. Another task scheduled on the same CPU then reuses the
same per-CPU buffer and overwrites trace->nr with a larger value.
copy_len is then computed from the inflated trace->nr and can exceed
the caller's buffer, causing an out-of-bounds write in the memcpy()
and in the build_id path.
The rcu_read_lock() taken here alone does not prevent this. It is
only taken on the may_fault path, and under CONFIG_PREEMPT_RCU it does
not disable preemption; it merely keeps perf's callchain buffer array
alive (freed via call_rcu()) and does nothing to stop another task
from reusing the entry.
Disable preemption around obtaining the callchain entry and copying
it into the caller's buffer, so the entry cannot be reused underneath
us and trace->nr stays bounded by max_depth. Build ID resolution may
fault and is therefore deferred until after preemption is re-enabled;
by then the instruction pointers have already been copied into buf,
so it operates only on that private copy. Note, preempt_disable() also
subsumes the buffer-lifetime guarantee the rcu_read_lock() provided,
since a preempt-disabled section is an RCU read-side critical section
for the callchain buffers' call_rcu() reclaim.
Fixes: c195651e565a ("bpf: add bpf_get_stack helper")
Reported-by: Tao Chen <chen.dylane@linux.dev>
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <borkmann@iogearbox.net>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-11-jolsa@kernel.org
Closes: https://lore.kernel.org/bpf/20260206090653.1336687-1-chen.dylane@linux.dev/
[ changed Fixes: commit ]
|
|
Both bpf_get_task_stack and bpf_get_task_stack_sleepable helpers that
use __bpf_get_task_stack have buf defined as ARG_PTR_TO_UNINIT_MEM
argument and we should initialize the buf on every return path.
Adding missing buf memset for __bpf_get_task_stack fail paths. This
provides deterministic buffer contents, which is useful when the buffer
is used directly as a map key.
Fixes: 06ab134ce8ec ("bpf: Refcount task stack in bpf_get_task_stack")
Fixes: b992f01e6615 ("bpf: Guard against accessing NULL pt_regs in bpf_get_task_stack()")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-10-jolsa@kernel.org
|
|
Now with the new callchain_* helper functions we can process trace_in
case directly in bpf_get_stack_pe function and remove it from
__bpf_get_stack which makes things easier for preemption fix in
following change.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-9-jolsa@kernel.org
|
|
The new callchain_finalize function calls the build-id retrieval
(if needed) and zeroes the buffer. This makes things easier for
preemption fix in following change.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-8-jolsa@kernel.org
|
|
The new callchain_store function stores trace entries buffer into
user supplied buffer. It covers both just-ip and buildid data.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-7-jolsa@kernel.org
|
|
The get_perf_callchain call needs disabled preemption plus we need
it disabled as long as we access its returned trace entries buffer.
Note the bpf_get_stackid_pe function is executed already with
preemption disabled.
Fixes: d5a3b1f69186 ("bpf: introduce BPF_MAP_TYPE_STACK_TRACE")
Reported-by: Tao Chen <chen.dylane@linux.dev>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-6-jolsa@kernel.org
Closes: https://lore.kernel.org/bpf/20260206090653.1336687-2-chen.dylane@linux.dev/
|
|
Replacing __bpf_get_stackid calls with sequence of following functions:
stackid_fastpath
stackid_new_bucket
stackid_install
This makes code more structured and allows us to easily disable
preemption only in bpf_get_stackid in following changes.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-5-jolsa@kernel.org
|
|
The new stackid_new_bucket allocates the new bucket and initializes it
with the trace data.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-4-jolsa@kernel.org
|
|
The new stackid_fastpath does the fast stack hash and trace check, that
does not need new bucket allocation. It covers both just-ip and buildid
code paths.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-3-jolsa@kernel.org
|
|
The new stackid_init function stores all the necessary bits for stackid
trace and it will be used by other functions in following changes.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-2-jolsa@kernel.org
|
|
syzbot is reporting KCOV state corruption on PREEMPT_RT kernels, for the
temporary storage used for saving/restoring remote KCOV state is currently
allocated as the per-CPU area.
On PREEMPT_RT kernels, softirq handlers run as preemptible task threads
(e.g., ksoftirqd). If a softirq context preempts a task running a remote
KCOV session, it safely saves the task's state into the per-CPU area.
However, if that softirq thread is subsequently preempted by a higher-
priority softirq thread on the same CPU, the second softirq will overwrite
the same per-CPU area, permanently destroying the original task's KCOV
state.
Fix this data corruption by moving the temporary storage from the per-CPU
area to the per-thread area. Since each softirq thread now owns its own
task context, nested softirq preemption no longer causes data overwrites.
Note that while the temporary storage is now on a per-thread basis, the
per-CPU kcov_percpu_data.lock must be retained, for we need to ensure that
kcov_remote_start() and kcov_remote_stop() operate atomically without
racing against asynchronous interrupts that manipulate the current task's
KCOV state.
It is likely that GFP_KERNEL allocation by vmalloc_node() in kcov_init()
has already called panic() before returning NULL, for there will be no
OOM-killable userspace processes when __init function of built-in module
runs. But this patch also fixes crashing the kernel when vmalloc_node()
in kcov_init() returned NULL, for kcov_init() left per-CPU irq_area == NULL
but kcov_remote_start() depends on per-CPU irq_area != NULL, resulting in
(1) doing vmalloc() in kcov_remote_start() despite !in_task() context
(2) out-of-array-bounds access if (1) succeeded but
kcov->remote_size < CONFIG_KCOV_IRQ_AREA_SIZE
(3) always leak memory allocated by (1), eventually killing all
OOM-killable userspace processes
problems.
Link: https://lore.kernel.org/43552d09-2ce2-4b19-b0d3-a2d1ab952145@I-love.SAKURA.ne.jp
Reported-by: syzbot+3f51ad7ac3ae57a6fdcc@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=3f51ad7ac3ae57a6fdcc
Reported-by: syzbot+47cf95ca1f9dcca872c8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=47cf95ca1f9dcca872c8
Reported-by: syzbot+8a173e13208949931dc7@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=8a173e13208949931dc7
Reported-by: syzbot+90984d3713722683112e@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=90984d3713722683112e
Analyzed-by: AI Mode in Google Search (no mail address)
Fixes: 5ff3b30ab57d ("kcov: collect coverage from interrupts")
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Clark Williams <williams@redhat.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Marco Elver <elver@google.com>
Cc: Mark Brown <broonie@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
parse() hands nla_strscpy() len as dstsize, and nla_strscpy() copies at
most dstsize - 1 bytes. When the attr payload comes in without a trailing
NUL, srclen == len >= dstsize and the last character of the cpumask string
gets cut off. Register "0-15" and you are silently listening on "0-1",
exit data for the rest never shows up.
The bug only bites when the sender doesn't NUL terminate the payload;
senders that include the NUL were always fine (srclen gets decremented for
the trailing NUL, so srclen < dstsize). Thats probably why this survived
20 years. And the policy is NLA_STRING, not NLA_NUL_STRING, so a payload
without the trailing NUL is legit input here.
Skip the kmalloc/nla_strscpy dance entirely and use nla_strdup(), which
already allocates srclen + 1 and terminates. The nla_len() bounds checks
stay as they were.
Link: https://lore.kernel.org/EC49FE41-7F5F-41E0-A07A-ABEB8ECA514D@grrlz.net
Fixes: f9fd8914c1ac ("[PATCH] per-task delay accounting taskstats interface: control exit data through cpumasks")
Signed-off-by: Bradley Morgan <include@grrlz.net>
Reported-by: Oleg Deomi <oleg.deomi@gmail.com>
Closes: https://lore.kernel.org/CAByWkfZ6b1=3H9pwkz-dDQOs9cZaF-HYQ6b9Yb0=Hq2r1Vv_Pw@mail.gmail.com
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Once destroy() returns the stack slot is no longer tracked as iterator
state, so zeroing it is dead work. Make the kfunc a no-op and inline the
call to a single BPF_JA 0 (the fixup can't drop the instruction outright,
so emit a nop; the JITs elide it).
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-5-puranjay@kernel.org
|
|
bpf_iter_num_next() runs on every bpf_for() iteration, so inlining it
drops a call from the loop body. R1 points to the iterator; the returned
pointer to s->cur is R1 itself, since s->cur is first.
s->cur and s->end are int, so the kfunc's s->cur + 1 >= s->end is a
signed 32-bit compare and the inlined code needs no sign extension.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-4-puranjay@kernel.org
|
|
bpf_for() expands to the bpf_iter_num_{new,next,destroy}() kfuncs, which
the verifier emits as regular calls. They are tiny and only touch the
8-byte on-stack iterator state, so open-code them in bpf_fixup_kfunc_call()
like the other special kfuncs there.
Start with bpf_iter_num_new(): R1 points to the iterator, R2/R3 hold
start/end. The inlined sequence mirrors the kfunc and returns the same
-EINVAL / -E2BIG / 0.
start > end is rejected first, so end - start fits in a u32; range-check
it as u32 on both sides ((u32)(end - start) in the kfunc). A movsx-based
check would emit a cpuv4 instruction that some JITs (x86-32, mips32,
sparc64) decode as a plain move and get wrong.
The emitted instructions are plain BPF, so the interpreter path stays
correct and no jit_required marking is needed.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-3-puranjay@kernel.org
|
|
The comment on the s->cur + 1 >= s->end check claims the (s64) cast is
needed to avoid overflow when s->cur == s->end == INT_MAX. It isn't:
s->cur + 1 is computed in int and wraps before the cast, so the cast
changes nothing (INT_MAX + 1 compares the same either way).
The wraparound is the point. bpf_iter_num_new() sets s->cur = start - 1,
which wraps to INT_MAX for start == INT_MIN, and the wrapping s->cur + 1
brings it back to start. (s64)s->cur + 1 would instead break iterators
starting at INT_MIN.
Drop the cast and reword the comment. No functional change; the wrap is
well-defined under -fno-strict-overflow.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-2-puranjay@kernel.org
|
|
CONFIG_SYSCTL used to make sense as a separate hidden bool before commit
61a47c1ad3a4 ("sysctl: Remove the sysctl system call"); it was selected
by both CONFIG_SYSCTL_SYSCALL and CONFIG_PROC_SYSCTL.
Today CONFIG_PROC_SYSCTL is the only selector, so the two are always
equal. Kill the hidden bool, rename the PROC_SYSCTL prompt to SYSCTL,
and s/CONFIG_PROC_SYSCTL/CONFIG_SYSCTL/ tree-wide.
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
cad_pid is global, and kill_cad_pid() is only used in the root namespace.
However, due to pid_table_root_permissions(), a non-root user can unshare
pid/user namespaces and modify it from the child namespace. This makes no
sense and is simply wrong.
Move it to kern_reboot_table[] where it logically belongs; this ensures
that only GLOBAL_ROOT_UID can read/modify this sysctl.
Note that this patch doesn't preserve "#ifdef CONFIG_PROC_SYSCTL" around
the "cad_pid"; CONFIG_PROC_SYSCTL selects CONFIG_SYSCTL, so it is always
set when kern_reboot_table[] is compiled.
Cc: stable@vger.kernel.org
Fixes: e054bcbe7e7a ("sysctl: move cad_pid into kernel/pid.c")
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Alexey Gladkov <legion@kernel.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Reviewed-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Fix kernel-doc warnings in kernel/sysctl.c by adding Returns.
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Add colon ":" after argument name where it is missing
Add doc for proc_int_conv and proc_dointvec_conv
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Remove "_minmax" from proc_doulongvec_minmax_conv as it does not enforce
min/max limits but serves as a generic converter for unsigned long
vectors. Update function declaration in sysctl.h, definition in
sysctl.c, and caller in jiffies.c accordingly.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Make do_proc_vec static and parametrize by proc_vec_type enum which
defines the type being processed and selects which converter is "live".
Signed-ness and size are calculated based on proc_vec_type and
table->data is now walked as raw bytes and advanced by the element size;
the converter still performs the actual typed load/store. Pass converter
as a union to avoid a cast from void*. The public
proc_do{int,uint,ulong}vec_conv() prototypes and all converter
signatures in kernel/, fs/ and the header are therefore unchanged.
Remove do_proc_doulongvec_minmax. proc_doulongvec_minmax_conv uses a
converter callback passed by the caller instead of conversions based on
conv{mul,div}. Create uni and bi-direction converters for milliseconds
to jiffies in proc_doulongvec_ms_jiffies_minmax; which is the only user
of proc_doulongvec_minmax_conv.
Replace do_proc_douintvec{,_w,_r} functions with a call to do_proc_vec.
Disallow vectors for uint by returning -EINVAL when more than one
element is detected.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Updates all douintvec converter function signatures to include a bool
*negp parameter. This is a preparation commit required to eventually run
all converters under the same function. The negp argument will be
ignored as it is not relevant for the uint type. Note that
do_proc_uint_conv_pipe_maxsz in pipe.c is also modified.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Move the converter assignment out of do_proc_dointvec into the caller.
Both the test for NULL and the assignment are meant to stay within the
sysctl.c context. This is in preparation of using a typed macro to for
the integer proc vector function.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok().
For a load-acquire that fetches into its own source register (dst_reg ==
src_reg), check_load_mem() overwrites src_reg's type with the type of the
loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the
source pointer and fails to reject the disallowed types (ctx, pkt,
flow_keys, sock).
Since bpf_convert_ctx_accesses() does not rewrite atomic loads, the raw
access to the underlying kernel object is left in place. The destination
type is taken from the ctx access itself, so a load-acquire of the sk
field of struct __sk_buff for example leaves the register typed as
PTR_TO_SOCK_COMMON_OR_NULL, which type_is_sk_pointer() does not match
either, while it actually holds unconverted struct sk_buff bytes. Once
the NULL check has passed this is a type confusion, not just a leak of
kernel data.
Validate src_reg with check_reg_arg() and check the source pointer type
with atomic_ptr_type_ok() before the load again, mirroring
check_atomic_rmw(). Out-of-range register numbers are already rejected
earlier by check_and_resolve_insns() (commit 503d21ef8eac ("bpf: Do
register range validation early")), and the only exemption there,
is_stack_arg_ldx(), requires BPF_LDX | BPF_MEM | BPF_DW and thus never
matches a BPF_ATOMIC insn. atomic_ptr_type_ok() can therefore not
dereference register state out of bounds, that is, the out-of-bounds
read addressed by the Fixes commit below does not reappear (as proven
also via selftest).
Fixes: c03bb2fa327e ("bpf: Fix out-of-bounds read in check_atomic_load/store()")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260804201917.253491-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
This helps to find a common subtree of two resources, which is important
when determining whether it's helpful to evict one resource in favor of
another.
To facilitate this, add a common helper to find the ancestor of two
cgroups using each cgroup's ancestor array.
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-2-07af96681bf8@gmx.de
|
|
Callers can use this feedback to be more aggressive in making space for
allocations of a cgroup if they know it is protected.
These are counterparts to memcg's mem_cgroup_below_{min,low}.
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-1-07af96681bf8@gmx.de
|
|
Be consistent in using vma_start_pgoff() and vma_end_pgoff(), which clearly
indicates which part of the VMA the page offset refers to and aids
greppability.
This is part of a broader series laying the ground to provide a virtual
page offset for MAP_PRIVATE-file backed anon folios.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-19-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> # for kernel/dma
Reviewed-by: Gregory Price <gourry@gourry.net>
Acked-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The family of vma_interval_tree_() functions manipulate the
address_space (which, of course, is generally referred to as 'mapping')
reverse mapping, but are named the 'VMA' interval tree.
VMAs may be mapped by an anon_vma, an address_space, or both. Therefore
calling the mapping interval tree a 'VMA' interval tree is rather
confusing.
This is also inconsistent with the anon_vma_interval_tree_*() functions
which explicitly reference the rmap object to which they pertain.
Rename the vma_interval_tree_*() functions to mapping_rmap_tree_*() to
correct this.
We will rename the anon rmap functions similarly in a subsequent patch.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-8-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The file-backed mapping interval tree functions vma_interval_tree_*()
accept a raw rb_root_cached pointer to determine the tree in which they are
operating.
However, in each case, this is always associated with an address_space data
type.
So simply pass a pointer to that instead to simplify the code, and more
clearly differentiate between these operations and those concerning
anonymous mappings.
While we're here, make the generated interval tree functions static as they
do not need to be used externally (any previously existing external users
have now been removed).
We also rename VMA parameters from 'node' to 'vma' as calling this a node
is simply confusing, update the input index types to pgoff_t since they
reference page offsets and rename the parameters to pgoff_start and
pgoff_last.
No functional change intended.
Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-6-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: SJ Park <sj@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Liam R. Howlett (Oracle) <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
mm/internal.h becomes more and more bloated.
Move declarations related to vmalloc to a new mm/vmalloc.h header.
No functional changes.
Link: https://lore.kernel.org/20260709-internal-h-v2-3-695631425968@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Muchun Song <muchun.song@linux.dev>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Pratyush Yadav <pratyush@kernel.org>
Acked-by: SJ Park <sj@kernel.org>
Cc: Alexander Graf <graf@amazon.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Marco Elver <elver@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
