| Age | Commit message (Collapse) | Author |
|---|
|
No user currently relies on sparse CPU masks, but the descriptor logic already
supports them via linear fallback. Remove the arbitrary limitation.
Link: https://patch.msgid.link/20260709160017.1729517-4-vdonnefort@google.com
Fixes: 2e67fabd8b77 ("ring-buffer: Introduce ring-buffer remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Pre-calculate desc->struct_len up-front in trace_remote_alloc_buffer()
with trace_buffer_desc_size() to fix double-counting.
While at it, use the accessor __first_ring_buffer_desc().
Link: https://patch.msgid.link/20260709160017.1729517-3-vdonnefort@google.com
Fixes: 96e43537af54 ("tracing: Introduce trace remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
If page allocation fails in trace_remote_alloc_buffer(), desc->nr_cpus
is not yet incremented for the current CPU. As a consequence, on error,
half-allocated rb_desc will not be freed in trace_remote_free_buffer().
Increment desc->nr_cpus as soon as the first allocation for the current
CPU has succeeded.
Link: https://patch.msgid.link/20260709160017.1729517-2-vdonnefort@google.com
Fixes: 96e43537af54 ("tracing: Introduce trace remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When switching a task's sched_class away from sched_ext, we get the
following sequence of events in __sched_setscheduler():
sched_change_begin()
switched_from_scx()
scx_disable_task(p)
ops.disable(p)
__setscheduler_params()
set_load_weight()
reweight_task_scx(p)
ops.set_weight(p)
p->sched_class = next_class;
sched_change_end()
...
Notably, ops.set_weight() is called _after_ ops.disable().
This violates the expected semantics of the callbacks, the expectation
being that ops.disable() can only be followed by ops.exit_task() or
ops.enable().
Skipping the weight adjustment for disabled tasks should be harmless
since the weight will be recalculated in scx_enable_task() if the task
ever rejoins SCX.
Fixes: 637b0682821b ("sched: Fold sched_class::switch{ing,ed}_{to,from}() into the change pattern")
Cc: stable@vger.kernel.org # v6.19+
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
sched_ext schedulers have state in BPF programs and kernel. scx_dump
provides kernel state and BPF program state on error, but this is static
in what it can provide.
Add a sched_ext_exit tracepoint in scx_claim_exit() so that BPF programs
can dynamically inspect scheduler specific state at the moment of exit.
Pass the exiting scx_sched so attached programs can read its state, and,
since exits propagate through a hierarchy of sub-schedulers, identify
which scheduler each event belongs to.
Signed-off-by: Pat Somaru <patso@likewhatevs.io>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
map_range() reads rb->aux_pages[], rb->aux_nr_pages and rb->aux_pgoff via
perf_mmap_to_page() while holding only event->mmap_mutex. Those fields are
serialized by rb->aux_mutex, and mmap_mutex is per event.
Thus, two events sharing one rb via PERF_EVENT_IOC_SET_OUTPUT can race
rb_alloc_aux() with map_range(), leading to a page-UAF scenario as follows:
CPU 0 CPU 1
===== =====
rb_alloc_aux() map_range()
[1]: allocate rb->aux_pages[0]
[2]: rb->aux_nr_pages++
[3]: perf_mmap_to_page()
returns rb->aux_pages[0]
[4]: map it as VM_PFNMAP
[5]: rb->aux_pgoff = 1
munmap the page
[6]: free rb->aux_pages[0]
Pages mapped as VM_PFNMAP have no refcount protection, so CPU 1 holds a
mapping to a freed physical frame.
Fix this by taking rb->aux_mutex across the page walk in map_range().
Fixes: b709eb872e19 ("perf: map pages in advance")
Signed-off-by: Lee Jia Jie <jiajie.lee@starlabs.sg>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
|
|
Now that nothing inflates tick_length beyond tick_length_base (the
adjtime path was converted to use time_offset in the previous commit),
the two fields are always equal.
Remove tick_length_base and keep tick_length as the single field.
Remove the per-second reset and the delta update in
ntp_update_frequency() since there is no separate base to track.
No functional change intended.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.6-1m
Link: https://patch.msgid.link/20260621220051.1030462-8-dwmw2@infradead.org
|
|
time_offset (the exponential PLL phase slew) and time_adjust (the
linear adjtime() slew) can be asked to move the clock in opposite
directions. second_overflow() folds only their *net* into the per-tick
skew_delta, so the cancelling overlap would never be drained from
either tracker by the per-tick code — and if they cancel exactly,
skew_delta is zero and neither converges at all.
Arguably we could just let one of them entirely cancel out the other
immediately, but that would be a change in userspace-visible behaviour.
Instead, preserve the existing behaviour by calculating the "conflict"
portion between the opposing skew each second, and transferring that
amount directly from one tracker to the other.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Link: https://patch.msgid.link/20260621220051.1030462-7-dwmw2@infradead.org
|
|
The legacy adjtime() slew (ADJ_OFFSET_SINGLESHOT) was the last user of
tick_length != tick_length_base: it slewed the clock by inflating
tick_length directly, which delivered the correction imprecisely (e.g.
delivering only 4997.5µs when asked for a 5ms skew).
Deliver it accurately through the same per-tick mechanism that is now
used for time_offset, allowing it to contribute to skew_delta and thus
drive the delivery through ntp_error and mult selection.
To allow for accurate accounting, store the sub-microsecond part of
time_adjust is separately, while keeping time_adjust in microseconds
as that's the external API.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Link: https://patch.msgid.link/20260621220051.1030462-6-dwmw2@infradead.org
|
|
Currently, the phase offset of time_offset and time_adjust is delivered
by adjusting tick_length in second_overflow(), and immediately draining
time_offset/time_adjust by the amount that the tick_length adjustment is
*estimated* to cause. This is fairly approximate, in part because it is
not always correct to assume that precisely NTP_INTERVAL_FREQ ticks will
occur between one call to second_overflow() and the next. It could also
over and under-run in the final second of delivery.
Instead of inflating tick_length, transfer the intended skew directly
into ntp_error each tick to achieve the desired rate.
In second_overflow(), calculate skew_delta which is the per-tick slew
rate, in the same units as time_offset: (ns << NTP_SCALE_SHIFT) / HZ.
In logarithmic_accumulation(), drain up to 'skew_delta' time units from
time_offset into ntp_error to drive the overall effective rate. The new
ntp_drain_skew() function returns the amount which is actually 'claimed'
by time_offset (and in a future patch, time_adjust). Any overrun which
is delivered by the changed 'mult' (as described below) but not claimed
by ntp_drain_skew() will remain in ntp_error to be corrected away in
subsequent ticks.
Simply transferring the precise amount from time_offset to ntp_error
would be sufficent to make the time *eventually* converge, however the
skew delivered is limited by the choice of { mult, mult+1 } each tick
and thus the convergence would be extremely slow.
In theory we could inflate ntp_err_mult with the magnitude of ntp_error
in the general case — but that would cause overcorrection in a tickless
kernel. Instead, in timekeeping_adjust(), take skew_delta into account
when calculating 'mult', such that the available {mult, mult+1} choices
bracket the overall effective rate *including* the skew, to avoid the
delta just building up in ntp_error.
The effect is that the inflated 'mult' causes ntp_error to grow because
xtime_interval is (e.g.) longer than the true tick_length. But then the
same delta is removed again as it's drained from time_offset.
This gives behaviour equivalent to the old tick_length += delta approach
but with exact per-tick accounting of the time_offset actually imparted
to the clock, and no overrun.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Link: https://patch.msgid.link/20260621220051.1030462-5-dwmw2@infradead.org
|
|
cycle_interval is an integer number of counter cycles per NTP interval,
so the real time it represents differs from the nominal
NTP_INTERVAL_LENGTH by up to half a counter period. For coarse
clocksources this is significant: the 3.579545 MHz ACPI PM timer at
HZ=1000 rounds 3579.545 cycles up to 3580, making each tick 1.000127 ms
(+127 PPM).
Commit a386b5af8edd ("time: Compensate for rounding on odd-frequency
clocksources") introduced xtime_remainder to compensate for exactly
this, citing the same 127 PPM ACPI PM example. The compensation is
correct and necessary, but it was applied inside the timekeeping
accumulation in timekeeping.c: subtracted in the mult computation in
timekeeping_adjust() and folded into the ntp_error update in
logarithmic_accumulation(). That keeps the base rate correct and leaves
NTP its full symmetric +/-MAXFREQ range rather than +373/-627 PPM, but
the NTP code in ntp.c never sees it: tick_length is computed without the
correction, so ntp.c's notion of how long a tick is disagrees with the
rate timekeeping actually produces.
Make the offset an explicit part of the NTP tick_length instead. Add
ntp_data::cs_tick_adj, a fixed per-second addend that
ntp_update_frequency() includes alongside ntp_tick_adj and time_freq.
tk_setup_internals() computes it from the difference between the real
cycle_interval duration and the nominal interval, stores it in the
timekeeper, and hands it to NTP through a new argument to ntp_clear() --
which already recomputes the frequency and is invoked after every
clocksource (re)configuration. timekeeping_init() now uses TK_UPDATE_ALL
for this; clearing NTP there is otherwise redundant since ntp_init() has
just initialised it.
ntp.c now computes the true tick rate, giving a single source of truth.
Like ntp_tick_adj, cs_tick_adj stays internal to the kernel: userspace
still sees the nominal 1.000000 ms tick via adjtimex and is unaware of
the addends. timekeeping_adjust() and logarithmic_accumulation() use
ntp_tick / xtime_interval directly, and xtime_remainder is removed.
The base-rate arithmetic is unchanged: ntp_tick becomes
xtime_interval << ntp_error_shift, so the mult division yields the same
base mult and the ntp_error accumulation still nets to zero per tick.
Beyond the cleanup of treating all the tick_length contributions
(nominal interval, ntp_tick_adj, cs_tick_adj, time_freq) consistently
as addends in one place, it also prepares for feed-forward discipline:
a future timekeeping_set_reference() will set tick_length to track an
absolute external reference such as a vmclock, and that path needs
ntp.c to own a tick_length that already reflects the clocksource
quantisation, with no hidden correction applied elsewhere.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260621220051.1030462-4-dwmw2@infradead.org
|
|
consume_remote_task() drops this_rq before unlinking a remote task from
its DSQ and locking src_rq. When called from ops.dispatch(),
scx_locked_rq() keeps pointing to this_rq across the lock dance. The
subsequent switch_rq_lock(src_rq, this_rq) cannot update the tracking
because its guard does not match, leaving the tracking stale while
src_rq is held.
Keep this_rq locked until the task has been unlinked and the DSQ lock
released, then use switch_rq_lock() to switch directly to src_rq. Use
the same helper to restore this_rq after losing the dequeue race. The
successful path already switches back through
move_remote_task_to_local_dsq(), so scx_locked_rq() now follows the
actually held rq throughout the consume path.
Suggested-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/all/455e701bca66bdecde530d225f4dba0a@kernel.org
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
scx_bpf_kick_cid() returned an error code, but the value conveys nothing
actionable and no caller consumes it. The kick is asynchronous, so a
successful return only means it was queued. An invalid @cid is already
reported through scx_error() by scx_cid_to_cpu(), and a missing scheduler
leaves nothing to kick.
Make scx_bpf_kick_cid() return void to match scx_bpf_kick_cpu(). The
cid-form kfuncs are not in practical use yet, so the ABI change is safe.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Direct writes to p->scx.slice and p->scx.dsq_vtime bypass
scx_bpf_task_set_slice/dsq_vtime() and the authority checks they carry.
Those checks exist for sub-schedulers, which attach only through the
cid-form struct_ops, so the direct writes only need to be closed there.
Give sched_ext_ops_cid its own verifier ops that reject the two fields.
cid-form is a new interface with no legacy users, so there is no
compatibility to keep. The cpu-form keeps direct writes, and the deprecation
warning they carried is dropped.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_rq.extra_enq_flags carries scx-specific enqueue flags across the
activate_task() boundary during a cross-rq task move in
move_remote_task_to_local_dsq(). Rename it to remote_activate_enq_flags to
name that role, and fix the stale comment that referenced the old
move_task_to_local_dsq() name.
No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The cid-form ops overlay their cpu-form siblings at the same struct slot.
Ops whose signature matches the sibling are invoked through the cpu-form
call sites unchanged, but set_cmask() takes an arena cmask address rather
than a cpumask, so scx_call_op_set_cpumask() calls ops_cid.set_cmask()
directly and hand-rolled the kf_tasks[] and locked_rq bracket that
SCX_CALL_OP_TASK() provides. The hand-rolled bracket reset locked_rq to
NULL on exit instead of restoring the saved value, so a nested call would
clobber the outer op's locked-rq tracking.
Parameterize the dispatch macros by the ops-table member and add
SCX_CALL_CID_OP_TASK(), which routes through sch->ops_cid. Convert
scx_call_op_set_cpumask() to it and drop the hand-rolled bracket. The only
behavioral change is that locked_rq is now saved and restored like every
other op call site.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The cmask ops can operate on BPF-arena cmasks which BPF programs may read
and write concurrently. The _RACY op variants existed to make such lockless
reads explicit but this turned out to be too restrictive. Mark the word
accesses in all the two-cmask ops with READ_ONCE/WRITE_ONCE instead and drop
the _RACY variants.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Pull to receive:
db4e9defd2e8 ("sched_ext: Record an error on errno-only sub-enable failure")
49b3378a750c ("sched_ext: Fix premature ops->priv publication in scx_alloc_and_add_sched()")
e6979d05c6a6 ("tools/sched_ext: scx - Fix cmask_subset(), cmask_equal() and cmask_weight()")
for further sub-sched changes and to resolve the conflicts with the
sub-sched updates on for-7.3.
db4e9defd2e8 adds scx_error() to the sub-enable err_disable sink which
for-7.3 moved from ext.c into sub.c. Resolved by applying the fix to
scx_sub_enable_workfn() in sub.c.
49b3378a750c drops RCU_INIT_POINTER() from an scx_alloc_and_add_sched()
unwind label whose body changed with for-7.3's stall_cpus addition.
Resolved by dropping the line from the updated unwind.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
scx_alloc_and_add_sched() publishes @sch through ops->priv before allocating
the cgroup path. If that allocation fails, the unwind path clears ops->priv
and frees @sch immediately. scx_prog_sched() callers can dereference
ops->priv from RCU context the moment it is set, so freeing without a grace
period can use-after-free a concurrent kfunc caller.
Move the publication below the cgroup path allocation so that every failure
path after publication frees @sch through kobject_put(), whose release path
defers the freeing by a grace period.
Fixes: 105dcd005be2 ("sched_ext: Introduce scx_prog_sched()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_sub_enable_workfn() has several failure paths that only return an errno
(e.g. -ENOMEM from an allocation) and jump to err_disable without calling
scx_error(). scx_flush_disable_work() runs the disable, and thus ops.exit(),
only when an error has been recorded, so an errno-only failure leaves the
half-initialized sub-scheduler linked.
Record an error at the err_disable sink so every errno-only failure runs the
disable path.
Fixes: ebeca1f930ea ("sched_ext: Introduce cgroup sub-sched support")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The comment for scx_bpf_dsq_insert() references "@dsp_id" in the
description body, but the parameter is named "@dsq_id" in both the
parameter list and the function signature.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Acked-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
check_kfunc_args() detects a kfunc argument named rdonly_buf_size or
rdwr_buf_size and stores reg->var_off.value into meta->r0_size, a u64,
and does not bound it. check_kfunc_call() later copies that value into
the returned register's mem_size field:
meta->r0_size = reg->var_off.value;
...
regs[BPF_REG_0].mem_size = meta.r0_size;
regs[BPF_REG_0].mem_size is u32. A constant whose upper 32 bits are set
gets truncated instead of causing a load-time rejection, so the verifier
records a PTR_TO_MEM register with an approximately 4 GiB mem_size for
whatever allocation the kfunc returned. A later access check against
that register uses the truncated, wrong bound.
Reject rdonly_buf_size/rdwr_buf_size values that exceed U32_MAX at the
point meta->r0_size is set.
Fixes: eb1f7f71c126 ("bpf/verifier: allow kfunc to return an allocated mem")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
When a spilled scalar is filled back with a sign-extending narrowing load
(BPF_MEMSX), check_stack_read_fixed_off() copies the spilled register
including its scalar id, but coerce_reg_to_size_sx() then sign-extends the
filled register's value. If the same slot is also filled with a plain
zero-extending load (BPF_MEM), both destination registers share the id yet
hold different values. A later 'if <zext-reg> == const' then refines the
sign-extended register through sync_linked_regs() to a value it does not
have at runtime (e.g. the verifier believes 0x80000000 while the register
is 0xffffffff80000000), which can be turned into an out-of-bounds access.
Drop the shared scalar id at the sign-extension site in check_mem_access()
when sign extension actually changes the value, mirroring the BPF_MOVSX
handling in check_alu_op() (no_sext = reg_umax < 2^(size*8-1)).
Fixes: 3cd5c890652b ("bpf: Let the verifier assign ids on stack fills")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
When emitting a record via legacy printing, it is possible that a handover
to another legacy printing context occurs. When a context has performed a
handover, the console SRCU read lock is released and the pointer to the
console struct might now be invalid. Therefore, after calling
nbcon_legacy_emit_next_record() or console_emit_next_record(), it is
necessary to check if a handover occurred _before_ further @con usage.
Sashiko pointed out that console_flush_one_record() was not doing this.
In console_flush_one_record(), after emitting a record, move the further
usage of @con after the handover check.
Fixes: c158834b223f ("printk: nbcon: Use nbcon consoles in console_flush_all()")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/lkml/20260630170903.099D61F000E9@smtp.kernel.org
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Link: https://patch.msgid.link/20260703141521.202813-1-john.ogness@linutronix.de
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
|
Commit 5816bf4273ed ("lsm,selinux: Add LSM blob support for BPF objects")
made the LSM hook wrappers for BPF object creation clean up the LSM
state internally upon denial, e.g. security_bpf_map_create() internally
calls security_bpf_map_free() when the bpf_map_create hook returns an
error. map_create() however still routes a denial to its free_map_sec
label, which invokes security_bpf_map_free() a second time, so the
bpf_map_free hook fires twice for a single denied map.
In-tree LSMs are unaffected in practice since the blob kfree() inside
security_bpf_map_free() is NULL-safe and idempotent and none of them
implement bpf_map_free, but a BPF LSM program attached to that hook
observes double invocations. Route the denial to free_map instead.
Fixes: 5816bf4273ed ("lsm,selinux: Add LSM blob support for BPF objects")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260709073422.379247-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
When an architecture (such as LoongArch, ARM64, and RISC-V) implements
bpf_jit_inlines_helper_call(), the verifier skips rewriting the helper
call offset (insn->imm) in bpf_do_misc_fixups(). This is because the
helper is expected to be inlined by the JIT compiler later. Therefore,
insn->imm remains as the raw helper enum ID.
However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0) or
if JIT compilation fails dynamically (e.g., due to OOM), the program
falls back to the BPF interpreter.
When the interpreter executes (__bpf_call_base + insn->imm) with the
unpatched raw ID, it jumps into an invalid address space, triggering
an instruction alignment fault or a kernel panic.
Although these helpers have valid C implementations in the kernel, the
omission of offset rewriting makes runtime interpreter fallback fatal.
Fix this by setting 'prog->jit_required = 1' when helper call rewriting
is skipped for JIT inlining. This ensures that such programs are safely
rejected if JIT is not available, preventing the runtime kernel panic.
Fixes: 2ddec2c80b44 ("riscv, bpf: inline bpf_get_smp_processor_id()")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Introduce a 'jit_required' bitfield flag in struct bpf_prog to track
whether a BPF program strictly requires the JIT compiler to run. This
prevents a dangerous runtime fallback to the interpreter for features
that are only implemented in the JIT compiler.
Currently, bpf_prog_has_kfunc_call() is used only for kernel function
calls, replace the kfunc-specific helper with the new 'jit_required'
flag. This makes it easy to support other JIT-only BPF features, such
as inlined helpers.
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Suggested-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
bpf_find_vma() reads task->mm and calls mmap_read_trylock(mm) without
holding a reference on the mm. On a foreign task, a concurrent exit_mm()
can free the mm_struct between the lockless read and the trylock,
resulting in a use-after-free. mm_struct is not SLAB_TYPESAFE_BY_RCU.
For the current task, task->mm is stable. For a foreign task, pin the mm
under task->alloc_lock and release it with mmput_async(), mirroring commit
d8e27d2d22b6 ("bpf: fix mm lifecycle in open-coded task_vma iterator").
Use spin_trylock() instead of get_task_mm() so BPF context does not block
on alloc_lock. Reject irqs-disabled contexts and !CONFIG_MMU on the
foreign-task path because dropping the mm reference is not safe there.
Race:
CPU0 (BPF program) CPU1 (exiting task)
============================ ==========================
bpf_find_vma(foreign_task):
mm = task->mm
exit_mm():
task->mm = NULL
mmput(mm) -> frees mm_struct
mmap_read_trylock(mm)
// UAF on mm
Fixes: 7c7e3d31e785 ("bpf: Introduce helper bpf_find_vma")
Signed-off-by: Sanghyun Park <sanghyun.park.cnu@gmail.com>
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/bpf/20260708072106.199637-2-sanghyun.park.cnu@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_prog_calc_tag() copies the instructions into a plain vmalloc() scratch
buffer to blind the map fds before hashing. The buffer scales with the
program, up to ~8MB at the 1M instruction limit, and is allocated on every
program load, but unlike the rest of the load-time scratch memory it is
not charged to the loader's memcg. Use GFP_KERNEL_ACCOUNT to account it
like the other allocations scoped to the verification/load.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The insn_aux_data array is allocated with a plain vzalloc(), while every
other allocation scoped to the verification - verifier states, explored
states, the cfg/scc arrays, liveness masks, jump history - is charged to
the loader's memcg via GFP_KERNEL_ACCOUNT.
At 136 bytes per instruction it is one of the largest verification-time
buffers, in the range of ~130MB for a program at the 1M instruction limit
(worst case), and it lives across the whole verification. The buffer is
also inconsistent with itself: when instruction patching grows it, the
vrealloc() in bpf_patch_insn_data() already passes GFP_KERNEL_ACCOUNT.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_get_btf_vmlinux() serializes the lazy vmlinux BTF parse with
bpf_verifier_lock, the same mutex bpf_check() holds across the whole
verification of an unprivileged program (if enabled; it's disabled
by default). The latter can potentially stall the mutex holder for
a long time (e.g. via userfaultfd), and therefore block first-time
bpf_get_btf_vmlinux() caller from any context, including privileged
program loads.
Give the vmlinux BTF initialization a dedicated btf_vmlinux_lock so
it is independent of the unprivileged verification mutex. The parse
only needs mutual exclusion against itself.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_get_btf_vmlinux() lazily parses the vmlinux BTF under the
bpf_verifier_lock, but publishes the result through a plain store
and re-checks it through a plain lockless load. Nothing orders
the stores initializing the struct btf inside btf_parse_vmlinux()
against the store publishing the pointer: On a weakly ordered
arch, a concurrent first-time caller taking the lockless fast
path could in principle observe the pointer before the parsed
contents are visible. The mutex_unlock() does not help such a
reader given it only synchronizes with a later acquisition of the
same lock. Thus, publish the pointer with smp_store_release()
and read it on the fast path with smp_load_acquire().
Acquire semantics are needed rather than a dependency-ordered
READ_ONCE(): btf_parse_vmlinux() also populates globals outside
the returned object (e.g. bpf_ctx_convert.t). An address
dependency would only order accesses performed through the
pointer and not cover other globals.
Fixes: 8580ac9404f6 ("bpf: Process in-kernel BTF")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
set_next_task_scx() updates the tick dependency before __schedule()
updates rq->curr. When switching from a non-EXT task, such as idle, to
an EXT task with a finite slice, sched_update_tick_dependency() checks
the outgoing task and can allow the tick to remain stopped.
The dependency can also be lost without a slice-type transition. After a
finite-slice task leaves the CPU idle, the enqueue path can clear the
dependency against the idle rq->curr. SCX_RQ_CAN_STOP_TICK still records
a finite slice, so another finite task skips the transition block and
can run without the ticks needed to expire its slice.
The reverse mismatch can also happen when the last finite-slice EXT task
is dequeued: sub_nr_running() updates the dependency before rq->curr
changes, so the outgoing task state can keep the dependency set after
the CPU goes idle.
Fix this by unconditionally enabling the scheduler tick whenever a
finite-slice EXT task is selected on a nohz_full CPU. Moreover, when the
last runnable EXT task leaves, ignore the outgoing EXT slice state so
the generic scheduler can correctly re-evaluate and clear the tick
dependency.
Fixes: 22a920209ab6 ("sched_ext: Implement tickless support")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The function calculates new_len as len << 1 for hex encoding. This
has two overflow risks: the shift itself can overflow when len is
large, and the result can be truncated when assigned to new_len
(declared as int) from the size_t calculation.
Fix by using check_shl_overflow() to catch shift overflow and
changing new_len and loop counter i to size_t to prevent truncation.
Cc: stable@vger.kernel.org
Fixes: 168b7173959f ("AUDIT: Clean up logging of untrusted strings")
Reviewed-by: Richard Guy Briggs <rgb@redhat.com>
Signed-off-by: Ricardo Robaina <rrobaina@redhat.com>
[PM: remove vertical whitspace noise]
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
dispatch_to_local_dsq() can run from scx_bpf_dsq_move_to_local() while
ops.dispatch() has recorded the current rq. Moving a task to a local DSQ
may switch to the source or destination rq before synchronously invoking
ops.dequeue() through the following path:
SCX_CALL_OP(dispatch, rq)
ops.dispatch()
scx_bpf_dsq_move_to_local()
scx_flush_dispatch_buf()
finish_dispatch()
dispatch_to_local_dsq()
scx_dispatch_enqueue()
local_dsq_post_enq()
call_task_dequeue()
SCX_CALL_OP_TASK(dequeue, locked_rq, ...)
The nested callback saves the recorded rq and restores it on return. If
the rq tracking does not follow the lock switch, update_locked_rq() can
trigger the following lockdep assertion while restoring an rq which is
no longer held:
WARNING: kernel/sched/sched.h:1641 at call_task_dequeue+0x160/0x170
Call Trace:
scx_dispatch_enqueue+0x2b0/0x460
dispatch_to_local_dsq+0x138/0x230
scx_flush_dispatch_buf+0x1af/0x220
scx_bpf_dsq_move_to_local___v2+0xe2/0x1c0
bpf__sched_ext_ops_dispatch+0x4b/0xa7
do_pick_task_scx+0x3b6/0x910
__pick_next_task+0x105/0x1f0
__schedule+0x3e7/0x1980
Introduce switch_rq_lock() to update the tracking state together with
each rq lock handoff. Use it in dispatch_to_local_dsq(),
move_remote_task_to_local_dsq() and the in-balance paths of
scx_dsq_move(), ensuring that scx_locked_rq() consistently refers to the
rq whose lock is actually held throughout the lock dance.
Fixes: 7fb39e4eb4c3 ("sched_ext: Save and restore scx_locked_rq across SCX_CALL_OP")
Cc: stable@vger.kernel.org # 7.1+
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The signed gen_loader used to police its own metadata map from within
BPF: emit_signature_match() read the kernel-cached map->sha[] back
through hardcoded struct bpf_map offsets and compared it against a hash
that compute_sha_update_offsets() baked into the signed instructions,
after a BPF_OBJ_GET_INFO_BY_FD round-trip to populate map->sha[].
The kernel now verifies the metadata at BPF_PROG_LOAD time by folding
the frozen contents of the loader's exclusive fd_array maps into the
signature, so the loader no longer checks anything itself. Generated
loaders thus carry no verification logic of their own anymore: Nothing
in the signing chain depends on emitted loader bytecode doing the right
thing.
On the loading side, skel_internal.h now sets fd_array_cnt for a signed
load so the kernel scans fd_array for the exclusive metadata map -
still frozen, as the kernel requires - and the BPF_OBJ_GET_INFO_BY_FD
round-trip to populate map->sha[] is gone. The struct bpf_map layout
BUILD_BUG_ON()s on the kernel side are removed as well: they only
pinned the ABI for the in-BPF read of map->sha[] that is no longer
needed. Same for the map->excl member. Note: gen_hash is retained; it
still marks a loader as signed so an untrusted host cannot re-dimension
maps or override initial values now covered by the signature.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708075343.358712-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
A signed gen_loader program carries the programs, maps and relocations
it installs in a metadata array map. The loader instructions are covered
by the PKCS#7 signature, but the metadata map is not: Today the loader
compares the map contents from within BPF against a hash baked into its
(signed) instructions, using the kernel-cached map hash. The kernel
itself never actually attests that the metadata the loader installs is
the metadata that was signed.
This split is the core of the long-standing objection to the BPF signing
scheme from the LSM / integrity side: the integrity check of a light
skeleton only completes once the loader program runs, that is, after the
security_bpf_prog_load() hook, so at admission time an LSM observes a
program whose payload has not yet been verified. Auditing the chain
link is also not a purely cryptographic operation: whoever signs or
reviews an lskel has to disassemble the loader's preamble to convince
themselves that the embedded hash check is present and correct [0][1].
Two acceptable fixes were identified in those threads: Complete the
integrity check before the admission hook fires, or add a second hook
that collects the verification result after the loader ran [2]. Covering
both the loader and its maps directly with the PKCS#7 signature is what
Blaise Boscaccy's patchsets proposed in several forms. Let's implement
the former, without growing the UAPI, and in particular as a single
unified scheme where the signature spans the raw bytes rather than
derived hashes.
A signed loader binds its metadata map(s) through the existing fd_array,
and an exclusive map is already bound to a program digest (excl_prog_hash).
So when a signature is present, collect the exclusive maps from fd_array
and append their frozen contents to the instructions before verification:
The signature now covers insns || metadata_0 || metadata_1 || [...] in
the fd_array order, and verification completes in bpf_check(), once the
fd_array maps are resolved into used_maps, before the LSM admission hook
and the rest of verification. A program is either BPF_SIG_UNSIGNED or
BPF_SIG_VERIFIED, with nothing in between. While folding the fd_array
maps, a non-exclusive map bound to a signed program is rejected, so every
map folded into the signature is exclusive. A signed loader that fails
to cover its metadata thus does not load, and BPF_SIG_VERIFIED always
means the instructions and every exclusive map are authentic. The maps
must be frozen so the hashed bytes cannot change before the loader runs;
the map <-> program digest binding is enforced by the verifier for every
used map. Binding maps through fd_array_cnt makes the verifier resolve
and excl-check them (excl_prog_sha vs prog->digest) before it would
otherwise compute the digest, so compute prog->digest up front in
bpf_check(), over the unmodified instructions the signature covers, for
a load that folds metadata.
Unsigned programs are not affected by the signature path; for them the
LSM admission hook merely moves below fd_array resolution, with minimal
bounded work in between. Note, signed loaders generated by older libbpf/
bpftool versions need to be regenerated; some of the recent fixes we've
had on the signed loader side require the latter already to close gaps.
Finally, some remarks around the security_bpf_prog_load() placement
given there was discussion on whether a new hook is needed or the existing
security_bpf_prog() hook should be reused [3]: For a new hook it would
mean that just for loading a single BPF program it has to pass through
four layers of LSM hooks:
1) security_bpf (cmd=PROG_LOAD): for gating various bpf subcmds
2) security_bpf_prog_load: historical admission hook (CAP/token,
prog_type, attach point), pre-verification
3) security_bpf_prog_verify_signature: newly asked admission hook,
same role as 2), plus the BPF signature verdict
4) security_bpf_prog: gate handing the prog fd back to userspace,
verification done & signature verified
The use-cases of 2) and 3) conflate, thus BPF community prefers to just
keep a total of 3 LSM hooks (as-is today): 3) makes 2) incoherent given
they are the /same class/ of hook, that is, access-control admission on
the load and split only by _what_ they can see. Worse, with the split,
for a signed BPF program security_bpf_prog_load 2) admits a program whose
signature has not been checked, so a policy gating at 2) is structurally
unable to express "admit only verified" and every such policy is forced
onto 3) *anyway*. In other words, one doesn't get two complementary hooks,
but rather, one real admission hook aka 3) plus a now-degraded /legacy/
hook 2) that can't answer the question operators actually want to ask.
Reusing security_bpf_prog() 4) for admission is no alternative either:
it fires only after the entire verifier (and JIT) pipeline ran, so
denying a not-yet-verified program at that point burns exactly the
work a denial is supposed to avoid, and by then the program has an id
assigned and the kallsyms/perf/audit load events fired. Policies are
free to also consume the signature verdict at 4), but admission control
belongs into security_bpf_prog_load(). Hence the latter remains the only
admission hook, merely moved past signature verification; with moving
large allocations further down into the BPF verifier, there is now only
minimal work between the old and new location: The preparation work in
bpf_check() is reordered such that only the minimally necessary setup
happens up front: Allocating the env, initializing the verifier log and
resolving the fd_array that a signed BPF metadata map needs. The worst
case allocation up until security_bpf_prog_load() is ~90K which is the
env itself (~54K) plus the continuous fd_array cache (at most 32K). The
insn_aux_data array is moved into a later stage in the verification.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/2f71d6c03698eb17d51f7247efde777627ee578a.camel@HansenPartnership.com [0]
Link: https://lore.kernel.org/lkml/ecf0521ed302db672672ebfbc670ecfba36a6e00.camel@HansenPartnership.com [1]
Link: https://lore.kernel.org/bpf/88703f00d5b7a779728451008626efa45e42db3d.camel@HansenPartnership.com [2]
Link: https://lore.kernel.org/bpf/DJOFY21DYUI4.19WKQ3NPZ4H5R@gmail.com [3]
Link: https://lore.kernel.org/bpf/20260708075343.358712-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The fd_array passed to BPF_PROG_LOAD carries the map and module BTF file
descriptors a program binds. The verifier reads it more than once during
a load: process_fd_array() walks it to bind the maps and BTFs, and
check_and_resolve_insns() and the kfunc BTF resolver later read it again
to resolve the program's BPF_PSEUDO_MAP_IDX* and module kfunc refs.
For signed BPF, we need these upfront in memory, thus resolve each fd to
its object once and cache it by fd_array index, then bind that cached
object for the rest of the load. env->fd_array becomes a small per-slot
{map, btf} cache rather than a bpfptr_t; every later reference is then
an in-bounds lookup of an already-resolved object, and an index outside
the cache is rejected instead of read from user memory:
- continuous (fd_array_cnt given): the caller declares the length and
every entry is resolved and bound up front (used also by the BPF
signed loader)
- sparse (no fd_array_cnt): left as the legacy path with no fd_array
cache; each reference reads its fd from the caller's fd_array and
resolves it on the spot. Deduplication in used_maps and the kfunc BTF
table keeps this correct, and only unsigned programs use this shape.
Split these into separate helpers to make it easier to follow.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
Link: https://lore.kernel.org/bpf/20260708075343.358712-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Remove unused arch_needs_cpu() hook. No architectures use it after
s390 removed its use case.
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Thomas Gleixner <tglx@kernel.org>
Acked-by: Rafael J. Wysocki (Intel) <rafael@kernel.org>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
|
|
task CPU
Background:
==========
Currently lockdep does not print out the held locks of non-current
tasks that are running on some other CPU, due to the fact that
the held locks array is in flux and may be unreliable to print.
Syzkaller on the other hand found it that the analysis of locking
bugs is easier if we print this information too, because the
more locking information the merrier. In particular races are
bound to have multiple tasks running on different CPUs, and
the exclusion of their held locks information is unnecessarily
limiting.
So while it's still true that printing out their held locks
array is racy, it's not as bad as it seems.
There's 16 internal callers to lockdep_print_held_locks():
- 14 callers call it with the current task, which should be
safe out of box.
- 1 caller, debug_show_all_locks(), calls it with RCU held,
which should guarantee that 'p' cannot go away under us.
- 1 caller, debug_show_held_locks(), exposes the internal API
with the constraint that it should only be called by drivers
or platform code if the task isn't actively running - we can
assume that if it nevertheless does, it will be Their Problem™.
As for held locks being changed from under debug_show_held_locks(),
while the task cannot go away, so the held-locks array itself is
safe (although potentially non-stable), AFAICS the worst-case race
can be garbage printed out by print_lock(), not any actual crashes.
In particular:
unsigned int class_idx = hlock->class_idx;
may be stale (belong to a lock that already got released on another
CPU), but it should still be a valid class index bound by
MAX_LOCKDEP_KEYS, and thus the lock_classes_in_use bitmap use
should be safe.
The other two accesses are ::acquire_ip and ::instance:
printk(KERN_CONT "%px", hlock->instance);
print_lock_name(hlock, lock);
printk(KERN_CONT ", at: %pS\n", (void *)hlock->acquire_ip);
But both are printed out as pointers, so no risk of dereference
of a dangling pointer. We may print a garbage pointer.
Also note that the check itself doesn't protect debug_show_held_locks()
from printing garbage, as there's nothing that keeps a task from
becoming runnable a nanosecond after we've run the task_is_running()
check. In fact I'd argue that it's better to make this function
*more* racy, for the simple robustness reason that we absolutely
do not want it to crash even in the racy case.
TL;DR: it should be fine to print the held locks of running
tasks too, as long as we print out the information as well
that a task is running, so that users are aware of any
racy output.
Implementation:
==============
Implement that change.
Also re-flow the function and streamline the printout into
a single statement for all cases, which changes
the 'no locks held by' / '%d lock[s] held by' phrasing that had a
dependency on English spelling of plurals, to a uniform:
locks held by bash/1234: %d
Which spells correctly for 0, 1 and higher values, and should also
be easier to parse both for humans and for scripts.
Finally, print out the last CPU a task has ran on. This is very
useful information for races and for locking bugs in particular.
This basically extends the 'on CPU#%d' message we print for
running tasks to all tasks we print.
Reported-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Suggested-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Tested-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Boqun Feng <boqun@kernel.org>
Cc: Gary Guo <gary@garyguo.net>
Cc: Mark Brown <broonie@kernel.org>
Cc: Theodore Tso <tytso@mit.edu>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Waiman Long <longman@redhat.com>
Link: https://patch.msgid.link/akoeSIQGwqd9cZwd@gmail.com
|
|
Add a matching sysctl to go with CONFIG_SYSCALL_USER_DISPATCH.
kernel.syscall_user_dispatch (default 1 - allow) controls whether
userspace may arm syscall user dispatch (both via prctl and ptrace).
Disarming is always permitted - same semantics as comparable knobs.
Disabling while a task has armed syscall user dispatch does not
cause it to become inactive - instead it remains active until the
user attempts to disable/re-enable via prctl or ptrace. On the
next attempt to re-enable, the prctl/ptrace call fails gracefully.
The alternative would cause programs translating non-linux syscalls
to interpret those syscalls as linux syscalls, resulting in undefined
userland behavior.
Signed-off-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260706140020.873735-3-gourry@gourry.net
|
|
Syscall User Dispatch is presently built under CONFIG_GENERIC_SYSCALL and
cannot be disabled independently.
Add CONFIG_SYSCALL_USER_DISPATCH to make it an optional feature.
Signed-off-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260706140020.873735-2-gourry@gourry.net
|
|
check_ptr_to_btf_access() lets program-type btf_struct_access callbacks
validate writes before the default BTF access path rejects non-read
accesses. That bypasses the read-only policy for untrusted BTF pointers
created by helpers such as bpf_rdonly_cast().
Reject non-read accesses through PTR_UNTRUSTED BTF pointers at the
common entry point, before the callback branch to handle all cases.
Fixes: 282de143ead9 ("bpf: Introduce allocated objects support")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
The verifier loses precision when simulating stack spills for the
following register types:
- PTR_TO_TP_BUFFER
- PTR_TO_INSN
- CONST_PTR_TO_DYNPTR
These types are not allow-listed in the is_spillable_regtype(),
because of that check_stack_write_fixed_off() takes the branch
that marks the slots STACK_MISC.
There are no technical reasons for this limitation.
This commit replaces an explicit list of pointer types in
is_spillable_regtype() with explicit list of non-pointer types.
The function is renamed to is_pointer_regtype() for clarity.
Reported-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Suggested-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260707-missing-spillable-types-v1-1-44a92121dc41@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
timekeeping_apply_adjustment() modifies xtime_nsec to ensure monotonicity
when mult changes:
xtime_nsec -= offset
This ensures that the time reported to userspace does not jump when the
multiplier is adjusted from one tick to the next. However, the ntp_error
accumulator which tracks the difference between intended and actual
clock position was not being updated to reflect this additional
discrepancy.
An earlier attempt at this compensation existed as:
ntp_error -= (interval - offset) << ntp_error_shift
but was removed in commit c2cda2a5bda9 ("timekeeping/ntp: Don't align
NTP frequency adjustments to ticks") because it was a major source of
NTP error. That's because (interval - offset) was wrong: the subtraction
of "interval" prematurely accounted for the changed xtime_interval of
the next tick, which would be correctly accounted in the next
accumulation anyway — a double subtraction.
What is actually needed is just the "offset" part: ntp_error must be
told that xtime_nsec moved by "offset" without a corresponding change
in the intended position. For the normal ±1 mult dithering this is
negligible (the adjustments cancel over time), but for larger mult
changes — such as when an external reference clock sets a new
frequency — the one-time uncompensated offset is significant.
Fix by adjusting ntp_error by the correct amount:
ntp_error += offset << ntp_error_shift
This keeps ntp_error consistent with the actual xtime_nsec position
after the adjustment, and ensures the discrepancy is correctly smoothed
away over time and the clock returns to where it should have been.
Fixes: c2cda2a5bda9 ("timekeeping/ntp: Don't align NTP frequency adjustments to ticks")
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.6-1m
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260621220051.1030462-3-dwmw2@infradead.org
|
|
The "old" time types use 32-bit seconds which are not y2038-safe.
Respect COMPAT_32BIT_TIME for functions using those types.
time(), stime() and gettimeofday() are disabled completely.
settimeofday() is kept as it is required to do the initial timewarping
after boot. However the 'tv' argument will be rejected.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Link: https://lore.kernel.org/lkml/e9487ebe-3730-438a-9c23-e45f75986ecc@app.fastmail.com/
Link: https://patch.msgid.link/20260702-vdso-compat_32bit_time-v3-1-db9f36d8d432@linutronix.de
|
|
The fault handler for the timens page does not have access to the target
task and therefore can not be invoked remotely.
Currently the handler relies on the fact that the vvar mapping is marked as
VM_IO and VM_PFNMAP for which the mm core always prevents remote access.
However the VM_IO and VM_PFNMAP flags are going to be removed.
Add an explicit check to prevent remote access to the mapping.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Nam Cao <namcao@linutronix.de>
Link: https://patch.msgid.link/20260630-vdso-mlockall-v4-4-6c93708ce723@linutronix.de
|
|
All architectures are now fully using the generic vDSO infrastructure.
They don't need these declarations anymore to implement the functions
in architecture-specific code.
Move them to the private header.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260707-timekeeping-header-cleanup-v1-2-e85ad96409a9@linutronix.de
|
|
This header uses some definitions from linux/hrtimer_bases.h.
Currently this header is included transitively, which will change.
Include the header explicitly.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260702-hrtimer-header-dependencies-v1-5-c50b19bda473@linutronix.de
|
|
This header uses some definitions from linux/hrtimer_bases.h.
Currently this header is included transitively, which will change.
Include the header explicitly.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260702-hrtimer-header-dependencies-v1-4-c50b19bda473@linutronix.de
|
