linux.git/kernel/unwind, branch nocache-cleanup

unwind_user/x86: Teach FP unwind about start of function

2025-10-29T09:29:58+00:00

When userspace is interrupted at the start of a function, before we
get a chance to complete the frame, unwind will miss one caller.

X86 has a uprobe specific fixup for this, add bits to the generic
unwinder to support this.

Suggested-by: Jens Remus 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/20251024145156.GM4068168@noisy.programming.kicks-ass.net

unwind: Implement compat fp unwind

2025-10-29T09:29:57+00:00

It is important to be able to unwind compat tasks too.

Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/20250924080119.613695709@infradead.org

unwind: Simplify unwind_user_next_fp() alignment check

2025-10-29T09:29:57+00:00

  2^log_2(n) == n

Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Steven Rostedt (Google) 
Link: https://patch.msgid.link/20250924080119.497867836@infradead.org

unwind: Make unwind_task_info::unwind_mask consistent

2025-10-29T09:29:57+00:00

The unwind_task_info::unwind_mask was manipulated using a mixture of:

  regular store
  WRITE_ONCE()
  try_cmpxchg()
  set_bit()
  atomic_long_*()

Clean up and make it consistently atomic_long_t.

Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/20250924080119.384384486@infradead.org

unwind: Simplify unwind_user_faultable()

2025-10-29T09:29:56+00:00

Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/20250924080119.271671514@infradead.org

unwind: Clarify calling context

2025-10-29T09:29:56+00:00

The get_cookie() function hard relies on IRQs being disabled, but this
isn't immediately obvious when reading the function.

Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Steven Rostedt (Google) 
Link: https://patch.msgid.link/20250924080119.122507632@infradead.org

unwind: Fix unwind_deferred_request() vs NMI

2025-10-29T09:29:56+00:00

task_work_add(RWA_RESUME) isn't NMI-safe, use TWA_NMI_CURRENT when
used from NMI context.

Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Steven Rostedt (Google) 
Link: https://patch.msgid.link/20250924080119.005422353@infradead.org

unwind: Finish up unwind when a task exits

2025-07-31T14:20:11+00:00

On do_exit() when a task is exiting, if a unwind is requested and the
deferred user stacktrace is deferred via the task_work, the task_work
callback is called after exit_mm() is called in do_exit(). This means that
the user stack trace will not be retrieved and an empty stack is created.

Instead, add a function unwind_deferred_task_exit() and call it just
before exit_mm() so that the unwinder can call the requested callbacks
with the user space stack.

Cc: Masami Hiramatsu 
Cc: Mathieu Desnoyers 
Cc: Josh Poimboeuf 
Cc: Peter Zijlstra 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Arnaldo Carvalho de Melo 
Cc: Namhyung Kim 
Cc: Thomas Gleixner 
Cc: Andrii Nakryiko 
Cc: Indu Bhagat 
Cc: "Jose E. Marchesi" 
Cc: Beau Belgrave 
Cc: Jens Remus 
Cc: Linus Torvalds 
Cc: Andrew Morton 
Cc: Jens Axboe 
Cc: Florian Weimer 
Cc: Sam James 
Link: https://lore.kernel.org/20250729182406.504259474@kernel.org
Signed-off-by: Steven Rostedt (Google)

unwind deferred: Use SRCU unwind_deferred_task_work()

2025-07-31T14:20:11+00:00

Instead of using the callback_mutex to protect the link list of callbacks
in unwind_deferred_task_work(), use SRCU instead. This gets called every
time a task exits that has to record a stack trace that was requested.
This can happen for many tasks on several CPUs at the same time. A mutex
is a bottleneck and can cause a bit of contention and slow down performance.

As the callbacks themselves are allowed to sleep, regular RCU cannot be
used to protect the list. Instead use SRCU, as that still allows the
callbacks to sleep and the list can be read without needing to hold the
callback_mutex.

Link: https://lore.kernel.org/all/ca9bd83a-6c80-4ee0-a83c-224b9d60b755@efficios.com/

Cc: "Paul E. McKenney" 
Cc: Masami Hiramatsu 
Cc: Josh Poimboeuf 
Cc: Peter Zijlstra 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Arnaldo Carvalho de Melo 
Cc: Namhyung Kim 
Cc: Thomas Gleixner 
Cc: Andrii Nakryiko 
Cc: Indu Bhagat 
Cc: "Jose E. Marchesi" 
Cc: Beau Belgrave 
Cc: Jens Remus 
Cc: Linus Torvalds 
Cc: Andrew Morton 
Cc: Jens Axboe 
Cc: Florian Weimer 
Cc: Sam James 
Link: https://lore.kernel.org/20250729182406.331548065@kernel.org
Suggested-by: Mathieu Desnoyers 
Signed-off-by: Steven Rostedt (Google)

unwind: Add USED bit to only have one conditional on way back to user space

2025-07-31T14:20:11+00:00

On the way back to user space, the function unwind_reset_info() is called
unconditionally (but always inlined). It currently has two conditionals.
One that checks the unwind_mask which is set whenever a deferred trace is
called and is used to know that the mask needs to be cleared. The other
checks if the cache has been allocated, and if so, it resets the
nr_entries so that the unwinder knows it needs to do the work to get a new
user space stack trace again (it only does it once per entering the
kernel).

Use one of the bits in the unwind mask as a "USED" bit that gets set
whenever a trace is created. This will make it possible to only check the
unwind_mask in the unwind_reset_info() to know if it needs to do work or
not and eliminates a conditional that happens every time the task goes
back to user space.

Cc: Masami Hiramatsu 
Cc: Mathieu Desnoyers 
Cc: Josh Poimboeuf 
Cc: Peter Zijlstra 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Arnaldo Carvalho de Melo 
Cc: Namhyung Kim 
Cc: Thomas Gleixner 
Cc: Andrii Nakryiko 
Cc: Indu Bhagat 
Cc: "Jose E. Marchesi" 
Cc: Beau Belgrave 
Cc: Jens Remus 
Cc: Linus Torvalds 
Cc: Andrew Morton 
Cc: Jens Axboe 
Cc: Florian Weimer 
Cc: Sam James 
Link: https://lore.kernel.org/20250729182406.155422551@kernel.org
Signed-off-by: Steven Rostedt (Google)