linux-stable.git/kernel/bpf/stackmap.c, branch v7.2.6 Linux kernel stable tree bpf: Fix mmap_lock leak in irq_work path 2026-09-14T11:40:32+00:00 Sanghyun Park sanghyun.park.cnu@gmail.com 2026-08-05T03:14:25+00:00 051f2da26ce377f683b938a3382e0f16d02f3139 [ Upstream commit fa9dcacdcdf487f0ffef64bf67622f1caed509f1 ] stack_map_get_build_id_offset() introduced a per-CPU irq_work to defer mmap_read_unlock() from NMI context, and bpf_find_vma() later reused the same mmap_unlock_work. Both callers only check whether the work is busy before taking mmap_lock, so a nested caller can reuse the slot before the first caller queues it. Two read locks may then be acquired while only one deferred unlock runs, leaking a read lock and blocking exit_mmap(). Reserve the per-CPU slot before mmap_read_trylock(). Use the same wrapper in stackmap and bpf_find_vma() so both callers release the reservation on trylock failure. Keep rejecting the slot while the irq_work remains busy. Release it after the irq_work callback unlocks the mm. Fixes: eac9153f2b58 ("bpf/stackmap: Fix deadlock with rq_lock in bpf_get_stack()") Reported-by: syzbot+cdd6c0925e12b0af60cc@syzkaller.appspotmail.com Reported-by: sashiko-bot@kernel.org Signed-off-by: Sanghyun Park <sanghyun.park.cnu@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Closes: https://syzkaller.appspot.com/bug?extid=cdd6c0925e12b0af60cc Closes: https://lore.kernel.org/r/20260630033745.B80201F000E9@smtp.kernel.org Link: https://lore.kernel.org/bpf/20260805031425.2157475-2-sanghyun.park.cnu@gmail.com Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit fa9dcacdcdf487f0ffef64bf67622f1caed509f1 ]

stack_map_get_build_id_offset() introduced a per-CPU irq_work to defer
mmap_read_unlock() from NMI context, and bpf_find_vma() later reused the
same mmap_unlock_work. Both callers only check whether the work is busy
before taking mmap_lock, so a nested caller can reuse the slot before the
first caller queues it. Two read locks may then be acquired while only one
deferred unlock runs, leaking a read lock and blocking exit_mmap().

Reserve the per-CPU slot before mmap_read_trylock(). Use the same wrapper
in stackmap and bpf_find_vma() so both callers release the reservation on
trylock failure. Keep rejecting the slot while the irq_work remains busy.
Release it after the irq_work callback unlocks the mm.

Fixes: eac9153f2b58 ("bpf/stackmap: Fix deadlock with rq_lock in bpf_get_stack()")
Reported-by: syzbot+cdd6c0925e12b0af60cc@syzkaller.appspotmail.com
Reported-by: sashiko-bot@kernel.org
Signed-off-by: Sanghyun Park <sanghyun.park.cnu@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Closes: https://syzkaller.appspot.com/bug?extid=cdd6c0925e12b0af60cc
Closes: https://lore.kernel.org/r/20260630033745.B80201F000E9@smtp.kernel.org
Link: https://lore.kernel.org/bpf/20260805031425.2157475-2-sanghyun.park.cnu@gmail.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
bpf: Clear buf on error in __bpf_get_task_stack 2026-09-14T11:40:22+00:00 Jiri Olsa jolsa@kernel.org 2026-08-03T21:01:46+00:00 f523359c4789bb0396d2d642464a6e2fd4f2299d [ Upstream commit f5d242825ca417bb6afe35fde6e8880f97ca43fb ] Both bpf_get_task_stack and bpf_get_task_stack_sleepable helpers that use __bpf_get_task_stack have buf defined as ARG_PTR_TO_UNINIT_MEM argument and we should initialize the buf on every return path. Adding missing buf memset for __bpf_get_task_stack fail paths. This provides deterministic buffer contents, which is useful when the buffer is used directly as a map key. Fixes: 06ab134ce8ec ("bpf: Refcount task stack in bpf_get_task_stack") Fixes: b992f01e6615 ("bpf: Guard against accessing NULL pt_regs in bpf_get_task_stack()") Reported-by: Sashiko <sashiko-bot@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-10-jolsa@kernel.org Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit f5d242825ca417bb6afe35fde6e8880f97ca43fb ]

Both bpf_get_task_stack and bpf_get_task_stack_sleepable helpers that
use __bpf_get_task_stack have buf defined as ARG_PTR_TO_UNINIT_MEM
argument and we should initialize the buf on every return path.

Adding missing buf memset for __bpf_get_task_stack fail paths. This
provides deterministic buffer contents, which is useful when the buffer
is used directly as a map key.

Fixes: 06ab134ce8ec ("bpf: Refcount task stack in bpf_get_task_stack")
Fixes: b992f01e6615 ("bpf: Guard against accessing NULL pt_regs in bpf_get_task_stack()")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-10-jolsa@kernel.org
Signed-off-by: Sasha Levin <sashal@kernel.org>
bpf: Remove trace_in argument from __bpf_get_stack 2026-09-14T11:40:22+00:00 Jiri Olsa jolsa@kernel.org 2026-08-03T21:01:45+00:00 34cbcb333f30e25579513a4bd7c6b8ffd3a470ec [ Upstream commit 58cfc2201d964163fe9c4a703136eb64db799f08 ] Now with the new callchain_* helper functions we can process trace_in case directly in bpf_get_stack_pe function and remove it from __bpf_get_stack which makes things easier for preemption fix in following change. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-9-jolsa@kernel.org Stable-dep-of: f5d242825ca4 ("bpf: Clear buf on error in __bpf_get_task_stack") Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 58cfc2201d964163fe9c4a703136eb64db799f08 ]

Now with the new callchain_* helper functions we can process trace_in
case directly in bpf_get_stack_pe function and remove it from
__bpf_get_stack which makes things easier for preemption fix in
following change.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-9-jolsa@kernel.org
Stable-dep-of: f5d242825ca4 ("bpf: Clear buf on error in __bpf_get_task_stack")
Signed-off-by: Sasha Levin <sashal@kernel.org>
bpf: Factor callchain_finalize function from __bpf_get_stack 2026-09-14T11:40:22+00:00 Jiri Olsa jolsa@kernel.org 2026-08-03T21:01:44+00:00 27ed6cf4c0ac5767d18100cd1b8de4c057c79f38 [ Upstream commit 014fbe5902dccdaef69114abdcdb15e3dbe55e34 ] The new callchain_finalize function calls the build-id retrieval (if needed) and zeroes the buffer. This makes things easier for preemption fix in following change. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-8-jolsa@kernel.org Stable-dep-of: f5d242825ca4 ("bpf: Clear buf on error in __bpf_get_task_stack") Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 014fbe5902dccdaef69114abdcdb15e3dbe55e34 ]

The new callchain_finalize function calls the build-id retrieval
(if needed) and zeroes the buffer. This makes things easier for
preemption fix in following change.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-8-jolsa@kernel.org
Stable-dep-of: f5d242825ca4 ("bpf: Clear buf on error in __bpf_get_task_stack")
Signed-off-by: Sasha Levin <sashal@kernel.org>
bpf: Factor callchain_store function from __bpf_get_stack 2026-09-14T11:40:22+00:00 Jiri Olsa jolsa@kernel.org 2026-08-03T21:01:43+00:00 b61aacc66a1f6845ebc1450a406b1795af53ffc3 [ Upstream commit cbb99938e7935c9f62c891573a4b09690e5e22de ] The new callchain_store function stores trace entries buffer into user supplied buffer. It covers both just-ip and buildid data. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-7-jolsa@kernel.org Stable-dep-of: f5d242825ca4 ("bpf: Clear buf on error in __bpf_get_task_stack") Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit cbb99938e7935c9f62c891573a4b09690e5e22de ]

The new callchain_store function stores trace entries buffer into
user supplied buffer. It covers both just-ip and buildid data.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-7-jolsa@kernel.org
Stable-dep-of: f5d242825ca4 ("bpf: Clear buf on error in __bpf_get_task_stack")
Signed-off-by: Sasha Levin <sashal@kernel.org>
bpf: Disable preemption in bpf_get_stackid 2026-09-11T09:51:25+00:00 Jiri Olsa jolsa@kernel.org 2026-09-08T12:48:13+00:00 c4efdc1d4cb3e9183e41a1b2a5e0901dbbb31cdf [ Upstream commit 15f1bd8574662f1b7b26aaa2e23ebf4066f0117d ] The get_perf_callchain call needs disabled preemption plus we need it disabled as long as we access its returned trace entries buffer. Note the bpf_get_stackid_pe function is executed already with preemption disabled. Fixes: d5a3b1f69186 ("bpf: introduce BPF_MAP_TYPE_STACK_TRACE") Reported-by: Tao Chen <chen.dylane@linux.dev> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/bpf/20260803210149.296496-6-jolsa@kernel.org Closes: https://lore.kernel.org/bpf/20260206090653.1336687-2-chen.dylane@linux.dev/ Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 15f1bd8574662f1b7b26aaa2e23ebf4066f0117d ]

The get_perf_callchain call needs disabled preemption plus we need
it disabled as long as we access its returned trace entries buffer.

Note the bpf_get_stackid_pe function is executed already with
preemption disabled.

Fixes: d5a3b1f69186 ("bpf: introduce BPF_MAP_TYPE_STACK_TRACE")
Reported-by: Tao Chen <chen.dylane@linux.dev>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-6-jolsa@kernel.org

Closes: https://lore.kernel.org/bpf/20260206090653.1336687-2-chen.dylane@linux.dev/
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
bpf: Use stack id functions instead of __bpf_get_stackid 2026-09-11T09:51:25+00:00 Jiri Olsa jolsa@kernel.org 2026-09-08T12:48:12+00:00 f1e97925827c5059fec3610e95aed4c9a3dfec79 [ Upstream commit 09b3fd6caa0b57f8a39254ee5db3af30bdd53c18 ] Replacing __bpf_get_stackid calls with sequence of following functions: stackid_fastpath stackid_new_bucket stackid_install This makes code more structured and allows us to easily disable preemption only in bpf_get_stackid in following changes. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-5-jolsa@kernel.org Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid") Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 09b3fd6caa0b57f8a39254ee5db3af30bdd53c18 ]

Replacing __bpf_get_stackid calls with sequence of following functions:

  stackid_fastpath
  stackid_new_bucket
  stackid_install

This makes code more structured and allows us to easily disable
preemption only in bpf_get_stackid in following changes.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-5-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
bpf: Factor stackid_new_bucket from __bpf_get_stackid 2026-09-11T09:51:25+00:00 Jiri Olsa jolsa@kernel.org 2026-09-08T12:48:11+00:00 b3024de68b817440040f26f2a56a666d708c95ec [ Upstream commit bb4e6f4e1b68fe60c04ca04c564c6624e837dbf4 ] The new stackid_new_bucket allocates the new bucket and initializes it with the trace data. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-4-jolsa@kernel.org Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid") Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit bb4e6f4e1b68fe60c04ca04c564c6624e837dbf4 ]

The new stackid_new_bucket allocates the new bucket and initializes it
with the trace data.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-4-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
bpf: Factor stackid_fastpath function from __bpf_get_stackid 2026-09-11T09:51:24+00:00 Jiri Olsa jolsa@kernel.org 2026-09-08T12:48:10+00:00 370069c1f627c2bb910eea768065e434eb7ed562 [ Upstream commit 0ca56befcffec3a6c9d1842eae06c74e1cf41f11 ] The new stackid_fastpath does the fast stack hash and trace check, that does not need new bucket allocation. It covers both just-ip and buildid code paths. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-3-jolsa@kernel.org Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid") Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 0ca56befcffec3a6c9d1842eae06c74e1cf41f11 ]

The new stackid_fastpath does the fast stack hash and trace check, that
does not need new bucket allocation. It covers both just-ip and buildid
code paths.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-3-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
bpf: Factor stackid_init function from __bpf_get_stackid 2026-09-11T09:51:24+00:00 Jiri Olsa jolsa@kernel.org 2026-09-08T12:48:09+00:00 b8cf96ff8c113a54a9055c1d09adad032706bede [ Upstream commit 15b837759a97237d647962f9943afe0d55af615a ] The new stackid_init function stores all the necessary bits for stackid trace and it will be used by other functions in following changes. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260803210149.296496-2-jolsa@kernel.org Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid") Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 15b837759a97237d647962f9943afe0d55af615a ]

The new stackid_init function stores all the necessary bits for stackid
trace and it will be used by other functions in following changes.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260803210149.296496-2-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>