| Age | Commit message (Collapse) | Author |
|---|
|
Extend the allocinfo filtering mechanism to allow users to filter tags
based on their accuracy.
[abhishekbapat@google.com: move `inaccurate` filtering criteria from `struct allocinfo_tag` to `struct allocinfo_filter`]
Link: https://lore.kernel.org/e4e49ec4a5960292aeeb9e196526c18dc95228a2.1785867739.git.abhishekbapat@google.com
Link: https://lore.kernel.org/396a5e4bc3b2990223ab355f2cd3ceb6aa15499e.1783532853.git.abhishekbapat@google.com
Signed-off-by: Abhishek Bapat <abhishekbapat@google.com>
Acked-by: Hao Ge <hao.ge@linux.dev>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Sourav Panda <souravpanda@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Extend the allocinfo filtering mechanism to allow users to filter tags
based on the total number of bytes allocated [min_size, max_size]. The
size range is inclusive.
Filtering by size involves retrieving allocinfo per-CPU counters, which is
an expensive operation. Hence, the performance of size-based filtering
will be worse than other filters.
Link: https://lore.kernel.org/0a7653b70ae0d64e967fbea0e933bc35f8ac656e.1783532853.git.abhishekbapat@google.com
Signed-off-by: Abhishek Bapat <abhishekbapat@google.com>
Acked-by: Hao Ge <hao.ge@linux.dev>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Sourav Panda <souravpanda@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Extend the capability of the IOCTL mechanism to filter allocations based
on tag's module name, function name, file name and line number.
Link: https://lore.kernel.org/6a6100c0c58cb2911f39126b9fe177a8c17db16f.1783532853.git.abhishekbapat@google.com
Signed-off-by: Abhishek Bapat <abhishekbapat@google.com>
Acked-by: Hao Ge <hao.ge@linux.dev>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Sourav Panda <souravpanda@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "alloc_tag: introduce IOCTL-based filtering for MAP", v8.
Currently, memory allocation profiling data is primarily exposed through
/proc/allocinfo. While useful for manual inspection, this text-based
interface poses challenges for production monitoring and large-scale
analysis:
1. Userspace must parse large amounts of text to extract specific
fields.
2. To find specific tags, userspace must read the entire dataset,
requiring many context switches and high data copying.
3. The kernel currently aggregates per-CPU counters for every allocation
size, even those the user intends to filter out immediately.
This series introduces a new IOCTL-based binary interface for allocinfo
that supports kernel-side filtering. By allowing the user to specify a
filter mask, we significantly reduce the work performed in-kernel and the
amount of data transferred to userspace. The IOCTL mechanism was chosen
for allocinfo to address the per-CPU counter aggregation bottleneck. A
traditional read() operation must report the total allocation count and
sizes for every code tag in the system. Doing so requires iterating
across all CPUs to sum their per-CPU counters for thousands of tags, which
introduces substantial runtime overhead.
The IOCTL interface allows userspace to push selective filtering criteria
directly into the kernel before the per-CPU counter aggregation. The
kernel aggregates per-CPU counters only for a small subset of tags that
match the filter. This results in significant performance improvement.
Beyond fast filtered retrieval, the IOCTL foundation allows introducing a
context capture mechanism in the future to capture the context for
specific allocations.
Performance measurements were conducted on an Intel Xeon Platinum 8481C
(224 CPUs) with caches dropped before each run.
The IOCTL mechanism shows a ~20x performance improvement for
filtered queries. The kernel avoids the expensive per-CPU counter
aggregation (alloc_tag_read) for any tags that fail the initial string
or location filters.
Scenario 1: Specific File Filtering (arch/x86/events/rapl.c)
1. Traditional (cat /proc/allocinfo | grep): 22ms (sys)
2. IOCTL Interface: 1ms (sys)
Scenario 2: Compound Filtering (Filename + Size)
1. Traditional: (cat ... | grep | awk): 21ms (sys)
2. IOCTL Interface: 1ms (sys)
Scenario 3: Size-Based Filtering (min_size = 1MB)
1. Traditional: (cat ... | awk): 21ms (sys)
2. IOCTL Interface: 14ms (sys)
This patch (of 6):
Add the following ioctl commands for /proc/allocinfo file:
ALLOCINFO_IOC_CONTENT_ID - gets content identifier which can be used to
check whether the file content has changed specifically due to module
load/unload. Every time a module is loaded / unloaded, the returned value
will be different. By comparing the identifier value at the beginning and
at the end of the content retrieval operation, users can validate
retrieved information for consistency.
ALLOCINFO_IOC_GET_AT - gets the record at the specified position. This is
the position of a record in /proc/allocinfo.
ALLOCINFO_IOC_GET_NEXT - gets the record next to the last retrieved one.
If no records were previously retrieved, returns the first record.
Note, function file and module names often have the same prefixes,
therefore when filtering for them, we compare the last 64 characters to
minimize the chances of name collisions.
[akpm@linux-foundation.org: include compat.h, per Suren]
Closes: https://lore.kernel.org/oe-kbuild-all/202607091820.qbjlGhKK-lkp@intel.com/
Link: https://lore.kernel.org/cover.1783532853.git.abhishekbapat@google.com
Link: https://lore.kernel.org/15596de2607ef13e7c77c6d74763f4ae992ec475.1783532853.git.abhishekbapat@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Abhishek Bapat <abhishekbapat@google.com>
Acked-by: Hao Ge <hao.ge@linux.dev>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Sourav Panda <souravpanda@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Currently, userspace has limited visibility into the exact active runtime
state of memory allocation profiling and its page extension compression
('sysctl.vm.mem_profiling={0|1|never}[,compressed]').
While reading the sysctl provides basic on/off status, it is currently
impossible for userspace to natively determine whether page-tag
compression was successfully enabled without scraping dmesg boot logs.
Add a new read-only sysctl representing how compression was configured
at boot time.
Link: https://lore.kernel.org/c795f8089f82841e8a6e00d7ca286da2b23aeb7b.1785950530.git.abhishekbapat@google.com
Signed-off-by: Abhishek Bapat <abhishekbapat@google.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Cc: Hao Ge <hao.ge@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Now that alloc_pages has an entrypoint that allows passing alloc_flags, we
can take advantage of this to start removing GFP flags that are only used
for mm-internal stuff.
This requires also plumbing the alloc_flags into some more of the
allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags
arg to go along with its callees, and we now need to pass those flags
deeper into the allocator so they can reach the alloc_tag code.
While moving the flag definition into page_alloc.h, also update the
comment per Hao's suggestion.
No functional change intended.
Link: https://lore.kernel.org/all/b4916118-3537-4e19-8bc8-1d103dd0d225@linux.dev/
Link: https://lore.kernel.org/20260703-alloc-trylock-v5-15-c87b714e19d3@google.com
Signed-off-by: Brendan Jackman <jackmanb@google.com>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Tested-by: Hao Ge <hao.ge@linux.dev>
Acked-by: Hao Ge <hao.ge@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The alloc tagging work is really mm-specific, so move alloc_tag.c to mm/
and additionally update the MAINTAINERS entry to place it within memory
management and port over the Kconfig and Makefile code to mm.
Link: https://lore.kernel.org/20260625184857.2193482-3-surenb@google.com
Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: SeongJae Park <sj@kernel.org>
Tested-by: Hao Ge <hao.ge@linux.dev>
Acked-by: Hao Ge <hao.ge@linux.dev>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Harry Yoo (Oracle) <harry@kernel.org>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
