mm: nommu: add sysctl_max_map_count() check for do_mmap()

The sysctl variable vm.max_map_count (sysctl_max_map_count) is not exposed
under !MMU configurations, but its default value (DEFAULT_MAX_MAP_COUNT)
is still used as a allocation limit.  Currently, this limit is enforced
when a VMA entry is split into two chunks (split_vma()), but it is not
checked during initial allocation (do_mmap()).  As a result, if a user
requests a large number of memory allocations, the system will continue
allocating until it hits an Out-Of-Memory (OOM) condition.

This commit introduces a check at the beginning of do_mmap() in nommu.c to
prevent this situation.

This issue was detected using the Linux Test Project (LTP) test linked
below.

Link: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/munmap/munmap04.c
Link: https://lore.kernel.org/20260702012830.667205-1-thehajime@gmail.com
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

0 files changed, 0 insertions, 0 deletions