linux-stable.git/include/linux/gfp_types.h, branch v6.18.35

mm/page_alloc: fix initialization of tags of the huge zero folio with init_on_free

2026-06-01T15:50:41+00:00

commit 6a288a4ddb4a994490505ab5f41c445f8e6b6467 upstream.

__GFP_ZEROTAGS semantics are currently a bit weird, but effectively this
flag is only ever set alongside __GFP_ZERO and __GFP_SKIP_KASAN.

If we run with init_on_free, we will zero out pages during
__free_pages_prepare(), to skip zeroing on the allocation path.

However, when allocating with __GFP_ZEROTAG set, post_alloc_hook() will
consequently not only skip clearing page content, but also skip clearing
tag memory.

Not clearing tags through __GFP_ZEROTAGS is irrelevant for most pages that
will get mapped to user space through set_pte_at() later: set_pte_at() and
friends will detect that the tags have not been initialized yet
(PG_mte_tagged not set), and initialize them.

However, for the huge zero folio, which will be mapped through a PMD
marked as special, this initialization will not be performed, ending up
exposing whatever tags were still set for the pages.

The docs (Documentation/arch/arm64/memory-tagging-extension.rst) state
that allocation tags are set to 0 when a page is first mapped to user
space.  That no longer holds with the huge zero folio when init_on_free is
enabled.

Fix it by decoupling __GFP_ZEROTAGS from __GFP_ZERO, passing to
tag_clear_highpages() whether we want to also clear page content.

Invert the meaning of the tag_clear_highpages() return value to have
clearer semantics.

Reproduced with the huge zero folio by modifying the check_buffer_fill
arm64/mte selftest to use a 2 MiB area, after making sure that pages have
a non-0 tag set when freeing (note that, during boot, we will not actually
initialize tags, but only set KASAN_TAG_KERNEL in the page flags).

	$ ./check_buffer_fill
	1..20
	...
	not ok 17 Check initial tags with private mapping, sync error mode and mmap memory
	not ok 18 Check initial tags with private mapping, sync error mode and mmap/mprotect memory
	...

This code needs more cleanups; we'll tackle that next, like
decoupling __GFP_ZEROTAGS from __GFP_SKIP_KASAN.

[akpm@linux-foundation.org: s/__GPF_ZERO/__GFP_ZERO/, per David]
Link: https://lore.kernel.org/20260421-zerotags-v2-1-05cb1035482e@kernel.org
Fixes: adfb6609c680 ("mm/huge_memory: initialise the tags of the huge zero folio")
Signed-off-by: David Hildenbrand (Arm) 
Reviewed-by: Catalin Marinas 
Tested-by: Lance Yang 
Cc: Brendan Jackman 
Cc: Dev Jain 
Cc: Johannes Weiner 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes (Oracle) 
Cc: Mark Brown 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Ryan Roberts 
Cc: Suren Baghdasaryan 
Cc: Will Deacon 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Greg Kroah-Hartman

mm: warn about illegal __GFP_NOFAIL usage in a more appropriate location and manner

2024-09-09T23:39:03+00:00

Three points for this change:

1. We should consolidate all warnings in one place. Currently, the
   order > 1 warning is in the hotpath, while others are in less
   likely scenarios. Moving all warnings to the slowpath will reduce
   the overhead for order > 1 and increase the visibility of other
   warnings.

2. We currently have two warnings for order: one for order > 1 in
   the hotpath and another for order > costly_order in the laziest
   path. I suggest standardizing on order > 1 since it's been in
   use for a long time.

3. We don't need to check for __GFP_NOWARN in this case. __GFP_NOWARN
   is meant to suppress allocation failure reports, but here we're
   dealing with bug detection, not allocation failures. So replace
   WARN_ON_ONCE_GFP by WARN_ON_ONCE.

[v-songbaohua@oppo.com: also update the doc for __GFP_NOFAIL with order > 1]
  Link: https://lkml.kernel.org/r/20240903223935.1697-1-21cnbao@gmail.com
Link: https://lkml.kernel.org/r/20240830202823.21478-4-21cnbao@gmail.com
Signed-off-by: Barry Song 
Suggested-by: Vlastimil Babka 
Reviewed-by: Vlastimil Babka 
Acked-by: David Hildenbrand 
Acked-by: Michal Hocko 
Cc: Christoph Hellwig 
Cc: Christoph Lameter 
Cc: Davidlohr Bueso 
Cc: David Rientjes 
Cc: "Eugenio Pérez" 
Cc: Hailong.Liu 
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Jason Wang 
Cc: Joonsoo Kim 
Cc: Kees Cook 
Cc: Linus Torvalds 
Cc: Lorenzo Stoakes 
Cc: Maxime Coquelin 
Cc: "Michael S. Tsirkin" 
Cc: Pekka Enberg 
Cc: Roman Gushchin 
Cc: Uladzislau Rezki (Sony) 
Cc: Xie Yongji 
Cc: Xuan Zhuo 
Cc: Yafang Shao 
Signed-off-by: Andrew Morton

mm: document __GFP_NOFAIL must be blockable

2024-09-09T23:39:03+00:00

Non-blocking allocation with __GFP_NOFAIL is not supported and may still
result in NULL pointers (if we don't return NULL, we result in busy-loop
within non-sleepable contexts):

static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
						struct alloc_context *ac)
{
	...
	/*
	 * Make sure that __GFP_NOFAIL request doesn't leak out and make sure
	 * we always retry
	 */
	if (gfp_mask & __GFP_NOFAIL) {
		/*
		 * All existing users of the __GFP_NOFAIL are blockable, so warn
		 * of any new users that actually require GFP_NOWAIT
		 */
		if (WARN_ON_ONCE_GFP(!can_direct_reclaim, gfp_mask))
			goto fail;
		...
	}
	...
fail:
	warn_alloc(gfp_mask, ac->nodemask,
			"page allocation failure: order:%u", order);
got_pg:
	return page;
}

Highlight this in the documentation of __GFP_NOFAIL so that non-mm
subsystems can reject any illegal usage of __GFP_NOFAIL with GFP_ATOMIC,
GFP_NOWAIT, etc.

Link: https://lkml.kernel.org/r/20240830202823.21478-3-21cnbao@gmail.com
Signed-off-by: Barry Song 
Acked-by: Michal Hocko 
Reviewed-by: Christoph Hellwig 
Acked-by: Vlastimil Babka 
Acked-by: Davidlohr Bueso 
Acked-by: David Hildenbrand 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: "Eugenio Pérez" 
Cc: Hailong.Liu 
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Jason Wang 
Cc: Joonsoo Kim 
Cc: Kees Cook 
Cc: Linus Torvalds 
Cc: Lorenzo Stoakes 
Cc: Maxime Coquelin 
Cc: "Michael S. Tsirkin" 
Cc: Pekka Enberg 
Cc: Roman Gushchin 
Cc: Uladzislau Rezki (Sony) 
Cc: Xuan Zhuo 
Cc: Christoph Hellwig 
Cc: Xie Yongji 
Cc: Yafang Shao 
Signed-off-by: Andrew Morton

mm: introduce __GFP_NO_OBJ_EXT flag to selectively prevent slabobj_ext creation

2024-04-26T03:55:51+00:00

Introduce __GFP_NO_OBJ_EXT flag in order to prevent recursive allocations
when allocating slabobj_ext on a slab.

Link: https://lkml.kernel.org/r/20240321163705.3067592-8-surenb@google.com
Signed-off-by: Suren Baghdasaryan 
Reviewed-by: Kees Cook 
Reviewed-by: Pasha Tatashin 
Reviewed-by: Vlastimil Babka 
Tested-by: Kees Cook 
Cc: Alexander Viro 
Cc: Alex Gaynor 
Cc: Alice Ryhl 
Cc: Andreas Hindborg 
Cc: Benno Lossin 
Cc: "Björn Roy Baron" 
Cc: Boqun Feng 
Cc: Christoph Lameter 
Cc: Dennis Zhou 
Cc: Gary Guo 
Cc: Kent Overstreet 
Cc: Miguel Ojeda 
Cc: Peter Zijlstra 
Cc: Tejun Heo 
Cc: Wedson Almeida Filho 
Signed-off-by: Andrew Morton

memblock tests: fix undefined reference to `BIT'

2024-04-04T08:08:33+00:00

commit 772dd0342727 ("mm: enumerate all gfp flags") define gfp flags
with the help of BIT, while gfp_types.h doesn't include header file for
the definition. This through an error on building memblock tests.

Let's include linux/bits.h to fix it.

Signed-off-by: Wei Yang 
CC: Suren Baghdasaryan 
CC: Michal Hocko 
Link: https://lore.kernel.org/r/20240402132701.29744-4-richard.weiyang@gmail.com
Signed-off-by: Mike Rapoport (IBM)

mm: enumerate all gfp flags

2024-03-05T01:01:18+00:00

Introduce GFP bits enumeration to let compiler track the number of used
bits (which depends on the config options) instead of hardcoding them. 
That simplifies __GFP_BITS_SHIFT calculation.

Link: https://lkml.kernel.org/r/20240224015800.2569851-1-surenb@google.com
Suggested-by: Petr Tesařík 
Signed-off-by: Suren Baghdasaryan 
Reviewed-by: Kees Cook 
Reviewed-by: Pasha Tatashin 
Acked-by: Michal Hocko 
Cc: Kent Overstreet 
Cc: Petr Tesarik 
Signed-off-by: Andrew Morton

gfp: gfp_types.h: fix typos & punctuation

2023-12-20T22:48:14+00:00

Correct typos/spellos and punctutation.

Link: https://lkml.kernel.org/r/20231213043316.10128-1-rdunlap@infradead.org
Signed-off-by: Randy Dunlap 
Signed-off-by: Andrew Morton

gfp: include __GFP_NOWARN in GFP_NOWAIT

2023-12-11T00:51:39+00:00

GFP_NOWAIT callers are always prepared for their allocations to fail
because they fail so frequently.  Forcing the callers to remember to add
__GFP_NOWARN is just annoying and leads to an endless stream of patches
for the places where we forgot to add it.

We can now remove __GFP_NOWARN from all the callers which specify
GFP_NOWAIT, but I'd rather wait a cycle and send patches to each
maintainer instead of creating a big pile of merge conflicts.

Link: https://lkml.kernel.org/r/20231109211507.2262419-1-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Signed-off-by: Andrew Morton

kasan: remove PG_skip_kasan_poison flag

2023-03-28T23:20:16+00:00

Code inspection reveals that PG_skip_kasan_poison is redundant with
kasantag, because the former is intended to be set iff the latter is the
match-all tag.  It can also be observed that it's basically pointless to
poison pages which have kasantag=0, because any pages with this tag would
have been pointed to by pointers with match-all tags, so poisoning the
pages would have little to no effect in terms of bug detection. 
Therefore, change the condition in should_skip_kasan_poison() to check
kasantag instead, and remove PG_skip_kasan_poison and associated flags.

Link: https://lkml.kernel.org/r/20230310042914.3805818-3-pcc@google.com
Link: https://linux-review.googlesource.com/id/I57f825f2eaeaf7e8389d6cf4597c8a5821359838
Signed-off-by: Peter Collingbourne 
Reviewed-by: Andrey Konovalov 
Cc: Andrey Ryabinin 
Cc: Catalin Marinas 
Cc: Evgenii Stepanov 
Cc: Vincenzo Frascino 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

mm: discard __GFP_ATOMIC

2023-02-03T06:33:13+00:00

__GFP_ATOMIC serves little purpose.  Its main effect is to set
ALLOC_HARDER which adds a few little boosts to increase the chance of an
allocation succeeding, one of which is to lower the water-mark at which it
will succeed.

It is *always* paired with __GFP_HIGH which sets ALLOC_HIGH which also
adjusts this watermark.  It is probable that other users of __GFP_HIGH
should benefit from the other little bonuses that __GFP_ATOMIC gets.

__GFP_ATOMIC also gives a warning if used with __GFP_DIRECT_RECLAIM.
There is little point to this.  We already get a might_sleep() warning if
__GFP_DIRECT_RECLAIM is set.

__GFP_ATOMIC allows the "watermark_boost" to be side-stepped.  It is
probable that testing ALLOC_HARDER is a better fit here.

__GFP_ATOMIC is used by tegra-smmu.c to check if the allocation might
sleep.  This should test __GFP_DIRECT_RECLAIM instead.

This patch:
 - removes __GFP_ATOMIC
 - allows __GFP_HIGH allocations to ignore watermark boosting as well
   as GFP_ATOMIC requests.
 - makes other adjustments as suggested by the above.

The net result is not change to GFP_ATOMIC allocations.  Other
allocations that use __GFP_HIGH will benefit from a few different extra
privileges.  This affects:
  xen, dm, md, ntfs3
  the vermillion frame buffer
  hibernation
  ksm
  swap
all of which likely produce more benefit than cost if these selected
allocation are more likely to succeed quickly.

[mgorman: Minor adjustments to rework on top of a series]
Link: https://lkml.kernel.org/r/163712397076.13692.4727608274002939094@noble.neil.brown.name
Link: https://lkml.kernel.org/r/20230113111217.14134-7-mgorman@techsingularity.net
Signed-off-by: NeilBrown 
Signed-off-by: Mel Gorman 
Acked-by: Vlastimil Babka 
Acked-by: Michal Hocko 
Cc: Matthew Wilcox 
Cc: Thierry Reding 
Signed-off-by: Andrew Morton