linux.git/mm/compaction.c, branch v3.5

mm, thp: abort compaction if migration page cannot be charged to memcg

2012-07-11T23:04:43+00:00

If page migration cannot charge the temporary page to the memcg,
migrate_pages() will return -ENOMEM.  This isn't considered in memory
compaction however, and the loop continues to iterate over all
pageblocks trying to isolate and migrate pages.  If a small number of
very large memcgs happen to be oom, however, these attempts will mostly
be futile leading to an enormous amout of cpu consumption due to the
page migration failures.

This patch will short circuit and fail memory compaction if
migrate_pages() returns -ENOMEM.  COMPACT_PARTIAL is returned in case
some migrations were successful so that the page allocator will retry.

Signed-off-by: David Rientjes 
Acked-by: Mel Gorman 
Cc: Minchan Kim 
Cc: Kamezawa Hiroyuki 
Cc: Rik van Riel 
Cc: Andrea Arcangeli 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Revert "mm: compaction: handle incorrect MIGRATE_UNMOVABLE type pageblocks"

2012-06-04T03:05:57+00:00

This reverts commit 5ceb9ce6fe9462a298bb2cd5c9f1ca6cb80a0199.

That commit seems to be the cause of the mm compation list corruption
issues that Dave Jones reported.  The locking (or rather, absense
there-of) is dubious, as is the use of the 'page' variable once it has
been found to be outside the pageblock range.

So revert it for now, we can re-visit this for 3.6.  If we even need to:
as Minchan Kim says, "The patch wasn't a bug fix and even test workload
was very theoretical".

Reported-and-tested-by: Dave Jones 
Acked-by: Hugh Dickins 
Acked-by: KOSAKI Motohiro 
Acked-by: Minchan Kim 
Cc: Bartlomiej Zolnierkiewicz 
Cc: Kyungmin Park 
Cc: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memcg: apply add/del_page to lruvec

2012-05-29T23:22:28+00:00

Take lruvec further: pass it instead of zone to add_page_to_lru_list() and
del_page_from_lru_list(); and pagevec_lru_move_fn() pass lruvec down to
its target functions.

This cleanup eliminates a swathe of cruft in memcontrol.c, including
mem_cgroup_lru_add_list(), mem_cgroup_lru_del_list() and
mem_cgroup_lru_move_lists() - which never actually touched the lists.

In their place, mem_cgroup_page_lruvec() to decide the lruvec, previously
a side-effect of add, and mem_cgroup_update_lru_size() to maintain the
lru_size stats.

Whilst these are simplifications in their own right, the goal is to bring
the evaluation of lruvec next to the spin_locking of the lrus, in
preparation for a future patch.

Signed-off-by: Hugh Dickins 
Cc: KOSAKI Motohiro 
Acked-by: KAMEZAWA Hiroyuki 
Acked-by: Michal Hocko 
Acked-by: Konstantin Khlebnikov 
Cc: Johannes Weiner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: remove lru type checks from __isolate_lru_page()

2012-05-29T23:22:25+00:00

After patch "mm: forbid lumpy-reclaim in shrink_active_list()" we can
completely remove anon/file and active/inactive lru type filters from
__isolate_lru_page(), because isolation for 0-order reclaim always
isolates pages from right lru list.  And pages-isolation for lumpy
shrink_inactive_list() or memory-compaction anyway allowed to isolate
pages from all evictable lru lists.

Signed-off-by: Konstantin Khlebnikov 
Acked-by: KAMEZAWA Hiroyuki 
Cc: Hugh Dickins 
Acked-by: Michal Hocko 
Cc: Glauber Costa 
Cc: Johannes Weiner 
Cc: Minchan Kim 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: compaction: handle incorrect MIGRATE_UNMOVABLE type pageblocks

2012-05-29T23:22:22+00:00

When MIGRATE_UNMOVABLE pages are freed from MIGRATE_UNMOVABLE type
pageblock (and some MIGRATE_MOVABLE pages are left in it) waiting until an
allocation takes ownership of the block may take too long.  The type of
the pageblock remains unchanged so the pageblock cannot be used as a
migration target during compaction.

Fix it by:

* Adding enum compact_mode (COMPACT_ASYNC_[MOVABLE,UNMOVABLE], and
  COMPACT_SYNC) and then converting sync field in struct compact_control
  to use it.

* Adding nr_pageblocks_skipped field to struct compact_control and
  tracking how many destination pageblocks were of MIGRATE_UNMOVABLE type.
   If COMPACT_ASYNC_MOVABLE mode compaction ran fully in
  try_to_compact_pages() (COMPACT_COMPLETE) it implies that there is not a
  suitable page for allocation.  In this case then check how if there were
  enough MIGRATE_UNMOVABLE pageblocks to try a second pass in
  COMPACT_ASYNC_UNMOVABLE mode.

* Scanning the MIGRATE_UNMOVABLE pageblocks (during COMPACT_SYNC and
  COMPACT_ASYNC_UNMOVABLE compaction modes) and building a count based on
  finding PageBuddy pages, page_count(page) == 0 or PageLRU pages.  If all
  pages within the MIGRATE_UNMOVABLE pageblock are in one of those three
  sets change the whole pageblock type to MIGRATE_MOVABLE.

My particular test case (on a ARM EXYNOS4 device with 512 MiB, which means
131072 standard 4KiB pages in 'Normal' zone) is to:

- allocate 120000 pages for kernel's usage
- free every second page (60000 pages) of memory just allocated
- allocate and use 60000 pages from user space
- free remaining 60000 pages of kernel memory
  (now we have fragmented memory occupied mostly by user space pages)
- try to allocate 100 order-9 (2048 KiB) pages for kernel's usage

The results:
- with compaction disabled I get 11 successful allocations
- with compaction enabled - 14 successful allocations
- with this patch I'm able to get all 100 successful allocations

NOTE: If we can make kswapd aware of order-0 request during compaction, we
can enhance kswapd with changing mode to COMPACT_ASYNC_FULL
(COMPACT_ASYNC_MOVABLE + COMPACT_ASYNC_UNMOVABLE).  Please see the
following thread:

	http://marc.info/?l=linux-mm&m=133552069417068&w=2

[minchan@kernel.org: minor cleanups]
Cc: Mel Gorman 
Cc: Minchan Kim 
Cc: Rik van Riel 
Cc: Marek Szyprowski 
Signed-off-by: Bartlomiej Zolnierkiewicz 
Signed-off-by: Kyungmin Park 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: mmzone: MIGRATE_CMA migration type added

2012-05-21T13:09:32+00:00

The MIGRATE_CMA migration type has two main characteristics:
(i) only movable pages can be allocated from MIGRATE_CMA
pageblocks and (ii) page allocator will never change migration
type of MIGRATE_CMA pageblocks.

This guarantees (to some degree) that page in a MIGRATE_CMA page
block can always be migrated somewhere else (unless there's no
memory left in the system).

It is designed to be used for allocating big chunks (eg. 10MiB)
of physically contiguous memory.  Once driver requests
contiguous memory, pages from MIGRATE_CMA pageblocks may be
migrated away to create a contiguous block.

To minimise number of migrations, MIGRATE_CMA migration type
is the last type tried when page allocator falls back to other
migration types when requested.

Signed-off-by: Michal Nazarewicz 
Signed-off-by: Marek Szyprowski 
Signed-off-by: Kyungmin Park 
Acked-by: Mel Gorman 
Reviewed-by: KAMEZAWA Hiroyuki 
Tested-by: Rob Clark 
Tested-by: Ohad Ben-Cohen 
Tested-by: Benjamin Gaignard 
Tested-by: Robert Nelson 
Tested-by: Barry Song

mm: compaction: export some of the functions

2012-05-21T13:09:30+00:00

This commit exports some of the functions from compaction.c file
outside of it adding their declaration into internal.h header
file so that other mm related code can use them.

This forced compaction.c to always be compiled (as opposed to being
compiled only if CONFIG_COMPACTION is defined) but as to avoid
introducing code that user did not ask for, part of the compaction.c
is now wrapped in on #ifdef.

Signed-off-by: Michal Nazarewicz 
Signed-off-by: Marek Szyprowski 
Acked-by: Mel Gorman 
Reviewed-by: KAMEZAWA Hiroyuki 
Tested-by: Rob Clark 
Tested-by: Ohad Ben-Cohen 
Tested-by: Benjamin Gaignard 
Tested-by: Robert Nelson 
Tested-by: Barry Song

mm: compaction: introduce isolate_freepages_range()

2012-05-21T13:09:29+00:00

This commit introduces isolate_freepages_range() function which
generalises isolate_freepages_block() so that it can be used on
arbitrary PFN ranges.

isolate_freepages_block() is left with only minor changes.

Signed-off-by: Michal Nazarewicz 
Signed-off-by: Marek Szyprowski 
Acked-by: Mel Gorman 
Reviewed-by: KAMEZAWA Hiroyuki 
Tested-by: Rob Clark 
Tested-by: Ohad Ben-Cohen 
Tested-by: Benjamin Gaignard 
Tested-by: Robert Nelson 
Tested-by: Barry Song

mm: compaction: introduce map_pages()

2012-05-21T13:09:28+00:00

This commit creates a map_pages() function which map pages freed
using split_free_pages().  This merely moves some code from
isolate_freepages() so that it can be reused in other places.

Signed-off-by: Michal Nazarewicz 
Signed-off-by: Marek Szyprowski 
Acked-by: Mel Gorman 
Reviewed-by: KAMEZAWA Hiroyuki 
Tested-by: Robert Nelson 
Tested-by: Barry Song

mm: compaction: introduce isolate_migratepages_range()

2012-05-21T13:09:27+00:00

This commit introduces isolate_migratepages_range() function which
extracts functionality from isolate_migratepages() so that it can be
used on arbitrary PFN ranges.

isolate_migratepages() function is implemented as a simple wrapper
around isolate_migratepages_range().

Signed-off-by: Michal Nazarewicz 
Signed-off-by: Marek Szyprowski 
Acked-by: Mel Gorman 
Reviewed-by: KAMEZAWA Hiroyuki 
Tested-by: Rob Clark 
Tested-by: Ohad Ben-Cohen 
Tested-by: Benjamin Gaignard 
Tested-by: Robert Nelson 
Tested-by: Barry Song