linux.git/mm/huge_memory.c, branch v2.6.38

thp+memcg-numa: fix BUG at include/linux/mm.h:370!

2011-03-14T15:29:50+00:00

THP's collapse_huge_page() has an understandable but ugly difference
in when its huge page is allocated: inside if NUMA but outside if not.
It's hardly surprising that the memcg failure path forgot that, freeing
the page in the non-NUMA case, then hitting a VM_BUG_ON in get_page()
(or even worse, using the freed page).

Signed-off-by: Hugh Dickins 
Reviewed-by: Minchan Kim 
Signed-off-by: Linus Torvalds

mm: use correct numa policy node for transparent hugepages

2011-03-05T01:53:39+00:00

Pass down the correct node for a transparent hugepage allocation.  Most
callers continue to use the current node, however the hugepaged daemon
now uses the previous node of the first to be collapsed page instead.
This ensures that khugepaged does not mess up local memory for an
existing process which uses local policy.

The choice of node is somewhat primitive currently: it just uses the
node of the first page in the pmd range.  An alternative would be to
look at multiple pages and use the most popular node.  I used the
simplest variant for now which should work well enough for the case of
all pages being on the same node.

[akpm@linux-foundation.org: coding-style fixes]
Acked-by: Andrea Arcangeli 
Signed-off-by: Andi Kleen 
Reviewed-by: KAMEZAWA Hiroyuki 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: preserve original node for transparent huge page copies

2011-03-05T01:53:39+00:00

This makes a difference for LOCAL policy, where the node cannot be
determined from the policy itself, but has to be gotten from the original
page.

Acked-by: Andrea Arcangeli 
Signed-off-by: Andi Kleen 
Reviewed-by: KAMEZAWA Hiroyuki 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: change alloc_pages_vma to pass down the policy node for local policy

2011-03-05T01:53:39+00:00

Currently alloc_pages_vma() always uses the local node as policy node for
the LOCAL policy.  Pass this node down as an argument instead.

No behaviour change from this patch, but will be needed for followons.

Acked-by: Andrea Arcangeli 
Signed-off-by: Andi Kleen 
Reviewed-by: KAMEZAWA Hiroyuki 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

thp: prevent hugepages during args/env copying into the user stack

2011-02-15T23:21:11+00:00

Transparent hugepages can only be created if rmap is fully
functional. So we must prevent hugepages to be created while
is_vma_temporary_stack() is true.

This also optmizes away some harmless but unnecessary setting of
khugepaged_scan.address and it switches some BUG_ON to VM_BUG_ON.

Signed-off-by: Andrea Arcangeli 
Acked-by: Rik van Riel 
Signed-off-by: Linus Torvalds

memcg: fix leak of accounting at failure path of hugepage collapsing

2011-02-12T00:12:20+00:00

mem_cgroup_uncharge_page() should be called in all failure cases after
mem_cgroup_charge_newpage() is called in huge_memory.c::collapse_huge_page()

 [ 4209.076861] BUG: Bad page state in process khugepaged  pfn:1e9800
 [ 4209.077601] page:ffffea0006b14000 count:0 mapcount:0 mapping:          (null) index:0x2800
 [ 4209.078674] page flags: 0x40000000004000(head)
 [ 4209.079294] pc:ffff880214a30000 pc->flags:2146246697418756 pc->mem_cgroup:ffffc9000177a000
 [ 4209.082177] (/A)
 [ 4209.082500] Pid: 31, comm: khugepaged Not tainted 2.6.38-rc3-mm1 #1
 [ 4209.083412] Call Trace:
 [ 4209.083678]  [] ? bad_page+0xe4/0x140
 [ 4209.084240]  [] ? free_pages_prepare+0xd6/0x120
 [ 4209.084837]  [] ? rwsem_down_failed_common+0xbd/0x150
 [ 4209.085509]  [] ? __free_pages_ok+0x32/0xe0
 [ 4209.086110]  [] ? free_compound_page+0x1b/0x20
 [ 4209.086699]  [] ? __put_compound_page+0x1c/0x30
 [ 4209.087333]  [] ? put_compound_page+0x4d/0x200
 [ 4209.087935]  [] ? put_page+0x45/0x50
 [ 4209.097361]  [] ? khugepaged+0x9e9/0x1430
 [ 4209.098364]  [] ? autoremove_wake_function+0x0/0x40
 [ 4209.099121]  [] ? khugepaged+0x0/0x1430
 [ 4209.099780]  [] ? kthread+0x96/0xa0
 [ 4209.100452]  [] ? kernel_thread_helper+0x4/0x10
 [ 4209.101214]  [] ? kthread+0x0/0xa0
 [ 4209.101842]  [] ? kernel_thread_helper+0x0/0x10

Signed-off-by: KAMEZAWA Hiroyuki 
Acked-by: Daisuke Nishimura 
Reviewed-by: Johannes Weiner 
Cc: Andrea Arcangeli 
Reviewed-by: Minchan Kim 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

thp: fix the wrong reported address of hwpoisoned hugepages

2011-02-03T00:03:19+00:00

When the tail page of THP is poisoned, the head page will be poisoned too.
 And the wrong address, address of head page, will be sent with sigbus
always.

So when the poisoned page is used by Guest OS which is running on KVM,
after the address changing(hva->gpa) by qemu, the unexpected process on
Guest OS will be killed by sigbus.

What we expected is that the process using the poisoned tail page could be
killed on Guest OS, but not that the process using the healthy head page
is killed.

Since it is not good to poison the healthy page, avoid poisoning other
than the page which is really poisoned.
  (While we poison all pages in a huge page in case of hugetlb,
   we can do this for THP thanks to split_huge_page().)

Here we fix two parts:
  1. Isolate the poisoned page only to make sure
     the reported address is the address of poisoned page.
  2. make the poisoned page work as the poisoned regular page.

[akpm@linux-foundation.org: fix spello in comment]
Signed-off-by: Jin Dongming 
Reviewed-by: Hidetoshi Seto 
Cc: Andrea Arcangeli 
Cc: Andi Kleen 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memcg: fix USED bit handling at uncharge in THP

2011-01-21T01:02:06+00:00

Now, under THP:

at charge:
  - PageCgroupUsed bit is set to all page_cgroup on a hugepage.
    ....set to 512 pages.
at uncharge
  - PageCgroupUsed bit is unset on the head page.

So, some pages will remain with "Used" bit.

This patch fixes that Used bit is set only to the head page.
Used bits for tail pages will be set at splitting if necessary.

This patch adds this lock order:
   compound_lock() -> page_cgroup_move_lock().

[akpm@linux-foundation.org: fix warning]
Signed-off-by: KAMEZAWA Hiroyuki 
Cc: Daisuke Nishimura 
Cc: Balbir Singh 
Cc: Johannes Weiner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

thp: keep highpte mapped until it is no longer needed

2011-01-21T01:02:05+00:00

Two users reported THP-related crashes on 32-bit x86 machines.  Their oops
reports indicated an invalid pte, and subsequent code inspection showed
that the highpte is actually used after unmap.

The fix is to unmap the pte only after all operations against it are
finished.

Signed-off-by: Johannes Weiner 
Reported-by: Ilya Dryomov 
Reported-by: werner 
Cc: Andrea Arcangeli 
Tested-by: Ilya Dryomov 
Tested-by: Steven Rostedt 
Signed-off-by: Linus Torvalds

thp: khugepaged: make khugepaged aware about madvise

2011-01-14T01:32:47+00:00

MADV_HUGEPAGE and MADV_NOHUGEPAGE were fully effective only if run after
mmap and before touching the memory.  While this is enough for most
usages, it's little effort to make madvise more dynamic at runtime on an
existing mapping by making khugepaged aware about madvise.

MADV_HUGEPAGE: register in khugepaged immediately without waiting a page
fault (that may not ever happen if all pages are already mapped and the
"enabled" knob was set to madvise during the initial page faults).

MADV_NOHUGEPAGE: skip vmas marked VM_NOHUGEPAGE in khugepaged to stop
collapsing pages where not needed.

[akpm@linux-foundation.org: tweak comment]
Signed-off-by: Andrea Arcangeli 
Cc: Michael Kerrisk 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds