linux.git/mm/bootmem.c, branch v2.6.30

bootmem, x86: further fixes for arch-specific bootmem wrapping

2009-03-01T07:06:56+00:00

Impact: fix new breakages introduced by previous fix

Commit c132937556f56ee4b831ef4b23f1846e05fde102 tried to clean up
bootmem arch wrapper but it wasn't quite correct.  Before the commit,
the followings were broken.

* Low level interface functions prefixed with __ ignored arch
  preference.

* reserve_bootmem(...) can't be mapped into
  reserve_bootmem_node(NODE_DATA(0)->bdata, ...) because the node is
  not preference here.  The region specified MUST fall into the
  specified region; otherwise, it will panic.

After the commit,

* If allocation fails for the arch preferred node, it should fallback
  to whatever is available.  Instead, it simply failed allocation.

There are too many internal details to allow generic wrapping and
still keep things simple for archs.  Plus, all that arch wants is a
way to prefer certain node over another.

This patch drops the generic wrapping around alloc_bootmem_core() and
add alloc_bootmem_core() instead.  If necessary, arch can define
bootmem_arch_referred_node() macro or function which takes all
allocation information and returns the preferred node.  bootmem
generic code will always try the preferred node first and then
fallback to other nodes as usual.

Breakages noted and changes reviewed by Johannes Weiner.

Signed-off-by: Tejun Heo 
Acked-by: Johannes Weiner

bootmem: clean up arch-specific bootmem wrapping

2009-02-24T02:57:20+00:00

Impact: cleaner and consistent bootmem wrapping

By setting CONFIG_HAVE_ARCH_BOOTMEM_NODE, archs can define
arch-specific wrappers for bootmem allocation.  However, this is done
a bit strangely in that only the high level convenience macros can be
changed while lower level, but still exported, interface functions
can't be wrapped.  This not only is messy but also leads to strange
situation where alloc_bootmem() does what the arch wants it to do but
the equivalent __alloc_bootmem() call doesn't although they should be
able to be used interchangeably.

This patch updates bootmem such that archs can override / wrap the
backend function - alloc_bootmem_core() instead of the highlevel
interface functions to allow simpler and consistent wrapping.  Also,
HAVE_ARCH_BOOTMEM_NODE is renamed to HAVE_ARCH_BOOTMEM.

Signed-off-by: Tejun Heo 
Cc: Johannes Weiner

bootmem: print request details before BUG_ON(them)

2009-01-06T23:59:10+00:00

Moving the request details print-out before the sanity checks that
might panic() enables us to analyse invalid requests without having
access to the line information of the stack dump.

Signed-off-by: Johannes Weiner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

misc: replace FUNCTION with func

2008-10-16T18:21:30+00:00

__FUNCTION__ is gcc-specific, use __func__

Signed-off-by: Harvey Harrison 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: fix aligning of node-relative indexes and offsets

2008-08-20T22:40:31+00:00

Absolute alignment requirements may never be applied to node-relative
offsets.  Andreas Herrmann spotted this flaw when a bootmem allocation on
an unaligned node was itself not aligned because the combination of an
unaligned node with an aligned offset into that node is not garuanteed to
be aligned itself.

This patch introduces two helper functions that align a node-relative
index or offset with respect to the node's starting address so that the
absolute PFN or virtual address that results from combining the two
satisfies the requested alignment.

Then all the broken ALIGN()s in alloc_bootmem_core() are replaced by these
helpers.

Signed-off-by: Johannes Weiner 
Reported-by: Andreas Herrmann 
Debugged-by: Andreas Herrmann 
Reviewed-by: Andreas Herrmann 
Tested-by: Andreas Herrmann 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem allocator: alloc_bootmem_core(): page-align the end offset

2008-08-15T15:35:41+00:00

This is the minimal sequence that jams the allocator:

void *p, *q, *r;
p = alloc_bootmem(PAGE_SIZE);
q = alloc_bootmem(64);
free_bootmem(p, PAGE_SIZE);
p = alloc_bootmem(PAGE_SIZE);
r = alloc_bootmem(64);

after this sequence (assuming that the allocator was empty or page-aligned
before), pointer "q" will be equal to pointer "r".

What's hapenning inside the allocator:
p = alloc_bootmem(PAGE_SIZE);
in allocator: last_end_off == PAGE_SIZE, bitmap contains bits 10000...
q = alloc_bootmem(64);
in allocator: last_end_off == PAGE_SIZE + 64, bitmap contains 11000...
free_bootmem(p, PAGE_SIZE);
in allocator: last_end_off == PAGE_SIZE + 64, bitmap contains 01000...
p = alloc_bootmem(PAGE_SIZE);
in allocator: last_end_off == PAGE_SIZE, bitmap contains 11000...
r = alloc_bootmem(64);

and now:

it finds bit "2", as a place where to allocate (sidx)

it hits the condition

if (bdata->last_end_off && PFN_DOWN(bdata->last_end_off) + 1 == sidx))
start_off = ALIGN(bdata->last_end_off, align);

-you can see that the condition is true, so it assigns start_off =
ALIGN(bdata->last_end_off, align); (that is PAGE_SIZE) and allocates
over already allocated block.

With the patch it tries to continue at the end of previous allocation only
if the previous allocation ended in the middle of the page.

Signed-off-by: Mikulas Patocka 
Acked-by: Johannes Weiner 
Cc: David Miller 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: replace node_boot_start in struct bootmem_data

2008-07-24T17:47:20+00:00

Almost all users of this field need a PFN instead of a physical address,
so replace node_boot_start with node_min_pfn.

[Lee.Schermerhorn@hp.com: fix spurious BUG_ON() in mark_bootmem()]
Signed-off-by: Johannes Weiner 
Cc: 
Signed-off-by: Lee Schermerhorn 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: revisit alloc_bootmem_section

2008-07-24T17:47:20+00:00

Since alloc_bootmem_core does no goal-fallback anymore and just returns
NULL if the allocation fails, we might now use it in alloc_bootmem_section
without all the fixup code for a misplaced allocation.

Also, the limit can be the first PFN of the next section as the semantics
is that the limit is _above_ the allocated region, not within.

Signed-off-by: Johannes Weiner 
Cc: Yasunori Goto 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: Make __alloc_bootmem_low_node fall back to other nodes

2008-07-24T17:47:20+00:00

__alloc_bootmem_node already does this, make the interface consistent.

Signed-off-by: Johannes Weiner 
Cc: Ingo Molnar 
Cc: Yinghai Lu 
Cc: Andi Kleen 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: respect goal more likely

2008-07-24T17:47:20+00:00

The old node-agnostic code tried allocating on all nodes starting from the
one with the lowest range.  alloc_bootmem_core retried without the goal if
it could not satisfy it and so the goal was only respected at all when it
happened to be on the first (lowest page numbers) node (or theoretically
if allocations failed on all nodes before to the one holding the goal).

Introduce a non-panicking helper that starts allocating from the node
holding the goal and falls back only after all thes tries failed, thus
moving the goal fallback code out of alloc_bootmem_core.

Make all other allocation functions benefit from this new helper.

Signed-off-by: Johannes Weiner 
Cc: Ingo Molnar 
Cc: Yinghai Lu 
Cc: Andi Kleen 
Cc: Yasunori Goto 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

linux.git/mm/bootmem.c, branch v2.6.30

bootmem, x86: further fixes for arch-specific bootmem wrapping

bootmem: clean up arch-specific bootmem wrapping

bootmem: print request details before BUG_ON(them)

misc: replace __FUNCTION__ with __func__

bootmem: fix aligning of node-relative indexes and offsets

bootmem allocator: alloc_bootmem_core(): page-align the end offset

bootmem: replace node_boot_start in struct bootmem_data

bootmem: revisit alloc_bootmem_section

bootmem: Make __alloc_bootmem_low_node fall back to other nodes

bootmem: respect goal more likely

misc: replace FUNCTION with func