linux.git/mm/page-writeback.c, branch v2.6.25

writeback: speed up writeback of big dirty files

2008-02-05T17:44:19+00:00

After making dirty a 100M file, the normal behavior is to start the
writeback for all data after 30s delays.  But sometimes the following
happens instead:

	- after 30s:    ~4M
	- after 5s:     ~4M
	- after 5s:     all remaining 92M

Some analyze shows that the internal io dispatch queues goes like this:

		s_io            s_more_io
		-------------------------
	1)	100M,1K         0
	2)	1K              96M
	3)	0               96M
1) initial state with a 100M file and a 1K file

2) 4M written, nr_to_write <= 0, so write more

3) 1K written, nr_to_write > 0, no more writes(BUG)

nr_to_write > 0 in (3) fools the upper layer to think that data have all
been written out.  The big dirty file is actually still sitting in
s_more_io.  We cannot simply splice s_more_io back to s_io as soon as s_io
becomes empty, and let the loop in generic_sync_sb_inodes() continue: this
may starve newly expired inodes in s_dirty.  It is also not an option to
draw inodes from both s_more_io and s_dirty, an let the loop go on: this
might lead to live locks, and might also starve other superblocks in sync
time(well kupdate may still starve some superblocks, that's another bug).

We have to return when a full scan of s_io completes.  So nr_to_write > 0
does not necessarily mean that "all data are written".  This patch
introduces a flag writeback_control.more_io to indicate that more io should
be done.  With it the big dirty file no longer has to wait for the next
kupdate invokation 5s later.

In sync_sb_inodes() we only set more_io on super_blocks we actually
visited.  This avoids the interaction between two pdflush deamons.

Also in __sync_single_inode() we don't blindly keep requeuing the io if the
filesystem cannot progress.  Failing to do so may lead to 100% iowait.

Tested-by: Mike Snitzer 
Signed-off-by: Fengguang Wu 
Cc: Michael Rubin 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: remove fastcall from mm/

2008-02-05T17:44:18+00:00

fastcall is always defined to be empty, remove it

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Harvey Harrison 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/page-writeback: highmem_is_dirtyable option

2008-02-05T17:44:18+00:00

Add vm.highmem_is_dirtyable toggle

A 32 bit machine with HIGHMEM64 enabled running DCC has an MMAPed file of
approximately 2Gb size which contains a hash format that is written
randomly by the dbclean process.  On 2.6.16 this process took a few
minutes.  With lowmem only accounting of dirty ratios, this takes about 12
hours of 100% disk IO, all random writes.

Include a toggle in /proc/sys/vm/highmem_is_dirtyable which can be set to 1 to
add the highmem back to the total available memory count.

[akpm@linux-foundation.org: Fix the CONFIG_DETECT_SOFTLOCKUP=y build]
Signed-off-by: Bron Gondwana 
Cc: Ethan Solomita 
Cc: Peter Zijlstra 
Cc: WU Fengguang 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/page-writeback.c: make a function static

2008-02-05T17:44:17+00:00

task_dirty_limit() can become static.

Signed-off-by: Adrian Bunk 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Revert "writeback: introduce writeback_control.more_io to indicate more io"

2008-01-15T05:21:29+00:00

This reverts commit 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b, as
requested by Fengguang Wu.  It's not quite fully baked yet, and while
there are patches around to fix the problems it caused, they should get
more testing.  Says Fengguang: "I'll resend them both for -mm later on,
in a more complete patchset".

See

	http://bugzilla.kernel.org/show_bug.cgi?id=9738

for some of this discussion.

Requested-by: Fengguang Wu 
Cc: Andrew Morton 
Cc: Peter Zijlstra 
Signed-off-by: Linus Torvalds

dirty page balancing: Get rid of broken unmapped_ratio logic

2007-11-16T00:41:52+00:00

This code harks back to the days when we didn't count dirty mapped
pages, which led us to try to balance the number of dirty unmapped pages
by how much unmapped memory there was in the system.

That makes no sense any more, since now the dirty counts include the
mapped pages.  Not to mention that the math doesn't work with HIGHMEM
machines anyway, and causes the unmapped_ratio to potentially turn
negative (which we do catch thanks to clamping it at a minimum value,
but I mention that as an indication of how broken the code is).

The code also was written at a time when the default dirty ratio was
much larger, and the unmapped_ratio logic effectively capped that large
dirty ratio a bit.  Again, we've since lowered the dirty ratio rather
aggressively, further lessening the point of that code.

Acked-by: Peter Zijlstra 
Signed-off-by: Linus Torvalds

mm: speed up writeback ramp-up on clean systems

2007-11-15T02:45:38+00:00

We allow violation of bdi limits if there is a lot of room on the system.
Once we hit half the total limit we start enforcing bdi limits and bdi
ramp-up should happen.  Doing it this way avoids many small writeouts on an
otherwise idle system and should also speed up the ramp-up.

Signed-off-by: Peter Zijlstra 
Reviewed-by: Fengguang Wu 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

spelling fixes: mm/

2007-10-19T23:27:18+00:00

Spelling fixes in mm/.

Signed-off-by: Simon Arlott 
Signed-off-by: Adrian Bunk

writeback: remove unnecessary wait in throttle_vm_writeout()

2007-10-17T15:43:02+00:00

We don't want to introduce pointless delays in throttle_vm_writeout() when
the writeback limits are not yet exceeded, do we?

Cc: Nick Piggin 
Cc: OGAWA Hirofumi 
Cc: Kumar Gala 
Cc: Pete Zaitcev 
Cc: Greg KH 
Reviewed-by: Rik van Riel 
Signed-off-by: Fengguang Wu 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

introduce I_SYNC

2007-10-17T15:43:02+00:00

I_LOCK was used for several unrelated purposes, which caused deadlock
situations in certain filesystems as a side effect.  One of the purposes
now uses the new I_SYNC bit.

Also document the various bits and change their order from historical to
logical.

[bunk@stusta.de: make fs/inode.c:wake_up_inode() static]
Signed-off-by: Joern Engel 
Cc: Dave Kleikamp 
Cc: David Chinner 
Cc: Anton Altaparmakov 
Cc: Al Viro 
Cc: Christoph Hellwig 
Signed-off-by: Adrian Bunk 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds