linux-stable.git/fs/nilfs2, branch v3.18

nilfs2: improve the performance of fdatasync()

2014-10-14T00:18:20+00:00

Support for fdatasync() has been implemented in NILFS2 for a long time,
but whenever the corresponding inode is dirty the implementation falls
back to a full-flegded sync().  Since every write operation has to
update the modification time of the file, the inode will almost always
be dirty and fdatasync() will fall back to sync() most of the time.  But
this fallback is only necessary for a change of the file size and not
for a change of the various timestamps.

This patch adds a new flag NILFS_I_INODE_SYNC to differentiate between
those two situations.

 * If it is set the file size was changed and a full sync is necessary.
 * If it is not set then only the timestamps were updated and
   fdatasync() can go ahead.

There is already a similar flag I_DIRTY_DATASYNC on the VFS layer with
the exact same semantics.  Unfortunately it cannot be used directly,
because NILFS2 doesn't implement write_inode() and doesn't clear the VFS
flags when inodes are written out.  So the VFS writeback thread can
clear I_DIRTY_DATASYNC at any time without notifying NILFS2.  So
I_DIRTY_DATASYNC has to be mapped onto NILFS_I_INODE_SYNC in
nilfs_update_inode().

Signed-off-by: Andreas Rohner 
Signed-off-by: Ryusuke Konishi 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: add missing blkdev_issue_flush() to nilfs_sync_fs()

2014-10-14T00:18:20+00:00

Under normal circumstances nilfs_sync_fs() writes out the super block,
which causes a flush of the underlying block device.  But this depends
on the THE_NILFS_SB_DIRTY flag, which is only set if the pointer to the
last segment crosses a segment boundary.  So if only a small amount of
data is written before the call to nilfs_sync_fs(), no flush of the
block device occurs.

In the above case an additional call to blkdev_issue_flush() is needed.
To prevent unnecessary overhead, the new flag nilfs->ns_flushed_device
is introduced, which is cleared whenever new logs are written and set
whenever the block device is flushed.  For convenience the function
nilfs_flush_device() is added, which contains the above logic.

Signed-off-by: Andreas Rohner 
Signed-off-by: Ryusuke Konishi 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: fix data loss with mmap()

2014-09-26T15:10:34+00:00

This bug leads to reproducible silent data loss, despite the use of
msync(), sync() and a clean unmount of the file system.  It is easily
reproducible with the following script:

  ----------------[BEGIN SCRIPT]--------------------
  mkfs.nilfs2 -f /dev/sdb
  mount /dev/sdb /mnt

  dd if=/dev/zero bs=1M count=30 of=/mnt/testfile

  umount /mnt
  mount /dev/sdb /mnt
  CHECKSUM_BEFORE="$(md5sum /mnt/testfile)"

  /root/mmaptest/mmaptest /mnt/testfile 30 10 5

  sync
  CHECKSUM_AFTER="$(md5sum /mnt/testfile)"
  umount /mnt
  mount /dev/sdb /mnt
  CHECKSUM_AFTER_REMOUNT="$(md5sum /mnt/testfile)"
  umount /mnt

  echo "BEFORE MMAP:\t$CHECKSUM_BEFORE"
  echo "AFTER MMAP:\t$CHECKSUM_AFTER"
  echo "AFTER REMOUNT:\t$CHECKSUM_AFTER_REMOUNT"
  ----------------[END SCRIPT]--------------------

The mmaptest tool looks something like this (very simplified, with
error checking removed):

  ----------------[BEGIN mmaptest]--------------------
  data = mmap(NULL, file_size - file_offset, PROT_READ | PROT_WRITE,
              MAP_SHARED, fd, file_offset);

  for (i = 0; i < write_count; ++i) {
        memcpy(data + i * 4096, buf, sizeof(buf));
        msync(data, file_size - file_offset, MS_SYNC))
  }
  ----------------[END mmaptest]--------------------

The output of the script looks something like this:

  BEFORE MMAP:    281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile
  AFTER MMAP:     6604a1c31f10780331a6850371b3a313  /mnt/testfile
  AFTER REMOUNT:  281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile

So it is clear, that the changes done using mmap() do not survive a
remount.  This can be reproduced a 100% of the time.  The problem was
introduced in commit 136e8770cd5d ("nilfs2: fix issue of
nilfs_set_page_dirty() for page at EOF boundary").

If the page was read with mpage_readpage() or mpage_readpages() for
example, then it has no buffers attached to it.  In that case
page_has_buffers(page) in nilfs_set_page_dirty() will be false.
Therefore nilfs_set_file_dirty() is never called and the pages are never
collected and never written to disk.

This patch fixes the problem by also calling nilfs_set_file_dirty() if the
page has no buffers attached to it.

[akpm@linux-foundation.org: s/PAGE_SHIFT/PAGE_CACHE_SHIFT/]
Signed-off-by: Andreas Rohner 
Tested-by: Andreas Rohner 
Signed-off-by: Ryusuke Konishi 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2014-08-11T18:44:11+00:00

Pull vfs updates from Al Viro:
 "Stuff in here:

   - acct.c fixes and general rework of mnt_pin mechanism.  That allows
     to go for delayed-mntput stuff, which will permit mntput() on deep
     stack without worrying about stack overflows - fs shutdown will
     happen on shallow stack.  IOW, we can do Eric's umount-on-rmdir
     series without introducing tons of stack overflows on new mntput()
     call chains it introduces.
   - Bruce's d_splice_alias() patches
   - more Miklos' rename() stuff.
   - a couple of regression fixes (stable fodder, in the end of branch)
     and a fix for API idiocy in iov_iter.c.

  There definitely will be another pile, maybe even two.  I'd like to
  get Eric's series in this time, but even if we miss it, it'll go right
  in the beginning of for-next in the next cycle - the tricky part of
  prereqs is in this pile"

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs: (40 commits)
  fix copy_tree() regression
  __generic_file_write_iter(): fix handling of sync error after DIO
  switch iov_iter_get_pages() to passing maximal number of pages
  fs: mark __d_obtain_alias static
  dcache: d_splice_alias should detect loops
  exportfs: update Exporting documentation
  dcache: d_find_alias needn't recheck IS_ROOT && DCACHE_DISCONNECTED
  dcache: remove unused d_find_alias parameter
  dcache: d_obtain_alias callers don't all want DISCONNECTED
  dcache: d_splice_alias should ignore DCACHE_DISCONNECTED
  dcache: d_splice_alias mustn't create directory aliases
  dcache: close d_move race in d_splice_alias
  dcache: move d_splice_alias
  namei: trivial fix to vfs_rename_dir comment
  VFS: allow ->d_manage() to declare -EISDIR in rcu_walk mode.
  cifs: support RENAME_NOREPLACE
  hostfs: support rename flags
  shmem: support RENAME_EXCHANGE
  shmem: support RENAME_NOREPLACE
  btrfs: add RENAME_NOREPLACE
  ...

nilfs2: integrate sysfs support into driver

2014-08-08T22:57:21+00:00

This patch integrates creation of sysfs groups and
attributes into NILFS file system driver.

It was found the issue with nilfs_sysfs_{create/delete}_snapshot_group
functions by Michael L Semon  in the first
version of the patch:

  BUG: sleeping function called from invalid context at kernel/locking/mutex.c:579
  in_atomic(): 1, irqs_disabled(): 0, pid: 32676, name: umount.nilfs2
  2 locks held by umount.nilfs2/32676:
   #0:  (&type->s_umount_key#21){++++..}, at: [<790c18e2>] deactivate_super+0x37/0x58
   #1:  (&(&nilfs->ns_cptree_lock)->rlock){+.+...}, at: [<791bf659>] nilfs_put_root+0x23/0x5a
  Preemption disabled at:[<791bf659>] nilfs_put_root+0x23/0x5a

  CPU: 0 PID: 32676 Comm: umount.nilfs2 Not tainted 3.14.0+ #2
  Hardware name: Dell Computer Corporation Dimension 2350/07W080, BIOS A01 12/17/2002
  Call Trace:
    dump_stack+0x4b/0x75
    __might_sleep+0x111/0x16f
    mutex_lock_nested+0x1e/0x3ad
    kernfs_remove+0x12/0x26
    sysfs_remove_dir+0x3d/0x62
    kobject_del+0x13/0x38
    nilfs_sysfs_delete_snapshot_group+0xb/0xd
    nilfs_put_root+0x2a/0x5a
    nilfs_detach_log_writer+0x1ab/0x2c1
    nilfs_put_super+0x13/0x68
    generic_shutdown_super+0x60/0xd1
    kill_block_super+0x1d/0x60
    deactivate_locked_super+0x22/0x3f
    deactivate_super+0x3e/0x58
    mntput_no_expire+0xe2/0x141
    SyS_oldumount+0x70/0xa5
    syscall_call+0x7/0xb

The reason of the issue was placement of
nilfs_sysfs_{create/delete}_snapshot_group() call under
nilfs->ns_cptree_lock protection.  But this protection is unnecessary and
wrong solution.  The second version of the patch fixes this issue.

[fengguang.wu@intel.com: nilfs_sysfs_create_mounted_snapshots_group can be static]
Reported-by: Michael L. Semon 
Signed-off-by: Vyacheslav Dubeyko 
Cc: Vyacheslav Dubeyko 
Cc: Ryusuke Konishi 
Tested-by: Michael L. Semon 
Signed-off-by: Fengguang Wu 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: add /sys/fs/nilfs2//mounted_snapshots/ group

2014-08-08T22:57:21+00:00

This patch adds creation of  group for every mounted
snapshot in /sys/fs/nilfs2//mounted_snapshots group.

The group contains details about mounted snapshot:
(1) inodes_count - show number of inodes for snapshot.
(2) blocks_count - show number of blocks for snapshot.

Signed-off-by: Vyacheslav Dubeyko 
Cc: Vyacheslav Dubeyko 
Cc: Ryusuke Konishi 
Cc: Michael L. Semon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: add /sys/fs/nilfs2//mounted_snapshots group

2014-08-08T22:57:21+00:00

This patch adds creation of /sys/fs/nilfs2//mounted_snapshots
group.

The mounted_snapshots group contains group for every
mounted snapshot.

Signed-off-by: Vyacheslav Dubeyko 
Cc: Vyacheslav Dubeyko 
Cc: Ryusuke Konishi 
Cc: Michael L. Semon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: add /sys/fs/nilfs2//checkpoints group

2014-08-08T22:57:21+00:00

This patch adds creation of /sys/fs/nilfs2//checkpoints
group.

The checkpoints group contains attributes that describe
details about volume's checkpoints:
(1) checkpoints_number - show number of checkpoints on volume.
(2) snapshots_number - show number of snapshots on volume.
(3) last_seg_checkpoint - show checkpoint number of the latest segment.
(4) next_checkpoint - show next checkpoint number.

Signed-off-by: Vyacheslav Dubeyko 
Cc: Vyacheslav Dubeyko 
Cc: Ryusuke Konishi 
Cc: Michael L. Semon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: add /sys/fs/nilfs2//segments group

2014-08-08T22:57:20+00:00

This patch adds creation of /sys/fs/nilfs2//segments
group.

The segments group contains attributes that describe
details about volume's segments:
(1) segments_number - show number of segments on volume.
(2) blocks_per_segment - show number of blocks in segment.
(3) clean_segments - show count of clean segments.
(4) dirty_segments - show count of dirty segments.

Signed-off-by: Vyacheslav Dubeyko 
Cc: Vyacheslav Dubeyko 
Cc: Ryusuke Konishi 
Cc: Michael L. Semon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

nilfs2: add /sys/fs/nilfs2//segctor group

2014-08-08T22:57:20+00:00

This patch adds creation of /sys/fs/nilfs2//segctor
group.

The segctor group contains attributes that describe
segctor thread activity details:
(1) last_pseg_block - show start block number of the latest segment.
(2) last_seg_sequence - show sequence value of the latest segment.
(3) last_seg_checkpoint - show checkpoint number of the latest segment.
(4) current_seg_sequence - show segment sequence counter.
(5) current_last_full_seg - show index number of the latest full segment.
(6) next_full_seg - show index number of the full segment index
to be used next.
(7) next_pseg_offset - show offset of next partial segment in
the current full segment.
(8) next_checkpoint - show next checkpoint number.
(9) last_seg_write_time - show write time of the last segment
in human-readable format.
(10) last_seg_write_time_secs - show write time of the last segment
in seconds.
(11) last_nongc_write_time - show write time of the last segment
not for cleaner operation in human-readable format.
(12) last_nongc_write_time_secs - show write time of the last segment
not for cleaner operation in seconds.
(13) dirty_data_blocks_count - show number of dirty data blocks.

Signed-off-by: Vyacheslav Dubeyko 
Cc: Vyacheslav Dubeyko 
Cc: Ryusuke Konishi 
Cc: Michael L. Semon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds