linux-stable.git/fs/btrfs/relocation.c, branch linux-4.3.y

Btrfs: fix regression running delayed references when using qgroups

2015-12-15T05:41:09+00:00

commit b06c4bf5c874a57254b197f53ddf588e7a24a2bf upstream.

In the kernel 4.2 merge window we had a big changes to the implementation
of delayed references and qgroups which made the no_quota field of delayed
references not used anymore. More specifically the no_quota field is not
used anymore as of:

  commit 0ed4792af0e8 ("btrfs: qgroup: Switch to new extent-oriented qgroup mechanism.")

Leaving the no_quota field actually prevents delayed references from
getting merged, which in turn cause the following BUG_ON(), at
fs/btrfs/extent-tree.c, to be hit when qgroups are enabled:

  static int run_delayed_tree_ref(...)
  {
     (...)
     BUG_ON(node->ref_mod != 1);
     (...)
  }

This happens on a scenario like the following:

  1) Ref1 bytenr X, action = BTRFS_ADD_DELAYED_REF, no_quota = 1, added.

  2) Ref2 bytenr X, action = BTRFS_DROP_DELAYED_REF, no_quota = 0, added.
     It's not merged with Ref1 because Ref1->no_quota != Ref2->no_quota.

  3) Ref3 bytenr X, action = BTRFS_ADD_DELAYED_REF, no_quota = 1, added.
     It's not merged with the reference at the tail of the list of refs
     for bytenr X because the reference at the tail, Ref2 is incompatible
     due to Ref2->no_quota != Ref3->no_quota.

  4) Ref4 bytenr X, action = BTRFS_DROP_DELAYED_REF, no_quota = 0, added.
     It's not merged with the reference at the tail of the list of refs
     for bytenr X because the reference at the tail, Ref3 is incompatible
     due to Ref3->no_quota != Ref4->no_quota.

  5) We run delayed references, trigger merging of delayed references,
     through __btrfs_run_delayed_refs() -> btrfs_merge_delayed_refs().

  6) Ref1 and Ref3 are merged as Ref1->no_quota = Ref3->no_quota and
     all other conditions are satisfied too. So Ref1 gets a ref_mod
     value of 2.

  7) Ref2 and Ref4 are merged as Ref2->no_quota = Ref4->no_quota and
     all other conditions are satisfied too. So Ref2 gets a ref_mod
     value of 2.

  8) Ref1 and Ref2 aren't merged, because they have different values
     for their no_quota field.

  9) Delayed reference Ref1 is picked for running (select_delayed_ref()
     always prefers references with an action == BTRFS_ADD_DELAYED_REF).
     So run_delayed_tree_ref() is called for Ref1 which triggers the
     BUG_ON because Ref1->red_mod != 1 (equals 2).

So fix this by removing the no_quota field, as it's not used anymore as
of commit 0ed4792af0e8 ("btrfs: qgroup: Switch to new extent-oriented
qgroup mechanism.").

The use of no_quota was also buggy in at least two places:

1) At delayed-refs.c:btrfs_add_delayed_tree_ref() - we were setting
   no_quota to 0 instead of 1 when the following condition was true:
   is_fstree(ref_root) || !fs_info->quota_enabled

2) At extent-tree.c:__btrfs_inc_extent_ref() - we were attempting to
   reset a node's no_quota when the condition "!is_fstree(root_objectid)
   || !root->fs_info->quota_enabled" was true but we did it only in
   an unused local stack variable, that is, we never reset the no_quota
   value in the node itself.

This fixes the remainder of problems several people have been having when
running delayed references, mostly while a balance is running in parallel,
on a 4.2+ kernel.

Very special thanks to Stéphane Lesimple for helping debugging this issue
and testing this fix on his multi terabyte filesystem (which took more
than one day to balance alone, plus fsck, etc).

Also, this fixes deadlock issue when using the clone ioctl with qgroups
enabled, as reported by Elias Probst in the mailing list. The deadlock
happens because after calling btrfs_insert_empty_item we have our path
holding a write lock on a leaf of the fs/subvol tree and then before
releasing the path we called check_ref() which did backref walking, when
qgroups are enabled, and tried to read lock the same leaf. The trace for
this case is the following:

  INFO: task systemd-nspawn:6095 blocked for more than 120 seconds.
  (...)
  Call Trace:
    [] schedule+0x74/0x83
    [] btrfs_tree_read_lock+0xc0/0xea
    [] ? wait_woken+0x74/0x74
    [] btrfs_search_old_slot+0x51a/0x810
    [] btrfs_next_old_leaf+0xdf/0x3ce
    [] ? ulist_add_merge+0x1b/0x127
    [] __resolve_indirect_refs+0x62a/0x667
    [] ? btrfs_clear_lock_blocking_rw+0x78/0xbe
    [] find_parent_nodes+0xaf3/0xfc6
    [] __btrfs_find_all_roots+0x92/0xf0
    [] btrfs_find_all_roots+0x45/0x65
    [] ? btrfs_get_tree_mod_seq+0x2b/0x88
    [] check_ref+0x64/0xc4
    [] btrfs_clone+0x66e/0xb5d
    [] btrfs_ioctl_clone+0x48f/0x5bb
    [] ? native_sched_clock+0x28/0x77
    [] btrfs_ioctl+0xabc/0x25cb
  (...)

The problem goes away by eleminating check_ref(), which no longer is
needed as its purpose was to get a value for the no_quota field of
a delayed reference (this patch removes the no_quota field as mentioned
earlier).

Reported-by: Stéphane Lesimple 
Tested-by: Stéphane Lesimple 
Reported-by: Elias Probst 
Reported-by: Peter Becker 
Reported-by: Malte Schröder 
Reported-by: Derek Dongray 
Reported-by: Erkki Seppala 
Cc: stable@vger.kernel.org  # 4.2+
Signed-off-by: Filipe Manana 
Reviewed-by: Qu Wenruo

btrfs: Remove unnecessary variants in relocation.c

2015-08-09T14:07:14+00:00

These arguments are not used in functions, remove them for cleanup
and make kernel stack happy.

Signed-off-by: Zhao Lei 
Signed-off-by: Chris Mason

btrfs: Cleanup: Remove objectid's init-value in create_reloc_inode()

2015-08-09T14:07:13+00:00

objectid's init-value is not used in any case, remove it.

Signed-off-by: Zhao Lei 
Reviewed-by: David Sterba 
Signed-off-by: Chris Mason

btrfs: Error handle for get_ref_objectid_v0() in relocate_block_group()

2015-08-09T14:07:13+00:00

We need error checking code for get_ref_objectid_v0() in
relocate_block_group(), to avoid unpredictable result, especially
for accessing uninitialized value(when function failed) after
this line.

Signed-off-by: Zhao Lei 
Reviewed-by: David Sterba 
Signed-off-by: Chris Mason

btrfs: Use ref_cnt for set_block_group_ro()

2015-08-09T14:07:12+00:00

More than one code call set_block_group_ro() and restore rw in fail.

Old code use bool bit to save blockgroup's ro state, it can not
support parallel case(it is confirmd exist in my debug log).

This patch use ref count to store ro state, and rename
set_block_group_ro/set_block_group_rw
to
inc_block_group_ro/dec_block_group_ro.

Signed-off-by: Zhao Lei 
Signed-off-by: Chris Mason

Btrfs: fix wrong check for btrfs_force_chunk_alloc()

2015-07-02T00:17:22+00:00

btrfs_force_chunk_alloc() return 1 for allocation chunk successfully.
This problem exists since commit c87f08ca4.

With this patch, we might fix some enospc problems for balances.

Signed-off-by: Wang Shilong 
Reviewed-by: Filipe Manana 
Tested-by: Filipe Manana 
Signed-off-by: Chris Mason

Btrfs: fix up read_tree_block to return proper error

2015-06-03T11:03:08+00:00

The return value of read_tree_block() can confuse callers as it always
returns NULL for either -ENOMEM or -EIO, so it's likely that callers
parse it to a wrong error, for instance, in btrfs_read_tree_root().

This fixes the above issue.

Signed-off-by: Liu Bo 
Reviewed-by: David Sterba 
Signed-off-by: Chris Mason

btrfs: qgroup: do a reservation in a higher level.

2015-04-13T14:52:50+00:00

There are two problems in qgroup:

a). The PAGE_CACHE is 4K, even when we are writing a data of 1K,
qgroup will reserve a 4K size. It will cause the last 3K in a qgroup
is not available to user.

b). When user is writing a inline data, qgroup will not reserve it,
it means this is a window we can exceed the limit of a qgroup.

The main idea of this patch is reserving the data size of write_bytes
rather than the reserve_bytes. It means qgroup will not care about
the data size btrfs will reserve for user, but only care about the
data size user is going to write. Then reserve it when user want to
write and release it in transaction committed.

In this way, qgroup can be released from the complex procedure in
btrfs and only do the reserve when user want to write and account
when the data is written in commit_transaction().

Signed-off-by: Dongsheng Yang 
Signed-off-by: Chris Mason

Btrfs: allow block group cache writeout outside critical section in commit

2015-04-10T21:07:22+00:00

We loop through all of the dirty block groups during commit and write
the free space cache.  In order to make sure the cache is currect, we do
this while no other writers are allowed in the commit.

If a large number of block groups are dirty, this can introduce long
stalls during the final stages of the commit, which can block new procs
trying to change the filesystem.

This commit changes the block group cache writeout to take appropriate
locks and allow it to run earlier in the commit.  We'll still have to
redo some of the block groups, but it means we can get most of the work
out of the way without blocking the entire FS.

Signed-off-by: Chris Mason

btrfs: sink blocksize parameter to tree_block_processed

2014-12-12T17:07:22+00:00

Signed-off-by: David Sterba