linux.git/drivers/md, branch v4.19-rc4

Merge tag 'for-4.19/dm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm

2018-09-14T05:12:55+00:00

Pull device mapper fixes from Mike Snitzer:

 - DM verity fix for crash due to using vmalloc'd buffers with the
   asynchronous crypto hadsh API.

 - Fix to both DM crypt and DM integrity targets to discontinue using
   CRYPTO_TFM_REQ_MAY_SLEEP because its use of GFP_KERNEL can lead to
   deadlock by recursing back into a filesystem.

 - Various DM raid fixes related to reshape and rebuild races.

 - Fix for DM thin-provisioning to avoid data corruption that was a
   side-effect of needing to abort DM thin metadata transaction due to
   running out of metadata space. Fix is to reserve a small amount of
   metadata space so that once it is used the DM thin-pool can finish
   its active transaction before switching to read-only mode.

* tag 'for-4.19/dm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm:
  dm thin metadata: try to avoid ever aborting transactions
  dm raid: bump target version, update comments and documentation
  dm raid: fix RAID leg rebuild errors
  dm raid: fix rebuild of specific devices by updating superblock
  dm raid: fix stripe adding reshape deadlock
  dm raid: fix reshape race on small devices
  dm: disable CRYPTO_TFM_REQ_MAY_SLEEP to fix a GFP_KERNEL recursion deadlock
  dm verity: fix crash on bufio buffer that was allocated with vmalloc

dm thin metadata: try to avoid ever aborting transactions

2018-09-10T21:03:18+00:00

Committing a transaction can consume some metadata of it's own, we now
reserve a small amount of metadata to cover this.  Free metadata
reported by the kernel will not include this reserve.

If any of the reserve has been used after a commit we enter a new
internal state PM_OUT_OF_METADATA_SPACE.  This is reported as
PM_READ_ONLY, so no userland changes are needed.  If the metadata
device is resized the pool will move back to PM_WRITE.

These changes mean we never need to abort and rollback a transaction due
to running out of metadata space.  This is particularly important
because there have been a handful of reports of data corruption against
DM thin-provisioning that can all be attributed to the thin-pool having
ran out of metadata space.

Signed-off-by: Joe Thornber 
Signed-off-by: Mike Snitzer

dm raid: bump target version, update comments and documentation

2018-09-06T21:07:58+00:00

Bump target version to reflect the documented fixes are available.
Also fix some code comments (typos and clarity).

Signed-off-by: Heinz Mauelshagen 
Signed-off-by: Mike Snitzer

dm raid: fix RAID leg rebuild errors

2018-09-06T21:07:56+00:00

On fast devices such as NVMe, a flaw in rs_get_progress() results in
false target status output when userspace lvm2 requests leg rebuilds
(symptom of the failure is device health chars 'aaaaaaaa' instead of
expected 'aAaAAAAA' causing lvm2 to fail).

The correct sync action state definitions already exist in
decipher_sync_action() so fix rs_get_progress() to use it.

Change decipher_sync_action() to return an enum rather than a string for
the sync states and call it from rs_get_progress().  Introduce
sync_str() to translate from enum to the string that is needed by
raid_status().

Signed-off-by: Heinz Mauelshagen 
Signed-off-by: Mike Snitzer

dm raid: fix rebuild of specific devices by updating superblock

2018-09-06T21:07:56+00:00

Update superblock when particular devices are requested via rebuild
(e.g. lvconvert --replace ...) to avoid spurious failure with the "New
device injected into existing raid set without 'delta_disks' or
'rebuild' parameter specified" error message.

Signed-off-by: Heinz Mauelshagen 
Signed-off-by: Mike Snitzer

dm raid: fix stripe adding reshape deadlock

2018-09-06T21:07:50+00:00

When initiating a stripe adding reshape, a deadlock between
md_stop_writes() waiting for the sync thread to stop and the running
sync thread waiting for inactive stripes occurs (this frequently happens
on single-core but rarely on multi-core systems).

Fix this deadlock by setting MD_RECOVERY_WAIT to have the main MD
resynchronization thread worker (md_do_sync()) bail out when initiating
the reshape via constructor arguments.

Signed-off-by: Heinz Mauelshagen 
Signed-off-by: Mike Snitzer

dm raid: fix reshape race on small devices

2018-09-06T18:11:00+00:00

Loading a new mapping table, the dm-raid target's constructor
retrieves the volatile reshaping state from the raid superblocks.

When the new table is activated in a following resume, the actual
reshape position is retrieved.  The reshape driven by the previous
mapping can already have finished on small and/or fast devices thus
updating raid superblocks about the new raid layout.

This causes the actual array state (e.g. stripe size reshape finished)
to be inconsistent with the one in the new mapping, causing hangs with
left behind devices.

This race does not occur with usual raid device sizes but with small
ones (e.g. those created by the lvm2 test suite).

Fix by no longer transferring stale/inconsistent raid_set state during
preresume.

Signed-off-by: Heinz Mauelshagen 
Signed-off-by: Mike Snitzer

dm: disable CRYPTO_TFM_REQ_MAY_SLEEP to fix a GFP_KERNEL recursion deadlock

2018-09-06T17:31:09+00:00

There's a XFS on dm-crypt deadlock, recursing back to itself due to the
crypto subsystems use of GFP_KERNEL, reported here:
https://bugzilla.kernel.org/show_bug.cgi?id=200835

* dm-crypt calls crypt_convert in xts mode
* init_crypt from xts.c calls kmalloc(GFP_KERNEL)
* kmalloc(GFP_KERNEL) recurses into the XFS filesystem, the filesystem
	tries to submit some bios and wait for them, causing a deadlock

Fix this by updating both the DM crypt and integrity targets to no
longer use the CRYPTO_TFM_REQ_MAY_SLEEP flag, which will change the
crypto allocations from GFP_KERNEL to GFP_ATOMIC, therefore they can't
recurse into a filesystem.  A GFP_ATOMIC allocation can fail, but
init_crypt() in xts.c handles the allocation failure gracefully - it
will fall back to preallocated buffer if the allocation fails.

The crypto API maintainer says that the crypto API only needs to
allocate memory when dealing with unaligned buffers and therefore
turning CRYPTO_TFM_REQ_MAY_SLEEP off is safe (see this discussion:
https://www.redhat.com/archives/dm-devel/2018-August/msg00195.html )

Cc: stable@vger.kernel.org
Signed-off-by: Mikulas Patocka 
Signed-off-by: Mike Snitzer

dm verity: fix crash on bufio buffer that was allocated with vmalloc

2018-09-04T15:25:25+00:00

Since commit d1ac3ff008fb ("dm verity: switch to using asynchronous hash
crypto API") dm-verity uses asynchronous crypto calls for verification,
so that it can use hardware with asynchronous processing of crypto
operations.

These asynchronous calls don't support vmalloc memory, but the buffer data
can be allocated with vmalloc if dm-bufio is short of memory and uses a
reserved buffer that was preallocated in dm_bufio_client_create().

Fix verity_hash_update() so that it deals with vmalloc'd memory
correctly.

Reported-by: "Xiao, Jin" 
Signed-off-by: Mikulas Patocka 
Fixes: d1ac3ff008fb ("dm verity: switch to using asynchronous hash crypto API")
Cc: stable@vger.kernel.org # 4.12+
Signed-off-by: Mike Snitzer

md-cluster: release RESYNC lock after the last resync message

2018-09-01T00:38:10+00:00

All the RESYNC messages are sent with resync lock held, the only
exception is resync_finish which releases resync_lockres before
send the last resync message, this should be changed as well.
Otherwise, we can see deadlock issue as follows:

clustermd2-gqjiang2:~ # cat /proc/mdstat
Personalities : [raid10] [raid1]
md0 : active raid1 sdg[0] sdf[1]
      134144 blocks super 1.2 [2/2] [UU]
      [===================>.]  resync = 99.6% (134144/134144) finish=0.0min speed=26K/sec
      bitmap: 1/1 pages [4KB], 65536KB chunk

unused devices: 
clustermd2-gqjiang2:~ # ps aux|grep md|grep D
root     20497  0.0  0.0      0     0 ?        D    16:00   0:00 [md0_raid1]
clustermd2-gqjiang2:~ # cat /proc/20497/stack
[] dlm_lock_sync+0x8e/0xc0 [md_cluster]
[] __sendmsg+0x98/0x130 [md_cluster]
[] sendmsg+0x20/0x30 [md_cluster]
[] resync_info_update+0xb5/0xc0 [md_cluster]
[] md_reap_sync_thread+0x134/0x170 [md_mod]
[] md_check_recovery+0x28c/0x510 [md_mod]
[] raid1d+0x42/0x800 [raid1]
[] md_thread+0x121/0x150 [md_mod]
[] kthread+0xff/0x140
[] ret_from_fork+0x35/0x40
[] 0xffffffffffffffff

clustermd-gqjiang1:~ # ps aux|grep md|grep D
root     20531  0.0  0.0      0     0 ?        D    16:00   0:00 [md0_raid1]
root     20537  0.0  0.0      0     0 ?        D    16:00   0:00 [md0_cluster_rec]
root     20676  0.0  0.0      0     0 ?        D    16:01   0:00 [md0_resync]
clustermd-gqjiang1:~ # cat /proc/mdstat
Personalities : [raid10] [raid1]
md0 : active raid1 sdf[1] sdg[0]
      134144 blocks super 1.2 [2/2] [UU]
      [===================>.]  resync = 97.3% (131072/134144) finish=8076.8min speed=0K/sec
      bitmap: 1/1 pages [4KB], 65536KB chunk

unused devices: 
clustermd-gqjiang1:~ # cat /proc/20531/stack
[] metadata_update_start+0xcd/0xd0 [md_cluster]
[] md_update_sb.part.61+0x97/0x820 [md_mod]
[] md_check_recovery+0x29b/0x510 [md_mod]
[] raid1d+0x42/0x800 [raid1]
[] md_thread+0x121/0x150 [md_mod]
[] kthread+0xff/0x140
[] ret_from_fork+0x35/0x40
[] 0xffffffffffffffff
clustermd-gqjiang1:~ # cat /proc/20537/stack
[] freeze_array+0xf2/0x140 [raid1]
[] recv_daemon+0x41e/0x580 [md_cluster]
[] md_thread+0x121/0x150 [md_mod]
[] kthread+0xff/0x140
[] ret_from_fork+0x35/0x40
[] 0xffffffffffffffff
clustermd-gqjiang1:~ # cat /proc/20676/stack
[] dlm_lock_sync+0x8e/0xc0 [md_cluster]
[] lock_token+0x2f/0xa0 [md_cluster]
[] lock_comm+0x32/0x90 [md_cluster]
[] sendmsg+0x15/0x30 [md_cluster]
[] resync_info_update+0x8a/0xc0 [md_cluster]
[] raid1_sync_request+0xa9a/0xb10 [raid1]
[] md_do_sync+0xbaa/0xf90 [md_mod]
[] md_thread+0x121/0x150 [md_mod]
[] kthread+0xff/0x140
[] ret_from_fork+0x35/0x40
[] 0xffffffffffffffff

Reviewed-by: NeilBrown 
Signed-off-by: Guoqing Jiang 
Signed-off-by: Shaohua Li