linux-stable.git/net/ceph/osd_client.c, branch linux-3.4.y

libceph: resend all writes after the osdmap loses the full flag

2014-03-31T04:40:30+00:00

commit 9a1ea2dbff11547a8e664f143c1ffefc586a577a upstream.

With the current full handling, there is a race between osds and
clients getting the first map marked full. If the osd wins, it will
return -ENOSPC to any writes, but the client may already have writes
in flight. This results in the client getting the error and
propagating it up the stack. For rbd, the block layer turns this into
EIO, which can cause corruption in filesystems above it.

To avoid this race, osds are being changed to drop writes that came
from clients with an osdmap older than the last osdmap marked full.
In order for this to work, clients must resend all writes after they
encounter a full -> not full transition in the osdmap. osds will wait
for an updated map instead of processing a request from a client with
a newer map, so resent writes will not be dropped by the osd unless
there is another not full -> full transition.

This approach requires both osds and clients to be fixed to avoid the
race. Old clients talking to osds with this fix may hang instead of
returning EIO and potentially corrupting an fs. New clients talking to
old osds have the same behavior as before if they encounter this race.

Fixes: http://tracker.ceph.com/issues/6938

Reviewed-by: Sage Weil 
Signed-off-by: Josh Durgin 
Signed-off-by: Greg Kroah-Hartman

libceph: unregister request in __map_request failed and nofail == false

2014-03-11T23:10:05+00:00

commit 73d9f7eef3d98c3920e144797cc1894c6b005a1e upstream.

For nofail == false request, if __map_request failed, the caller does
cleanup work, like releasing the relative pages.  It doesn't make any sense
to retry this request.

Signed-off-by: Jianpeng Ma 
Reviewed-by: Sage Weil 
[bwh: Backported to 3.2: adjust indentation]
Signed-off-by: Ben Hutchings 
Cc: Rui Xiang 
Signed-off-by: Greg Kroah-Hartman

libceph: wrap auth ops in wrapper functions

2013-06-20T18:58:47+00:00

commit 27859f9773e4a0b2042435b13400ee2c891a61f4 upstream.

Use wrapper functions that check whether the auth op exists so that callers
do not need a bunch of conditional checks.  Simplifies the external
interface.

Signed-off-by: Sage Weil 
Reviewed-by: Alex Elder 
Signed-off-by: Greg Kroah-Hartman

libceph: add update_authorizer auth method

2013-06-20T18:58:46+00:00

commit 0bed9b5c523d577378b6f83eab5835fe30c27208 upstream.

Currently the messenger calls out to a get_authorizer con op, which will
create a new authorizer if it doesn't yet have one.  In the meantime, when
we rotate our service keys, the authorizer doesn't get updated.  Eventually
it will be rejected by the server on a new connection attempt and get
invalidated, and we will then rebuild a new authorizer, but this is not
ideal.

Instead, if we do have an authorizer, call a new update_authorizer op that
will verify that the current authorizer is using the latest secret.  If it
is not, we will build a new one that does.  This avoids the transient
failure.

This fixes one of the sorry sequence of events for bug

	http://tracker.ceph.com/issues/4282

Signed-off-by: Sage Weil 
Reviewed-by: Alex Elder 
Signed-off-by: Greg Kroah-Hartman

libceph: must hold mutex for reset_changed_osds()

2013-06-20T18:58:43+00:00

commit 14d2f38df67fadee34625fcbd282ee22514c4846 upstream.

An osd client has a red-black tree describing its osds, and
occasionally we would get crashes due to one of these trees tree
becoming corrupt somehow.

The problem turned out to be that reset_changed_osds() was being
called without protection of the osd client request mutex.  That
function would call __reset_osd() for any osd that had changed, and
__reset_osd() would call __remove_osd() for any osd with no
outstanding requests, and finally __remove_osd() would remove the
corresponding entry from the red-black tree.  Thus, the tree was
getting modified without having any lock protection, and was
vulnerable to problems due to concurrent updates.

This appears to be the only osd tree updating path that has this
problem.  It can be fairly easily fixed by moving the call up
a few lines, to just before the request mutex gets dropped
in kick_requests().

This resolves:
    http://tracker.ceph.com/issues/5043

Signed-off-by: Alex Elder 
Reviewed-by: Sage Weil 
Signed-off-by: Greg Kroah-Hartman

rbd: remove linger unconditionally

2013-01-17T16:51:20+00:00

In __unregister_linger_request(), the request is being removed
from the osd client's req_linger list only when the request
has a non-null osd pointer.  It should be done whether or not
the request currently has an osd.

This is most likely a non-issue because I believe the request
will always have an osd when this function is called.

Signed-off-by: Alex Elder 
Reviewed-by: Sage Weil 
(cherry picked from commit 61c74035626beb25a39b0273ccf7d75510bc36a1)
Signed-off-by: Greg Kroah-Hartman

ceph: don't reference req after put

2013-01-17T16:51:20+00:00

In __unregister_request(), there is a call to list_del_init()
referencing a request that was the subject of a call to
ceph_osdc_put_request() on the previous line.  This is not
safe, because the request structure could have been freed
by the time we reach the list_del_init().

Fix this by reversing the order of these lines.

Signed-off-by: Alex Elder 
Reviewed-off-by: Sage Weil 
(cherry picked from commit 7d5f24812bd182a2471cb69c1c2baf0648332e1f)
Signed-off-by: Greg Kroah-Hartman

libceph: remove 'osdtimeout' option

2013-01-17T16:51:20+00:00

This would reset a connection with any OSD that had an outstanding
request that was taking more than N seconds.  The idea was that if the
OSD was buggy, the client could compensate by resending the request.

In reality, this only served to hide server bugs, and we haven't
actually seen such a bug in quite a while.  Moreover, the userspace
client code never did this.

More importantly, often the request is taking a long time because the
OSD is trying to recover, or overloaded, and killing the connection
and retrying would only make the situation worse by giving the OSD
more work to do.

Signed-off-by: Sage Weil 
Reviewed-by: Alex Elder 
(cherry picked from commit 83aff95eb9d60aff5497e9f44a2ae906b86d8e88)
Signed-off-by: Greg Kroah-Hartman

libceph: avoid using freed osd in __kick_osd_requests()

2013-01-17T16:51:20+00:00

If an osd has no requests and no linger requests, __reset_osd()
will just remove it with a call to __remove_osd().  That drops
a reference to the osd, and therefore the osd may have been free
by the time __reset_osd() returns.  That function offers no
indication this may have occurred, and as a result the osd will
continue to be used even when it's no longer valid.

Change__reset_osd() so it returns an error (ENODEV) when it
deletes the osd being reset.  And change __kick_osd_requests() so it
returns immediately (before referencing osd again) if __reset_osd()
returns *any* error.

Signed-off-by: Alex Elder 
Reviewed-by: Sage Weil 
(cherry picked from commit 685a7555ca69030739ddb57a47f0ea8ea80196a4)
Signed-off-by: Greg Kroah-Hartman

libceph: always reset osds when kicking

2013-01-17T16:51:19+00:00

When ceph_osdc_handle_map() is called to process a new osd map,
kick_requests() is called to ensure all affected requests are
updated if necessary to reflect changes in the osd map.  This
happens in two cases:  whenever an incremental map update is
processed; and when a full map update (or the last one if there is
more than one) gets processed.

In the former case, the kick_requests() call is followed immediately
by a call to reset_changed_osds() to ensure any connections to osds
affected by the map change are reset.  But for full map updates
this isn't done.

Both cases should be doing this osd reset.

Rather than duplicating the reset_changed_osds() call, move it into
the end of kick_requests().

Signed-off-by: Alex Elder 
Reviewed-by: Sage Weil 
(cherry picked from commit e6d50f67a6b1a6252a616e6e629473b5c4277218)
Signed-off-by: Greg Kroah-Hartman