Thanks a lot... "ceph daemon mds.<id> session ls" was a good starting point. What is happening: I am in an OpenStack environment and start a VM. Afterwards, I mount a Manila share via ceph-fuse. I get a new client session in state "open" on the MDS node. Everything looks fine so far. The problem arises when you add or remove a floating IP from the VM. Now you get a "stale" connection that is removed after a default timeout of 300 seconds. Unfortunately, the connection is never re-established and accessing the mount causes everything to hang indefinitely. Is there a way to handle this use case transparently for the user in ceph-fuse? Is it possible to re-establish a connection to the MDS? Regards, Jens Gesendet: Mittwoch, 30. November 2016 um 12:38 Uhr Von: "John Spray" <jspray@xxxxxxxxxx> An: "Jens Offenbach" <wolle5050@xxxxxx> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx> Betreff: Re: Mount of CephFS hangs On Wed, Nov 30, 2016 at 6:39 AM, Jens Offenbach <wolle5050@xxxxxx> wrote: > Hi, > I am confronted with a persistent problem during mounting of the CephFS. I am using Ubuntu 16.04 and solely ceph-fuse. The CephFS gets mounted by muliple machines and very ofen (not always, but in most cases) the mount process hangs and does not continue. "df -h" also hangs and nothing happens. Everything seems to be up an running. On the MDS side, I have the following in the log: > > 2016-11-30 07:09:57.208736 7f680bc65700 0 -- 10.30.200.141:6800/1358 >> 10.30.216.130:0/3257360291 pipe(0x5572ceaa0800 sd=22 :6800 s=2 pgs=2 cs=1 l=0 c=0x5572c4843500).fault with nothing to send, going to standby > 2016-11-30 07:10:27.833523 7f6812394700 0 log_channel(cluster) log [WRN] : 1 slow requests, 1 included below; oldest blocked for > 30.532515 secs > 2016-11-30 07:10:27.833631 7f6812394700 0 log_channel(cluster) log [WRN] : slow request 30.532515 seconds old, received at 2016-11-30 07:09:57.300940: client_request(client.588036:6 setattr uid=3001 gid=3001 #10000000851 2016-11-30 07:09:57.302543) currently failed to xlock, waiting > 2016-11-30 07:14:52.841056 7f6812394700 0 log_channel(cluster) log [INF] : closing stale session client.641683 10.30.216.130:0/3257360291 after 300.119493 > 2016-11-30 07:17:02.844691 7f6812394700 0 log_channel(cluster) log [INF] : closing stale session client.588036 10.30.216.130:0/1984817088 after 304.588537 > 2016-11-30 07:17:02.859557 7f6809740700 0 -- 10.30.200.141:6800/1358 >> 10.30.216.130:0/1984817088 pipe(0x5572ce891400 sd=23 :6800 s=0 pgs=0 cs=0 l=0 c=0x5572c4843080).accept we reset (peer sent cseq 2), sending RESETSESSION > 2016-11-30 07:17:18.344852 7f6809740700 0 -- 10.30.200.141:6800/1358 >> 10.30.216.130:0/1984817088 pipe(0x5572ce891400 sd=23 :6800 s=2 pgs=4 cs=1 l=0 c=0x5572c4843080).reader missed message? skipped from seq 0 to 114931623 Which of those clients is the one mounting? You can resolve the "client.588036" etc to hostnames by looking at the "ceph daemon mds.<id> session ls" output on the MDS node. Also, what version of Ceph is install on the servers and clients? John > It seems to be a network issue, but the network is up and running without any failuries. > > What can I do to solve this issue and to make the mount process more reliable? > > Thanks a lot, > Jens > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[https://deref-gmx.net/mail/client/ZV9wEgi2Jig/dereferrer/?redirectUrl=http%3A%2F%2Flists.ceph.com%2Flistinfo.cgi%2Fceph-users-ceph.com] _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |