Thank you for log archives. I went to dentist -; Please do not forget CCing ceph-users from the next because there is a bunch of really **awesome** guys; Can you re-attach log files again so that they see? Shinobu ----- Original Message ----- From: "谷枫" <feicheche@xxxxxxxxx> To: "Shinobu Kinjo" <skinjo@xxxxxxxxxx> Sent: Saturday, September 12, 2015 5:32:31 PM Subject: Re: ceph-fuse auto down The mds log shows that the client session timeout then mds close the socket right? I think this happend behind the ceph-fuse crash. So the root cause is the ceph-fuse crash . 2015-09-12 15:09 GMT+08:00 谷枫 <feicheche@xxxxxxxxx>: > Hi,Shinobu > I notice some useful message in the mds-log. > > 2015-09-12 14:29 GMT+08:00 谷枫 <feicheche@xxxxxxxxx>: > >> >> _usr_bin_ceph-fuse.0.crash.client1.tar.gz >> <https://drive.google.com/file/d/0Bw059OTYnFqAbDA2R0RlS3ZHNlU/view?usp=drive_web> >> >> _usr_bin_ceph-fuse.0.crash.client2.tar.gz >> <https://drive.google.com/file/d/0Bw059OTYnFqAMXVCWTV6UXVnRjg/view?usp=drive_web> >> Hi,Shinobu >> I check the /var/log/dmesg carefully again, but not find the useful >> message about ceph-fuse crash. >> So i attach two _usr_bin_ceph-fuse.0.crash files on two clients to you. >> Please let me know if you want more about other log. Thank you very much! >> >> 2015-09-12 13:01 GMT+08:00 Shinobu Kinjo <skinjo@xxxxxxxxxx>: >> >>> Ah, you are using ubuntu, sorry for that. >>> How about: >>> >>> /var/log/dmesg >>> >>> I believe you can attach file not paste. >>> Pasting a bunch of logs would not be good for me -; >>> >>> And when did you notice that cephfs was hung? >>> >>> Shinobu >>> >>> ----- Original Message ----- >>> From: "谷枫" <feicheche@xxxxxxxxx> >>> To: "Shinobu Kinjo" <skinjo@xxxxxxxxxx> >>> Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> >>> Sent: Saturday, September 12, 2015 1:50:05 PM >>> Subject: Re: ceph-fuse auto down >>> >>> hi,Shinobu >>> There is no /var/log/messages on my system but i saw the /var/log/syslog >>> and no useful messages be found. >>> I discover the /var/crash/_usr_bin_ceph-fuse.0.crash with grep the "fuse" >>> on the system. >>> Below is the message in it : >>> ProcStatus: >>> Name: ceph-fuse >>> State: D (disk sleep) >>> Tgid: 2903 >>> Ngid: 0 >>> Pid: 2903 >>> PPid: 1 >>> TracerPid: 0 >>> Uid: 0 0 0 0 >>> Gid: 0 0 0 0 >>> FDSize: 64 >>> Groups: 0 >>> VmPeak: 7428552 kB >>> VmSize: 6838728 kB >>> VmLck: 0 kB >>> VmPin: 0 kB >>> VmHWM: 1175864 kB >>> VmRSS: 343116 kB >>> VmData: 6786232 kB >>> VmStk: 136 kB >>> VmExe: 5628 kB >>> VmLib: 7456 kB >>> VmPTE: 3404 kB >>> VmSwap: 0 kB >>> Threads: 37 >>> SigQ: 1/64103 >>> SigPnd: 0000000000000000 >>> ShdPnd: 0000000000000000 >>> SigBlk: 0000000000001000 >>> SigIgn: 0000000000001000 >>> SigCgt: 00000001c18040eb >>> CapInh: 0000000000000000 >>> CapPrm: 0000003fffffffff >>> CapEff: 0000003fffffffff >>> CapBnd: 0000003fffffffff >>> Seccomp: 0 >>> Cpus_allowed: ffff >>> Cpus_allowed_list: 0-15 >>> Mems_allowed: 00000000,00000001 >>> Mems_allowed_list: 0 >>> voluntary_ctxt_switches: 25 >>> nonvoluntary_ctxt_switches: 2 >>> Signal: 11 >>> Uname: Linux 3.19.0-28-generic x86_64 >>> UserGroups: >>> CoreDump: base64 >>> >>> Is this useful infomations? >>> >>> >>> 2015-09-12 12:33 GMT+08:00 Shinobu Kinjo <skinjo@xxxxxxxxxx>: >>> >>> > There should be some complains in /var/log/messages. >>> > Can you attach? >>> > >>> > Shinobu >>> > >>> > ----- Original Message ----- >>> > From: "谷枫" <feicheche@xxxxxxxxx> >>> > To: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> >>> > Sent: Saturday, September 12, 2015 1:30:49 PM >>> > Subject: ceph-fuse auto down >>> > >>> > Hi,all >>> > My cephfs cluster deploy on three nodes with Ceph Hammer 0.94.3 on >>> Ubuntu >>> > 14.04 the kernal version is 3.19.0. >>> > >>> > I mount the cephfs with ceph-fuse on 9 clients,but some of them >>> (ceph-fuse >>> > process) auto down sometimes and i can't find the reason seems like >>> there >>> > is no other logs can be found except this file >>> > /var/log/ceph/ceph-client.admin.log that without useful messages for >>> me. >>> > >>> > When the ceph-fuse down . The mount driver is gone. >>> > How can i find the reason of this problem. Can some guys give me good >>> > ideas? >>> > >>> > Regards >>> > >>> > >>> > _______________________________________________ >>> > ceph-users mailing list >>> > ceph-users@xxxxxxxxxxxxxx >>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> > >>> >> >> > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |