Hi,Shinobu
I found the logrotate script at /etc/logrotate.d/ceph. In this script osd mon mds will be reload when rotate done.
The logrotate and the ceph-fuse crash at same time mainly.
So i think the problem with this matter.
How do you think?
The code snippet in /etc/logrotate.d/ceph:
*******************************************
for daemon in osd mon mds ; do
find -L /var/lib/ceph/$daemon/ -mindepth 1 -maxdepth 1 -regextype posix-egrep -regex '.*/[A-Za-z0-9]+-[A-Za-z0-9._-]+' -printf '%P\n' \
| while read f; do
if [ -e "/var/lib/ceph/$daemon/$f/done" -o -e "/var/lib/ceph/$daemon/$f/ready" ] && [ -e "/var/lib/ceph/$daemon/$f/upstart" ] && [ ! -e "/var/lib/ceph/$daemon/$f/sysvinit" ]; then
cluster="${f%%-*}"
id="${f#*-}"
initctl reload ceph-$daemon cluster="$cluster" id="$id" 2>/dev/null || :
fi
done
done
*******************************************
Thank you!
2015-09-14 8:50 GMT+08:00 谷枫 <feicheche@xxxxxxxxx>:
I attach filesystem to local use this command: ceph-fuse -k /etc/ceph.new/ceph.client.admin.keyring -m 10.3.1.11,10.3.1.12,10.3.1.13:6789 /data.The key is right.I attach the client1.tar in last mail. Please check it . Thank you !2015-09-13 15:12 GMT+08:00 Shinobu Kinjo <skinjo@xxxxxxxxxx>:How do you attach filesystem to local file?
Make sure, keyring is located at:
/etc/ceph.new/ceph.client.admin.keyring
And your cluster, public networks are fine.
If you face same problem again, check:
uptime
And how about this:
> tar cvf <host name>.tar \
> /sys/class/net/<interface name>/statistics/*
When did you face this issue?
>From the beginning or...?
Shinobu
----- Original Message -----
From: "谷枫" <feicheche@xxxxxxxxx>
To: "Shinobu Kinjo" <skinjo@xxxxxxxxxx>
Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx>
Sent: Sunday, September 13, 2015 12:06:25 PM
Subject: Re: ceph-fuse auto down
All clients use same ceph-fuse version. All of them by this problem
troubled. Just crash time different.
2015-09-13 10:39 GMT+08:00 Shinobu Kinjo <skinjo@xxxxxxxxxx>:
> So you are using same version on other clients?
> But only one client has problem?
>
> Can you provide:
>
> /sys/class/net/<interface name>/statistics/*
>
> just do:
>
> tar cvf <host name>.tar \
> /sys/class/net/<interface name>/statistics/*
>
> Can you hold when same issue happen next?
> No reboot is necessary.
>
> But if you have to reboot, of course you can.
>
> Shinobu
>
> ----- Original Message -----
> From: "谷枫" <feicheche@xxxxxxxxx>
> To: "Shinobu Kinjo" <skinjo@xxxxxxxxxx>
> Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx>
> Sent: Sunday, September 13, 2015 11:30:57 AM
> Subject: Re: ceph-fuse auto down
>
> Yes, when some ceph-fuse crash , the mount driver has gone, and can't
> remount . Reboot the server is the only way I can do.
> But other client with ceph-fuse mount on them working well. Can writing /
> reading data on them.
>
> ceph-fuse --version
> ceph version 0.94.3 (95cefea9fd9ab740263bf8bb4796fd864d9afe2b)
>
> ceph -s
> cluster 0fddc8e0-9e64-4049-902a-2f0f6d531630
> health HEALTH_OK
> monmap e1: 3 mons at {ceph01=
> 10.3.1.11:6789/0,ceph02=10.3.1.12:6789/0,ceph03=10.3.1.13:6789/0}
> election epoch 8, quorum 0,1,2 ceph01,ceph02,ceph03
> mdsmap e29: 1/1/1 up {0=ceph04=up:active}, 1 up:standby
> osdmap e26: 4 osds: 4 up, 4 in
> pgmap v94931: 320 pgs, 3 pools, 90235 MB data, 241 kobjects
> 289 GB used, 1709 GB / 1999 GB avail
> 320 active+clean
> client io 1023 kB/s rd, 1210 kB/s wr, 72 op/s
>
> 2015-09-13 10:23 GMT+08:00 Shinobu Kinjo <skinjo@xxxxxxxxxx>:
>
> > Can you give us package version of ceph-fuse?
> >
> > > Multi ceph-fuse crash just now today.
> >
> > Did you just mount filesystem or was there any
> > activity on filesystem?
> >
> > e.g: writing / reading data
> >
> > Can you give us output of on cluster side:
> >
> > ceph -s
> >
> > Shinobu
> >
> > ----- Original Message -----
> > From: "谷枫" <feicheche@xxxxxxxxx>
> > To: "Shinobu Kinjo" <skinjo@xxxxxxxxxx>
> > Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx>
> > Sent: Sunday, September 13, 2015 10:51:35 AM
> > Subject: Re: ceph-fuse auto down
> >
> > sorry Shinobu,
> > I don't understand what's the means what you pasted.
> > Multi ceph-fuse crash just now today.
> > The ceph-fuse completely unusable for me now.
> > Maybe i must change the kernal mount with it.
> >
> > 2015-09-12 20:08 GMT+08:00 Shinobu Kinjo <skinjo@xxxxxxxxxx>:
> >
> > > In _usr_bin_ceph-fuse.0.crash.client2.tar
> > >
> > > What I'm seeing now is:
> > >
> > > 3 Date: Sat Sep 12 06:37:47 2015
> > > ...
> > > 6 ExecutableTimestamp: 1440614242
> > > ...
> > > 7 ProcCmdline: ceph-fuse -k /etc/ceph.new/ceph.client.admin.keyring
> -m
> > > 10.3.1.11,10.3.1.12,10.3.1.13 /grdata
> > > ...
> > > 30 7f32de7fe000-7f32deffe000 rw-p 00000000 00:00 0
> > > [stack:17270]
> > > ...
> > > 250 7f341021d000-7f3410295000 r-xp 00000000 fd:01 267219
> > > /usr/lib/x86_64-linux-gnu/nss/libfreebl3.so
> > > ...
> > > 255 7f341049b000-7f341054f000 r-xp 00000000 fd:01 266443
> > > /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
> > > ...
> > > 260 7f3410754000-7f3410794000 r-xp 00000000 fd:01 267222
> > > /usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so
> > > ...
> > > 266 7f3411197000-7f341119a000 r-xp 00000000 fd:01 264953
> > > /usr/lib/x86_64-linux-gnu/libplds4.so
> > > ...
> > > 271 7f341139f000-7f341159e000 ---p 00004000 fd:01 264955
> > > /usr/lib/x86_64-linux-gnu/libplc4.so
> > > ...
> > > 274 7f34115a0000-7f34115c5000 r-xp 00000000 fd:01 267214
> > > /usr/lib/x86_64-linux-gnu/libnssutil3.so
> > > ...
> > > 278 7f34117cb000-7f34117ce000 r-xp 00000000 fd:01 1189512
> > > /lib/x86_64-linux-gnu/libdl-2.19.so
> > > ...
> > > 287 7f3411d94000-7f3411daa000 r-xp 00000000 fd:01 1179825
> > > /lib/x86_64-linux-gnu/libgcc_s.so.1
> > > ...
> > > 294 7f34122b0000-7f3412396000 r-xp 00000000 fd:01 266069
> > > /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
> > > ...
> > > 458 State: D (disk sleep)
> > > ...
> > > 359 VmPeak: 5250648 kB
> > > 360 VmSize: 4955592 kB
> > > ...
> > >
> > > What were you trying to do?
> > >
> > > Shinobu
> > >
> > >
> >
>
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
