Dear all
--
Bui Minh Tien
I use ceph 0.56.4 - CentOS 6.3 up kernel 3.8.8-1.el6.elrepo.x86_64.
I use multiple versions of the kernel, but I still encountering status:
libceph: osd3 172.30.33.2:6810 socket closed (con state OPEN)
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: osd3 172.30.33.2:6810 socket error on write
libceph: mon0 172.30.33.2:6789 socket closed (con state OPEN)
libceph: mon0 172.30.33.2:6789 session lost, hunting for new mon
libceph: mon0 172.30.33.2:6789 socket error on write
libceph: mds0 172.30.33.5:6800 socket closed (con state OPEN)
libceph: wrong peer, want 172.30.33.5:6800/6609, got 172.30.33.5:6800/9999
libceph: mds0 172.30.33.5:6800 socket error on read
libceph: wrong peer, want 172.30.33.5:6800/6609, got 172.30.33.5:6800/9999
libceph: mds0 172.30.33.5:6800 socket error on read
libceph: wrong peer, want 172.30.33.5:6800/6609, got 172.30.33.5:6800/9999
libceph: mds0 172.30.33.5:6800 socket error on read
libceph: wrong peer, want 172.30.33.5:6800/6609, got 172.30.33.5:6800/9999
libceph: mds0 172.30.33.5:6800 socket error on read
This has led to:
2013-04-25 07:42:50.309221 osd.3 [WRN] 16 slow requests, 5 included below; oldest blocked for > 112.518922 secs
2013-04-25 07:42:50.309226 osd.3 [WRN] slow request 81.114924 seconds old, received at 2013-04-25 07:41:29.194206: osd_op(client.16426.1:1342 10000253435.00000515 [write 0~4194304] 0.a3a0bc5c snapc 1=[]) currently started
2013-04-25 07:42:50.309229 osd.3 [WRN] slow request 81.107046 seconds old, received at 2013-04-25 07:41:29.202084: osd_op(client.16426.1:1349 10000253435.0000051c [write 0~4194304 [1@-1]] 0.79e1bf44 snapc 1=[]) currently no flag points reached
2013-04-25 07:42:50.309234 osd.3 [WRN] slow request 81.099973 seconds old, received at 2013-04-25 07:41:29.209157: osd_op(client.16426.1:1358 10000253435.00000525 [write 0~4194304] 0.d758b955 snapc 1=[]) currently started
2013-04-25 07:42:50.309237 osd.3 [WRN] slow request 81.093918 seconds old, received at 2013-04-25 07:41:29.215212: osd_op(client.16426.1:1364 10000253435.0000052b [write 0~4194304] 0.66b34c17 snapc 1=[]) currently started
2013-04-25 07:42:50.309241 osd.3 [WRN] slow request 81.088057 seconds old, received at 2013-04-25 07:41:29.221073: osd_op(client.16426.1:1365 10000253435.0000052c [write 0~4194304] 0.a5442446 snapc 1=[]) currently started
2013-04-25 07:42:56.310042 osd.3 [WRN] 17 slow requests, 1 included below; oldest blocked for > 118.519799 secs
2013-04-25 07:42:56.310047 osd.3 [WRN] slow request 80.232497 seconds old, received at 2013-04-25 07:41:36.077510: osd_sub_op(mds.0.92:213 1.77 ea298b77/200.0000348a/head//1 [] v 1912'15556 snapset=0=[]:[] snapc=0=[]) v7 currently started
2013-04-25 07:42:49.523067 osd.6 [WRN] 2 slow requests, 1 included below; oldest blocked for > 115.651487 secs
2013-04-25 07:42:49.523072 osd.6 [WRN] slow request 80.650518 seconds old, received at 2013-04-25 07:41:28.872524: osd_op(mds.0.92:213 200.0000348a [write 3153504~1418] 1.ea298b77) v4 currently waiting for subops from [3]
2013-04-25 07:43:16.313000 osd.3 [WRN] 18 slow requests, 1 included below; oldest blocked for > 138.522756 secs
2013-04-25 07:43:16.313005 osd.3 [WRN] slow request 80.234601 seconds old, received at 2013-04-25 07:41:56.078363: osd_sub_op(mds.0.92:216 1.77 ea298b77/200.0000348a/head//1 [] v 1912'15557 snapset=0=[]:[] snapc=0=[]) v7 currently started
2013-04-25 07:43:09.526520 osd.6 [WRN] 3 slow requests, 1 included below; oldest blocked for > 135.654919 secs
2013-04-25 07:43:09.526525 osd.6 [WRN] slow request 80.653303 seconds old, received at 2013-04-25 07:41:48.873171: osd_op(mds.0.92:216 200.0000348a [write 3154922~1976] 1.ea298b77) v4 currently waiting for subops from [3]
2013-04-25 07:43:38.316232 osd.3 [WRN] 18 slow requests, 5 included below; oldest blocked for > 160.525944 secs
2013-04-25 07:43:38.316237 osd.3 [WRN] slow request 160.525944 seconds old, received at 2013-04-25 07:40:57.790208: osd_op(client.16426.1:1281 10000253435.000004d9 [write 0~4194304] 0.fc1b1d44 snapc 1=[]) currently started
2013-04-25 07:43:38.316241 osd.3 [WRN] slow request 160.517433 seconds old, received at 2013-04-25 07:40:57.798719: osd_op(client.16426.1:1282 10000253435.000004da [write 0~4194304] 0.b7baa9ea snapc 1=[]) currently commit sent
2013-04-25 07:43:38.316245 osd.3 [WRN] slow request 160.510521 seconds old, received at 2013-04-25 07:40:57.805631: osd_op(client.16426.1:1287 10000253435.000004df [write 0~4194304] 0.fe03215c snapc 1=[]) currently started
How to fix it?
Thanks
Bui Minh Tien
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
