Interesting! What kernel versions were running on the 13.10 and centos 6.4 clusters? -Sam On Fri, Oct 4, 2013 at 6:33 PM, huangyellowhuang <huangyellowhuang@xxxxxxx> wrote: > Hi,all > We test the ceph version 0.69 (6ca6f2f9f754031f4acdb971b71c92c9762e18c3) on > Ubuntu server 13.10 and centos 6.4 final > Our cluster configuration: > 3 host machine, each runs 3 OSDs(use XFS as backend fs),MON and MDS runs on > one of the three host, > We have one KClient on Ubuntu server 13.10 > > The cluster runs on Ubuntu works fine and a few ‘slow requests’ msgs, about > 100MB/s write speed. > But the cluster runs on centos is very bad, avg 30MB/s write speed, many osd > requests slow: > 2013-10-05 08:35:09.931145 mon.0 [INF] pgmap v928: 192 pgs: 192 active+clean > ; 50873 MB data, 101716 MB used, 13857 GB / 13956 GB avail; 115 MB/s wr, 28 > op/s > 2013-10-05 08:35:12.087614 mon.0 [INF] pgmap v929: 192 pgs: 192 active+clean > ; 50901 MB data, 101780 MB used, 13857 GB / 13956 GB avail; 32593 KB/s wr, 8 > op/s > 2013-10-05 08:35:03.963979 osd.0 [WRN] 37 slow requests, 1 included below; o > ldest blocked for > 798.235962 secs > 2013-10-05 08:35:03.963984 osd.0 [WRN] slow request 240.831078 seconds old, > received at 2013-10-05 08:31:03.132836: osd_op(mds.0.1:375 200.00000000 [wri > tefull 0~84] 1.844f3494 e47) v4 currently no flag points reached > 2013-10-05 08:35:08.965134 osd.0 [WRN] 37 slow requests, 1 included below; o > ldest blocked for > 803.237127 secs > 2013-10-05 08:35:08.965139 osd.0 [WRN] slow request 480.312618 seconds old, > received at 2013-10-05 08:27:08.652461: osd_op(mds.0.1:307 200.00000000 [wri > tefull 0~84] 1.844f3494 e47) v4 currently no flag points reached > 2013-10-05 08:35:10.965619 osd.0 [WRN] 37 slow requests, 1 included below; o > ldest blocked for > 805.237600 secs > 2013-10-05 08:35:10.965624 osd.0 [WRN] slow request 120.946652 seconds old, > received at 2013-10-05 08:33:10.018900: osd_op(mds.0.1:404 200.00000000 [wri > tefull 0~84] 1.844f3494 e47) v4 currently no flag points reached > 2013-10-05 08:35:11.965986 osd.0 [WRN] 37 slow requests, 1 included below; o > ldest blocked for > 806.237800 secs > 2013-10-05 08:35:11.965992 osd.0 [WRN] slow request 60.474314 seconds old, r > eceived at 2013-10-05 08:34:11.491438: osd_op(mds.0.1:430 200.00000000 [writ > efull 0~84] 1.844f3494 e47) v4 currently no flag points reached > > And we need to build a cluster have 4 hosts, each has 18 OSDs and 1 kclient, > every kclient server as samba server that serves 4 samba clients. > 1) Which linux distrubtion should we used? Centos or Ubuntu? > 2) What results in ceph performances so large on different distribution? > 3) It seems the bottleneck is underlayer fs can not handle requests fast as > ceph expect, bc the ‘slow requests’ shows if a request does not handle after > 30s. > > Thanks! > > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
