Hi,all We test the ceph version 0.69 (6ca6f2f9f754031f4acdb971b71c92c9762e18c3) on Ubuntu server 13.10 and centos 6.4 final Our cluster configuration: 3 host machine, each runs 3 OSDs(use XFS as backend fs),MON and MDS runs on one of the three host, We have one KClient on Ubuntu server 13.10 The cluster runs on Ubuntu works fine and a few ?slow requests? msgs, about 100MB/s write speed. But the cluster runs on centos is very bad, avg 30MB/s write speed, many osd requests slow: 2013-10-05 08:35:09.931145 mon.0 [INF] pgmap v928: 192 pgs: 192 active+clean ; 50873 MB data, 101716 MB used, 13857 GB / 13956 GB avail; 115 MB/s wr, 28 op/s 2013-10-05 08:35:12.087614 mon.0 [INF] pgmap v929: 192 pgs: 192 active+clean ; 50901 MB data, 101780 MB used, 13857 GB / 13956 GB avail; 32593 KB/s wr, 8 op/s 2013-10-05 08:35:03.963979 osd.0 [WRN] 37 slow requests, 1 included below; o ldest blocked for > 798.235962 secs 2013-10-05 08:35:03.963984 osd.0 [WRN] slow request 240.831078 seconds old, received at 2013-10-05 08:31:03.132836: osd_op(mds.0.1:375 200.00000000 [wri tefull 0~84] 1.844f3494 e47) v4 currently no flag points reached 2013-10-05 08:35:08.965134 osd.0 [WRN] 37 slow requests, 1 included below; o ldest blocked for > 803.237127 secs 2013-10-05 08:35:08.965139 osd.0 [WRN] slow request 480.312618 seconds old, received at 2013-10-05 08:27:08.652461: osd_op(mds.0.1:307 200.00000000 [wri tefull 0~84] 1.844f3494 e47) v4 currently no flag points reached 2013-10-05 08:35:10.965619 osd.0 [WRN] 37 slow requests, 1 included below; o ldest blocked for > 805.237600 secs 2013-10-05 08:35:10.965624 osd.0 [WRN] slow request 120.946652 seconds old, received at 2013-10-05 08:33:10.018900: osd_op(mds.0.1:404 200.00000000 [wri tefull 0~84] 1.844f3494 e47) v4 currently no flag points reached 2013-10-05 08:35:11.965986 osd.0 [WRN] 37 slow requests, 1 included below; o ldest blocked for > 806.237800 secs 2013-10-05 08:35:11.965992 osd.0 [WRN] slow request 60.474314 seconds old, r eceived at 2013-10-05 08:34:11.491438: osd_op(mds.0.1:430 200.00000000 [writ efull 0~84] 1.844f3494 e47) v4 currently no flag points reached And we need to build a cluster have 4 hosts, each has 18 OSDs and 1 kclient, every kclient server as samba server that serves 4 samba clients. 1) Which linux distrubtion should we used? Centos or Ubuntu? 2) What results in ceph performances so large on different distribution? 3) It seems the bottleneck is underlayer fs can not handle requests fast as ceph expect, bc the ?slow requests? shows if a request does not handle after 30s. Thanks! -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
