On Sat, Oct 19, 2013 at 7:14 AM, Yan, Zheng <ukernel@xxxxxxxxx> wrote: > On Sat, Oct 19, 2013 at 8:58 PM, hjwsm1989-gmail <hjwsm1989@xxxxxxxxx> wrote: >> Hi, >> I'm testing ceph with samba. >> I have 20 OSD nodes on 4 hosts >> dy01:1MON, 5 OSDs, 1 samba server >> dy02: 1MDS, 5OSDs, 1 samba server >> dy03: 5 OSDs, 1 samba server >> dy04:5 OSDs, 1 samba server >> every samba server exports the ceph kclient mount point. >> >> When I set 3 samba clients connect to a samba server(dy01), and each write a >> single 18GB file, gets 110MB/s speed each. >> But when I use 4 clients, the cluster will report 'slow request' msg, and >> this time I list files in samba client dir, it gets stuck. >> And the test tool reports frame losing. >> For the list operation related to MDS, I think the MDS op messages should >> have higher priority than clients op. >> Have I understood correctly? > > Hi, > > I think you are right, the MDS needs higher priority. could you modify > the mds code > and re-test. > > In src/ceph_mds.cc, add > "messenger->set_default_send_priority(CEPH_MSG_PRIO_HIGH);" > after > "messenger->set_cluster_protocol(CEPH_MDS_PROTOCOL);" > > Regards > Yan, Zheng If this actually turns out to be useful, we'll want to briefly go through the code base and adjust the other levels — a quick grep indicates the MDS sets some traffic to LOW that should correspondingly be bumped up to NORMAL; the OSDs are currently using HIGH for acks and commits, but we probably want those messages to continue having a higher priority than any clients (MDS included) are setting messages at. (I'm not sure it actually will; I suspect what's happening here is just that 4 clients are passing some limit that 3 aren't.) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
