On Mon, Feb 6, 2012 at 10:20 AM, Jim Schutt <jaschut@xxxxxxxxxx> wrote: > On 02/06/2012 10:22 AM, Yehuda Sadeh Weinraub wrote: >> >> On Mon, Feb 6, 2012 at 8:20 AM, Jim Schutt<jaschut@xxxxxxxxxx> wrote: > > >>> >>> The above suggests to me that the slowdown is a result >>> of requests not getting submitted at the same rate as >>> when things are running well. >>> >> >> Yeah, it really looks like that. My suggestions wouldn't help there. >> >> I do see that when things go well the number of writes per device is >> capped at ~200 writes per second and the throughput per device is >> ~100MB/sec. Is 100MB/sec the expected device throughput? > > > Pretty much, at least for the outer tracks on a drive. I've seen > ~108 MB/s with dd to a block device. Also, I've got 8 drives per > SAS adapter with 6 Gb/s links, so it seems unlikely to me that my > disk subsystem is any sort of significant bottleneck. Well, you might try changing your throttling settings on the OSDs. ms_dispatch_throttle_bytes defaults to 100<<20 (100MB) and is used for throttling dispatch; osd_max_client_bytes defaults to 500<<20 (500MB) and is used to limit the amount of client data in memory (ie; messages are included in this throttler for their entire lifetime, not just while waiting for dispatch). -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
