> You may want to try increasing your read_ahead_kb on the OSD data disks and > see if that helps read speeds. Jumping into this thread late, so I'm not sure if this was covered, but: Remember that readahead on the OSDs will only help up to the size of the object (4MB). To get good read performance in general what is really needed is for the librbd user to do readahead so that the next object(s) are being fetched before they are needed. I don't think this happens with 'dd' (opening a block device as a file does not trigger the kernel VM readahead code, IIRC). Unless Sylvian implemented this in his tool explicitly, it won't happen there either. sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html