-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 17/12/14 05:26, VELARTIS Philipp Dürhammer wrote: > Hello, > > > > Read speed inside our vms (most of them windows) is only ¼ of the > write speed. > > Write speed is about 450MB/s – 500mb/s and > > Read is only about 100/MB/s > > > > Our network is 10Gbit for OSDs and 10GB for MONS. We have 3 Servers > with 15 osds each We saw similar things, until we started playing around with read ahead parameters inside the VMs. Our environment is almost 100% Ubuntu, but the same basic principles should hold. I'm pretty sure that there have been previous posts to the lists about this, but the value(s) we tweaked are: /sys/block/$device/queue/read_ahead_kb It defaults to 128, but we had pretty drastic increases to read speed all of the way up to around 8192 (8 MB) with no obvious regressions to random read speed. I'm not sure what the equivalent option is in Windows, sorry. Unfortunately this is a per VM (per disk per VM, even) setting, but it can be automated to some degree. We have a udev rule snippet pushed out to each VM in order to set the value(s). It may also be worth investigating read ahead options on the storage nodes themselves, both at the OS and disk controller levels. This isn't something we've yet been able to test, however. - -- David Clarke Systems Architect Catalyst IT -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJUkJuQAAoJEPH5xSy8rJPMGnYP/iwfzBJMHsJ9oYetY+s0U6id IeA6iAK3DjTSJmJE0reO7olUBZ6Kq1T/u5yW/wA/qvdPw/UTb0G9NsY80DXsc2nc 74EUM4o6TiqzauUqxWxiLyh/vaBg5WtFmmJlCwGxclWrCuazeTYmTTUmN76WKJ4G nrR2m+HlZMz0jaOcwoTB5dQALVvJ4g4zRg4Cz4M8sLJX13pEZDRC1FGfiAA0OvxZ kUFZBtvFhQxAVisTBc7gqiz9PwAk1vn8WlfaUD2h1DNNoQBE4wqLW8QgWbGrnrWl mIrtKH5c8ykDIR2aaSgcWiYw3QrOgU/2PyRKw/yrMuhany7VXPA3oJMnr7HZNAUn Z0t3bPBjrsdayERwJJz4PscZuIVvAoJs8He+ssg7BT5/R9jRJawL23FDxcbtUmp6 1io/y2GSL7SQhR9vlRl6/AFGJc24nU52wRzkUHCcCMVz3AgBv7bfyBE+7DzaJTNs /0MvIea8u8owDnWP/YTfYLNBOwi6WqZG4m8IqrygvlCTO5Ijtl1CUhvxFKaVzdGi FAoG+k3x5HXdw8T1sgMeiF6kLx0ifw6pe5J4bxkNMkoNvqS9f3vPARe+FGgtl/QM inli/qGipqYvqfgzONNILNkpp+k67dAjCfNT22jvAUtnFDEOmYMSCIDcwoi0rVAP Yqn0EwypITm1jEDRrqof =wu0K -----END PGP SIGNATURE----- _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |