Which SSD are you using? Dsync flag will dramatically slow down most SSDs. You¹ve got to be very careful about the SSD you pick. Warren Wang On 12/14/15, 5:49 AM, "Nikola Ciprich" <nikola.ciprich@xxxxxxxxxxx> wrote: >Hello, > >i'm doing some measuring on test (3 nodes) cluster and see strange >performance >drop for sync writes.. > >I'm using SSD for both journalling and OSD. It should be suitable for >journal, giving about 16.1KIOPS (67MB/s) for sync IO. > >(measured using fio --filename=/dev/xxx --direct=1 --sync=1 --rw=write >--bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based >--group_reporting --name=journal-test) > >On top of this cluster, I have running KVM guest (using qemu librbd >backend). >Overall performance seems to be quite good, but the problem is when I try >to measure sync IO performance inside the guest.. I'm getting only about >600IOPS, >which I think is quite poor. > >The problem is, I don't see any bottlenect, OSD daemons don't seem to be >hanging on >IO, neither hogging CPU, qemu process is also not somehow too much >loaded.. > >I'm using hammer 0.94.5 on top of centos 6 (4.1 kernel), all debugging >disabled, > >my question is, what results I can expect for synchronous writes? I >understand >there will always be some performance drop, but 600IOPS on top of storage >which >can give as much as 16K IOPS seems to little.. > >Has anyone done similar measuring? > >thanks a lot in advance! > >BR > >nik > > >-- >------------------------------------- >Ing. Nikola CIPRICH >LinuxBox.cz, s.r.o. >28.rijna 168, 709 00 Ostrava > >tel.: +420 591 166 214 >fax: +420 596 621 273 >mobil: +420 777 093 799 >www.linuxbox.cz > >mobil servis: +420 737 238 656 >email servis: servis@xxxxxxxxxxx >------------------------------------- This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. *** Walmart Confidential *** _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |