On Wednesday, 20 November 2013, Dimitri Maziuk wrote:
On 11/18/2013 01:19 AM, YIP Wai Peng wrote:
> Hi Dima,
>
> Benchmark FYI.
>
> $ /usr/sbin/bonnie++ -s 0 -n 5:1m:4k
> Version 1.97 ------Sequential Create------ --------Random Create--------
> altair -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> 5:1048576:4096 18 1 609 11 604 9 18 1 436 10 686 9
> Latency 1187ms 70907us 261ms 2352ms 205ms 111ms
Is this on an FS on RBD on the NFS server, mounted and exported?
Yes it is. And benchmarked on the client nfs machine.
I get
> Version 1.96 ------Sequential Create------ --------Random Create--------
> nautilus -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> 5:1048576:4096 19 1 5422 99 123 2 41 3 4525 99 127 2
> Latency 86381us 746us 44125us 303ms 746us 60720us
on an ext4 filesystem mounted on top of DRBD device backed by an LSI
raid (10, battery-backed) on the main node and an mdadm raid-0 on the
passive node. This is on the server with he FS exported and mounted (but
not getting much use -- I ran it after hours).
Hm, so maybe this nfsceph is not _that_ bad after all! :) Your read clearly wins, so I'm guessing the drdb write is the slow one. Which drdb mode are you using?
On a client & during
working hours I get
> Version 1.96 ------Sequential Create------ --------Random Create--------
> stingray -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> 5:1048576:4096 3 0 1952 29 1385 13 2 0 1553 18 575 5
> Latency 16383ms 19662us 153ms 8482ms 1935us 4467ms
--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
