Hi epk,
Which ioengine are you using? if it's librbd, you might try playing
with librbd readahead as well:
# don't disable readahead after a certain number of bytes
rbd readahead disable after bytes = 0
# Set the librbd readahead to whatever:
rbd readahead max bytes = 4194304
If it's with kvm+guests, you may be better off playing with the guest
readahead but you can try the librbd readahead if you want.
Another thing to watch out for is fragmentation. btrfs OSDs for example
will fragment terribly after small random writes to RBD images due to
how copy-on-write works. That can cause havoc with RBD sequential reads
in general.
Mark
On 07/26/2016 06:38 PM, EP Komarla wrote:
Hi,
I am showing below fio results for Sequential Read on my Ceph cluster.
I am trying to understand this pattern:
- why there is a dip in the performance for block sizes 32k-256k?
- is this an expected performance graph?
- have you seen this kind of pattern before
My cluster details:
Ceph: Hammer release
Cluster: 6 nodes (dual Intel sockets) each with 20 OSDs and 4 SSDs (5
OSD journals on one SSD)
Client network: 10Gbps
Cluster network: 10Gbps
FIO test:
- 2 Client servers
- Sequential Read
- Run time of 600 seconds
- Filesize = 1TB
- 10 rbd images per client
- Queue depth=16
Any ideas on tuning this cluster? Where should I look first?
Thanks,
- epk
Legal Disclaimer:
The information contained in this message may be privileged and
confidential. It is intended to be read only by the individual or entity
to whom it is addressed or by their designee. If the reader of this
message is not the intended recipient, you are on notice that any
distribution of this message, in any form, is strictly prohibited. If
you have received this message in error, please immediately notify the
sender and delete or destroy any copy of this message!
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
