Hi Udo,
thanks, just to make sure, further increased the readahead:
$ sudo blockdev --getra /dev/rbd0
1048576
$ cat /sys/block/rbd0/queue/read_ahead_kb
524288
No difference here. First one is sectors (512 bytes), second one KB.
The second read (after drop cache) is somewhat faster (10%-20%) but not
much.
I also found this info
http://tracker.ceph.com/issues/9192
Maybe Ilya can help us, he knows probably best how this can be improved.
Thanks and cheers,
Mike
On 4/21/16 4:32 PM, Udo Lembke wrote:
Hi Mike,
Am 21.04.2016 um 09:07 schrieb Mike Miller:
Hi Nick and Udo,
thanks, very helpful, I tweaked some of the config parameters along
the line Udo suggests, but still only some 80 MB/s or so.
this mean you have reached factor 3 (this are round about the value I
see with single thread on RBD too). Better than nothing.
Kernel 4.3.4 running on the client machine and comfortable readahead
configured
$ sudo blockdev --getra /dev/rbd0
262144
Still not more than about 80-90 MB/s.
they are two possibilities for read-ahead.
Take a look here (and change with echo)
cat /sys/block/rbd0/queue/read_ahead_kb
Perhaps there are slightly differences?
For writing the parallelization is amazing and I see very impressive
speeds, but why is reading performance so much behind? Why is it not
parallelized the same way writing is? Is this something coming up in
the jewel release? Or is it planned further down the road?
If you read an big file and clear your cache ("echo 3 >
/proc/sys/vm/drop_caches") on the client, is the second read very fast?
I assume yes.
In this case the readed data is in the cache on the osd-nodes... so
tuning must be there (and I'm very interesting in improvements).
Udo
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
