On 07/09/2013 03:20 AM, Sebastien Han wrote:
Hi all,
While running some benchmarks with the internal rados benchmarker I
noticed something really strange. First of all, this is the line I used
to run it:
$ sudo rados -p 07:59:54_performance bench 300 write -b 4194304 -t 1
--no-cleanup
So I want to test an IO with a concurrency of 1. I had a look at the
code and also strace the process and I noticed that the IOs are send one
by one sequentially. Thus it does what I expect from it.
However while monitoring the disks usage on all my OSDs, I found out
that they were all loaded (writing, both journals and filestore) which
is kind of weird since all the IOs are send one by one. I was expecting
that only one OSDs at a time will be writing.
Obviously there is no replication going on since I changed the rep size
to 1.
$ ceph osd dump |grep "07:59:54_performance"
pool 323 '07:59:54_performance' rep size 1 min_size 1 crush_ruleset 0
object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 1306 owner 0
Thanks in advance guy.
Hi Sebastien,
Do your SAS controllers have writeback cache perchance? If so, you may
be seeing the OSDs immediately acknowledge journal writes before the
data actually makes it to disk allowing the client to send the next
operation before the first actually is truly committed. Even without WB
cache, you'll probably see a lot of concurrent journal writes and OSD
writes since the OSD writes are just happening in the background via
buffered IO. There's also drive cache helping out here unless you've
explicitly disabled it. I suspect some combination of above is what you
are seeing.
Mark
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
