Hi Alexandre,
If you can run blktrace during your test on one of the OSD data disks
and send me the results I can take a look at them. Also, the rados
bench settings and output would be useful too.
Thanks,
Mark
On 6/6/12 11:05 AM, Alexandre DERUMIER wrote:
Hi, I have rebuild my cluster with ubuntu precise,
-kernel 3.2
-ceph 0.47.2
-libc6 2.15
-3 nodes - 5 osd (xfs) by node and 1 tmpfs with 5 journal file.
I had launch rados bench,
and I see again constant writes to xfs....
Maybe this is related to tmpfs ?
I'll retry with kernel 3.4 from intank tomorrow.
I'll also try with journal on a physical disk with xfs partition.
I'll keep you in touch.
----- Mail original -----
De: "Mark Nelson"<mark.nelson@xxxxxxxxxxx>
À: "Alexandre DERUMIER"<aderumier@xxxxxxxxx>
Cc: "Amon Ott"<a.ott@xxxxxxxxxxxx>, ceph-devel@xxxxxxxxxxxxxxx, "Yann Dupont"<Yann.Dupont@xxxxxxxxxxxxxx>
Envoyé: Lundi 4 Juin 2012 14:59:58
Objet: Re: Infiniband 40GB
On 6/4/12 6:40 AM, Alexandre DERUMIER wrote:
Hi,
I'm currently doing some tests with xfs, debian wheezy with standard libc6 (2.11.3-3) and 3.2 kernel.
I'm doing some iostats(3 nodes with 5 osd), and I see constant writes to disks.(as the datas are flushed each second from journal to disk).
Journal is big enough (20GB tmpfs) to handle 30s of write.
Do you think it's related to the missing syncfs() support ?
-Alexandre
Hi Alexandre,
I've included some seekwatcher results for rados bench tests using 16
concurrent 4MB writes on XFS OSD. One shows ubuntu oneiric and the
other precise (ie no syncfs support vs syncfs support in libc).
Unfortunately the original test was on 0.46 and the second test was on
0.47.2, so multiple things changed between the tests. Both were tested
with kernel 3.4. Interestingly the seeks/second don't seem to drop much
but the overall performance has about doubled. This was using a single
7200rpm disk for the OSD data disk and a seperate 7200rpm disk for the
journal in both cases. I'd definitely try 0.47.2 with a new libc though
and see how that works for you.
ceph 0.46/oneiric:
http://nhm.ceph.com/movies/mailinglist-tests/xfs-osd0-oneiric-3.4.mpg
ceph 0.47.2/precise:
http://nhm.ceph.com/movies/mailinglist-tests/xfs-osd0-precise-3.4.mpg
Mark
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
