> how much time to flush from journal to disks ? >>I don't know how to measure this. Do an iostat, you must see timelapse of write inactivity on disk (datas are written to journal) , then after a timelapse of write activity on disk.(data flushed from journal to disk) >>As ceph starts to write to journal and >>disk in parallel this is strange, from doc: http://ceph.com/wiki/OSD_journal the journal mode should be write-ahead with xfs. So write to journal first then flush to disk each 30sec. maybe your tmpfs is too small, and flushs occurs at 50% of free space on journal. If by exemple, your flush occurs each 1 or 2seconds, this can cause very slow write. >>and tmpfs isn't even shown in iostat. indeed, iostat doesn't work with tmpfs... ----- Mail original ----- De: "Stefan Priebe - Profihost AG" <s.priebe@xxxxxxxxxxxx> À: "Alexandre DERUMIER" <aderumier@xxxxxxxxx> Cc: ceph-devel@xxxxxxxxxxxxxxx, "Mark Nelson" <mark.nelson@xxxxxxxxxxx> Envoyé: Dimanche 27 Mai 2012 11:11:13 Objet: Re: poor OSD performance using kernel 3.4 Can really nobody help? Am 25.05.2012 17:47, schrieb Alexandre DERUMIER: > Hi Stephan, > Do you have same performance with read ? Read is fine for both versions see here: 3.0.30 Write: Total time run: 30.872357 Total writes made: 1095 Write size: 4194304 Bandwidth (MB/sec): 141.874 Average Latency: 0.450187 Max latency: 2.00672 Min latency: 0.091783 Read: Total time run: 22.907021 Total reads made: 1095 Read size: 4194304 Bandwidth (MB/sec): 191.208 Average Latency: 0.333954 Max latency: 1.71987 Min latency: 0.041373 3.4.0 Write: Total time run: 124.573247 Total writes made: 647 Write size: 4194304 Bandwidth (MB/sec): 20.775 Average Latency: 3.08058 Max latency: 65.2522 Min latency: 0.089587 Read: Total time run: 13.191562 Total reads made: 647 Read size: 4194304 Bandwidth (MB/sec): 196.186 Average Latency: 0.322895 Max latency: 1.22392 Min latency: 0.043784 > Did you have done some iostats ? Yes - I/O is heavily jumping between 0 and 60MB/s but of the time it's 0 or around 10MB/s. > how much time to flush from journal to disks ? I don't know how to measure this. As ceph starts to write to journal and disk in parallel and tmpfs isn't even shown in iostat. Greets, Stefan -- -- Alexandre D erumier Ingénieur Système Fixe : 03 20 68 88 90 Fax : 03 20 68 90 81 45 Bvd du Général Leclerc 59100 Roubaix - France 12 rue Marivaux 75002 Paris - France -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
