Hi Stefan, Do you see a difference if you force filestore journal writeahead for btrfs instead parrallel ? filestore journal writeahead = 1 filestore journal parallel = 0 ----- Mail original ----- De: "Stefan Priebe" <s.priebe@xxxxxxxxxxxx> À: "Mark Nelson" <mnelson@xxxxxxxxxx>, "Sage Weil" <sage@xxxxxxxxxxxx> Cc: "ceph-devel" <ceph-devel@xxxxxxxxxxxxxxx> Envoyé: Lundi 5 Janvier 2015 21:33:22 Objet: Re: 10 times higher disk load with btrfs Am 05.01.2015 um 21:29 schrieb Mark Nelson: > > > On 01/05/2015 02:20 PM, Stefan Priebe wrote: >> Hi Sage, >> >> Am 05.01.2015 um 20:25 schrieb Sage Weil: >>> On Mon, 5 Jan 2015, Stefan Priebe wrote: >>>> >>>> Am 05.01.2015 um 19:36 schrieb Stefan Priebe: >>>>> Hi devs, >>>>> >>>>> while btrfs is now declared as stable ;-) i wanted to retest btrfs on >>>>> our production cluster on 2 out of 54 osds. So if they crash it >>>>> doesn't >>>>> hurt. >>>>> >>>>> While if those OSDs run XFS have spikes of 20MB/s every 4-7s. The same >>>>> OSDs after formatting them with btrfs have spikes of 190MB/s every >>>>> 4-7s. >>>>> >>>>> Why does just another filesystem raises the disk load by a factor of >>>>> 10? >>>> >>>> OK this seems to happen cause ceph is creating every 5s a new >>>> subvolume / >>>> snap. Is this really expected / needed? >>> >>> You can disable it with >>> >>> filestore btrfs snap = false >>> >>> I'm curious how much this drops the load down; originally the >>> snaps were no more expensive than a regular sync but perhaps this >>> has changed... >> >> - with XFS the average write is at 9Mb/s >> - with btrfs (filestore_btrfs_snap=true) write is at 40Mb/s >> - with btrfs (filestore_btrfs_snap=false) write is at 20Mb/s > > Is that the average and not the spikes? It looks like before the spikes > were 20MB/s and 190MB/s? Yes these are average values. Spikes: - with XFS the spike write is at 20Mb/s - with btrfs (filestore_btrfs_snap=true) spike write is 200Mb/s - with btrfs (filestore_btrfs_snap=false) spike is still 185Mb/s but avg is 1/2 (20Mb/s) see above > >> >> Stefan >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
