Hello, On Tue, 04 Nov 2014 20:49:02 +0100 Udo Lembke wrote: > Hi, > since a long time I'm looking for performance improvements for our > ceph-cluster. > The last expansion got better performance, because we add another node > (with 12 OSDs). The storage utilization was after that 60%. > Another node of course does more than lower per OSD disk utilization, it also adds more RAM (cached objects), more distribution of requests, etc. So the question here is, did the usage (number of client IOPS) stay the same and just the total amount of stored data did grow? > Now we reach again 69% (the next nodes are waiting for installation) and > the performance drop! OK, we also change the ceph-version from 0.72.x to > firefly. > But I'm wonder if there an relationship between utilization an > performance?! The OSDs are xfs disks, but now i start to use ext4, > because of the bad fragmentation on a xfs-filesystem (yes, I use the > mountoption allocsize=4M allready). > Does defragmenting (all of) the XFS backed OSDs help? > Has anybody the same effect? > I have nothing anywhere near that full, but I can confirm that XFS fragments worse than ext4 and the less said about BTRFS, the better. ^.^ Also defragmenting (not that they needed it) ext4 volumes felt more lightweight than XFS. Since you now have ext4 OSDs, how about doing a osd bench and fio on those compared to XFS backed ones? Other than the above, Mark listed a number of good reasons why OSDs (HDDs) become slower when getting fuller besides fragmentation. Christian > Udo > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |