Do I correctly assume that these nodes hosted only the OSDs, and the monitors were on a separate node? On Mon, Jun 18, 2012 at 10:56 AM, Mark Nelson <mark.nelson@xxxxxxxxxxx> wrote: > Hi Guys, > > I've been tracking down some performance issues over the past month with our > internal test nodes and believe I have narrowed it down to something related > to Ubuntu Oneiric. Tests done on nodes running Ubuntu Precise are > significantly faster. > > One of the major differences between the releases is the support for syncfs > in libc. Theoretically this shouldn't have a big effect on btrfs so I'm not > totally sure that this is the culprit. Having said that, previous tests > showed good SSD performance on Oneiric leading me to believe the lower > latency mitigates the effect. Some of spinning disk seekwatcher results for > Oneiric are quite strange with long periods of inactivity on the OSD data > disks. > > I wanted to post these results for those of you who have had performance > problems in the past. If you are continuing to have issues, you may want to > try testing on precise and see if you notice any changes. It is possible > that all of this could be specific to our internal testing nodes, so I > wouldn't mind hearing if other people have seen similar behavior. > > These tests were done using rados bench with 16 concurrent requests. There > are two nodes that each have a single 7200rpm OSD data disk and journal on a > second 7200rpm disk. Replication is set at the default level (2). Kernel > is 3.4 in all cases. > > Here's a run down (Numbers are MB/s) > > 4KB Requests > > BTRFS EXT4 XFS > Ceph 0.46/Oneiric: 0.073 0.694 0.723 > Ceph 0.46/Precise: 2.15 2.031 1.546 > Ceph 0.47.2/Oneiric: 1.072 0.836 0.749 > Ceph 0.47.2/Precise: 2.566 2.579 1.498 > > 128KB Requests: > > BTRFS EXT4 XFS > Ceph 0.46/Oneiric: 11.874 20.066 12.641 > Ceph 0.46/Precise: 49.304 39.736 38.982 > Ceph 0.47.2/Oneiric: 13.81 19.05 12.739 > Ceph 0.47.2/Precise: 47.943 49.655 36.764 > > > 4MB Requests: > > BTRFS EXT4 XFS > Ceph 0.46/Oneiric: 110.202 26.58 15.445 > Ceph 0.46/Precise: 135.975 128.759 106.426 > Ceph 0.47.2/Oneiric: 91.337 46.277 23.897 > Ceph 0.47.2/Precise: 136.906 134.955 106.545 > > I've posted seekwatcher results for all of the tests: > > Ceph 0.46/Oneiric: http://nhm.ceph.com/movies/sprint/test2 > Ceph 0.46/Precise: http://nhm.ceph.com/movies/sprint/test3 > Ceph 0.47.2/Oneiric: http://nhm.ceph.com/movies/sprint/test4 > Ceph 0.47.2/Precise: http://nhm.ceph.com/movies/sprint/test5 > > Mark > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
