On 07/03/2014 08:11 AM, VELARTIS Philipp D?rhammer wrote: > Hi, > > I have a ceph cluster setup (with 45 sata disk journal on disks) and get > only 450mb/sec writes seq (maximum playing around with threads in rados > bench) with replica of 2 > > Which is about ~20Mb writes per disk (what y see in atop also) > theoretically with replica2 and having journals on disk should be 45 X > 100mb (sata) / 2 (replica) / 2 (journal writes) which makes it 1125 > satas in reality have 120mb/sec so the theoretical output should be more. > > I would expect to have between 40-50mb/sec for each sata disk > > Can somebody confirm that he can reach this speed with a setup with > journals on the satas (with journals on ssd speed should be 100mb per disk)? > or does ceph only give about ? of the speed for a disk? (and not the ? > as expected because of journals) > > My setup is 3 servers with: 2 x 2.6ghz xeons, 128gb ram 15 satas for > ceph (and ssds for system) 1 x 10gig for external traffic, 1 x 10gig for > osd traffic > with reads I can saturate the network but writes is far away. And I > would expect at least to saturate the 10gig with sequential writes also In addition to the advice wido is providing (which I wholeheartedly agree with!), you might want to check your controller/disk configuration. If you have journals on the same disks as the data, some times putting the disks into single-disk RAID0 LUNs with writeback cache enabled can help keep journal and data writes from causing seek contention. This only works if you have a controller with cache and a battery though. > > Thank you > > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
 |