On 07/03/2014 03:11 PM, VELARTIS Philipp D?rhammer wrote: > Hi, > > I have a ceph cluster setup (with 45 sata disk journal on disks) and get > only 450mb/sec writes seq (maximum playing around with threads in rados > bench) with replica of 2 > How many threads? > Which is about ~20Mb writes per disk (what y see in atop also) > theoretically with replica2 and having journals on disk should be 45 X > 100mb (sata) / 2 (replica) / 2 (journal writes) which makes it 1125 > satas in reality have 120mb/sec so the theoretical output should be more. > > I would expect to have between 40-50mb/sec for each sata disk > > Can somebody confirm that he can reach this speed with a setup with > journals on the satas (with journals on ssd speed should be 100mb per disk)? > or does ceph only give about ? of the speed for a disk? (and not the ? > as expected because of journals) > Did you verify how much each machine is doing? It could be that the data is not distributed evenly and that on a certain machine the drives are doing 50MB/sec. > My setup is 3 servers with: 2 x 2.6ghz xeons, 128gb ram 15 satas for > ceph (and ssds for system) 1 x 10gig for external traffic, 1 x 10gig for > osd traffic > with reads I can saturate the network but writes is far away. And I > would expect at least to saturate the 10gig with sequential writes also > Should be possible, but with 3 servers the data distribution might not be optimal causing a lower write performance. I've seen 10Gbit write performance on multiple clusters without any problems. > Thank you > > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Wido den Hollander Ceph consultant and trainer 42on B.V. Phone: +31 (0)20 700 9902 Skype: contact42on
 |