On Monday, May 4, 2015, Christian Balzer <chibi@xxxxxxx> wrote:
On Mon, 13 Apr 2015 10:39:57 +0530 Sanjoy Dasgupta wrote:
> Hi!
>
> This is an often discussed and clarified topic, but Reason why I am
> asking is because
>
> If We use a RAID controller with Lot of Cache (FBWC) and Configure each
> Drive as Single Drive RAID0, then Write to disks will benefit by using
> FBWC and accelerate I/O performance. Is this correct assumption ?
>
In the case of Ceph, the journal writes (assuming journals on the HDDs,
not separate SSDs) will benefit from this indeed.
Each of my OSD nodes has 7 2TB disks on one RAID card with 1GB of FBWC with BBU.
I get pretty good performance. With 6 of the above osd nodes and no replication, using rados bench, I can get around 7500 4k write iops and it can peak to over 10k iops.
The write throughput at 4MB is around 1000 MB/s, It will peak to around 1309 MB/s.
> Also, it indeed helps, what are the downside of using RAID Controller
> with Cache in above manner (Apart from Cost of RAID controller) ?
>
If you have too much money or existing HW, knock yourself out. ^.^
Aside from the cost, the cache should be battery backed, additional costs
and maintenance issues.
Setting up RAID0 drives can be painful (megacli must die), and adds
another step to OSD replacement/deployment.
The resulting drive may or may not have all SMART features exposed and
(not applicable in your case) won't support TRIM.
Lastly the cache tends to be small when shared with many HDDs and there is
also competition over it by reads and writes, but that's something to keep
in mind, not a disadvantage per se.
Yes, all of the above.
I think it is worth the money.
I thought I wouldn't need SSD journals, and the raid cards were cheaper than a few good SSDs.
Due to my odd workload, I end up with all of my writes being below 256k in size. I have to use iscsi proxy nodes to connect to my VMware hosts. This limits my VM's throughput to around 200MB/s which isn't fast enough for my application. I'm planning on building new osd nodes with the same raid cards and SSD journals to help further coalesce the small writes.
Christian
--
Christian Balzer Network/Systems Engineer
chibi@xxxxxxx Global OnLine Japan/Fusion Communications
http://www.gol.com/
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
