Again, penultimately you will need to sit down, compile and compare the numbers. Start with this: http://ark.intel.com/products/family/83425/Data-Center-SSDs Pay close attention to the 3610 SSDs, while slightly more expensive they offer 10 times the endurance. Guestimate the amount of data written to your cluster per day, break that down to the load a journal SSD will see and then multiply by at least 5 to be on the safe side. Then see which SSD will fit your expected usage pattern. You didn't mention your network, but I assume it's 10Gb/s? At 135MB/s writes the 100GB DC S3500 will not cut the mustard in any shape or form when journaling for 4 HDDs. With 2 HDDs it might be a so-so choice, but still falling short. Most currenth 7.2K RPM HDDs these days can do around 150MB/s writes, however that's neither uniform, nor does Ceph do anything resembling a sequential write (which is where these speeds come from), so in my book 80-120MB/s on the SSD journal per HDD are enough. A speed hit is one thing, more than halving your bandwidth is bad, especially when thinking about backfilling. Journal size doesn't matter that much, 10GB is fine, 20GB x4 is OK with the 100GB DC drives, with 5xx consumer models I'd leave at least 50% free. Christian On Sun, 1 Mar 2015 15:08:10 -0600 Tony Harris wrote: > Now, I've never setup a journal on a separate disk, I assume you have 4 > partitions at 10GB / partition, I noticed in the docs they referred to 10 > GB, as a good starter. Would it be better to have 4 partitions @ 10g ea > or 4 @20? > > I know I'll take a speed hit, but unless I can get my work to buy the > drives, they will have to sit with what my personal budget can afford and > be willing to donate ;) > > -Tony > > On Sun, Mar 1, 2015 at 2:54 PM, Andrei Mikhailovsky <andrei@xxxxxxxxxx> > wrote: > > > I am not sure about the enterprise grade and underprovisioning, but for > > the Intel 520s i've got 240gbs (the speeds of 240 is a bit better than > > 120s). and i've left 50% underprovisioned. I've got 10GB for journals > > and I am using 4 osds per ssd. > > > > Andrei > > > > > > ------------------------------ > > > > *From: *"Tony Harris" <nethfel@xxxxxxxxx> > > *To: *"Andrei Mikhailovsky" <andrei@xxxxxxxxxx> > > *Cc: *ceph-users@xxxxxxxxxxxxxx, "Christian Balzer" <chibi@xxxxxxx> > > *Sent: *Sunday, 1 March, 2015 8:49:56 PM > > > > *Subject: *Re: SSD selection > > > > Ok, any size suggestion? Can I get a 120 and be ok? I see I can get > > DCS3500 120GB for within $120/drive so it's possible to get 6 of > > them... > > > > -Tony > > > > On Sun, Mar 1, 2015 at 12:46 PM, Andrei Mikhailovsky > > <andrei@xxxxxxxxxx> wrote: > > > >> > >> I would not use a single ssd for 5 osds. I would recommend the 3-4 > >> osds max per ssd or you will get the bottleneck on the ssd side. > >> > >> I've had a reasonable experience with Intel 520 ssds (which are not > >> produced anymore). I've found Samsung 840 Pro to be horrible! > >> > >> Otherwise, it seems that everyone here recommends the DC3500 or DC3700 > >> and it has the best wear per $ ratio out of all the drives. > >> > >> Andrei > >> > >> > >> ------------------------------ > >> > >> *From: *"Tony Harris" <nethfel@xxxxxxxxx> > >> *To: *"Christian Balzer" <chibi@xxxxxxx> > >> *Cc: *ceph-users@xxxxxxxxxxxxxx > >> *Sent: *Sunday, 1 March, 2015 4:19:30 PM > >> *Subject: *Re: SSD selection > >> > >> > >> Well, although I have 7 now per node, you make a good point and I'm > >> in a position where I can either increase to 8 and split 4/4 and have > >> 2 ssds, or reduce to 5 and use a single osd per node (the system is > >> not in production yet). > >> > >> Do all the DC lines have caps in them or just the DC s line? > >> > >> -Tony > >> > >> On Sat, Feb 28, 2015 at 11:21 PM, Christian Balzer <chibi@xxxxxxx> > >> wrote: > >> > >>> On Sat, 28 Feb 2015 20:42:35 -0600 Tony Harris wrote: > >>> > >>> > Hi all, > >>> > > >>> > I have a small cluster together and it's running fairly well (3 > >>> > nodes, > >>> 21 > >>> > osds). I'm looking to improve the write performance a bit though, > >>> which > >>> > I was hoping that using SSDs for journals would do. But, I was > >>> wondering > >>> > what people had as recommendations for SSDs to act as journal > >>> > drives. If I read the docs on ceph.com correctly, I'll need 2 ssds > >>> > per node (with 7 drives in each node, I think the recommendation > >>> > was 1ssd per > >>> 4-5 > >>> > drives?) so I'm looking for drives that will work well without > >>> > breaking the bank for where I work (I'll probably have to purchase > >>> > them myself and donate, so my budget is somewhat small). Any > >>> > suggestions? I'd prefer one that can finish its write in a power > >>> > outage case, the only one I know of off hand is the intel dcs3700 > >>> > I think, but at $300 it's WAY above my affordability range. > >>> > >>> Firstly, an uneven number of OSDs (HDDs) per node will bite you in > >>> the proverbial behind down the road when combined with journal SSDs, > >>> as one of > >>> those SSDs will wear our faster than the other. > >>> > >>> Secondly, how many SSDs you need is basically a trade-off between > >>> price, performance, endurance and limiting failure impact. > >>> > >>> I have cluster where I used 4 100GB DC S3700s with 8 HDD OSDs, > >>> optimizing the write paths and IOPS and failure domain, but not the > >>> sequential speed or cost. > >>> > >>> Depending on what your write load is and the expected lifetime of > >>> this cluster, you might be able to get away with DC S3500s or even > >>> better the new DC S3610s. > >>> Keep in mind that buying a cheap, low endurance SSD now might cost > >>> you more down the road if you have to replace it after a year > >>> (TBW/$). > >>> > >>> All the cheap alternatives to DC level SSDs tend to wear out too > >>> fast, have no powercaps and tend to have unpredictable (caused by > >>> garbage collection) and steadily decreasing performance. > >>> > >>> Christian > >>> -- > >>> Christian Balzer Network/Systems Engineer > >>> chibi@xxxxxxx Global OnLine Japan/Fusion Communications > >>> http://www.gol.com/ > >>> > >> > >> > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users@xxxxxxxxxxxxxx > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >> > >> > >> > > > > -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |