Hello, re-adding the mailing list. On Mon, 16 Feb 2015 17:54:01 +0300 Mike wrote: > Hello > > 05.02.2015 08:35, Christian Balzer пишет: > > > > Hello, > > > >>> > >>>> LSI 2308 IT > >>>> 2 x SSD Intel DC S3700 400GB > >>>> 2 x SSD Intel DC S3700 200GB > >>> Why the separation of SSDs? > >>> They aren't going to be that busy with regards to the OS. > >> > >> We would like to use 400GB SSD for a cache pool, and 200GB SSD for > >> the journaling. > >> > > Don't, at least not like that. > > First and foremost, SSD based OSDs/pools have different requirements, > > especially when it comes to CPU. > > Mixing your HDD and SSD based OSDs in the same chassis is a generally > > a bad idea. > > Why? If we have for example SuperServer 6028U-TR4+ with proper > configuration (4 x SSD DC S3700 for cache pool/8 x 6-8Tb SATA HDD for > Cold storage/E5-2695V3 CPU/128Gb RAM), why it's still bad idea? It's > something inside Ceph don't work well? > Ceph in and by itself will of course work. But your example up there is total overkill on one hand and simply not balanced on the other hand. You'd be much better off (both performance and price wise) if you'd go with something less powerful for a HDD storage node like this: http://www.supermicro.com/products/system/2U/6027/SSG-6027R-E1R12T.cfm with 2 400GB Intels in the back for journals and 16 cores total. While your SSD based storage nodes would be nicely dense by using something like: http://www.supermicro.com/products/system/2U/2028/SYS-2028TP-DC0TR.cfm with 2 E5-2690 v3 per node (I'd actually rather prefer E5-2687W v3, but those are running too hot). Alternatively one of the 1U cases with up to 10 SSDs. Also maintaining a crush map that separates the SSD from HDD pools is made a lot easier, less error prone by segregating nodes into SSD and HDD ones. There are several more reasons below. > For me cache pool it's 1-st fast small storage between big slow storage. > That's the idea, yes. But besides the problems with performance I'm listing again below, that "small" is another, very difficult to judge in advance problem. By mixing your cache pool SSD OSDs into the HDD OSD chassis, you're making yourself inflexible in that area (as in just add another SSD cache pool node when needed). > You don't need journal anymore and if you need you can enlarge fast > storage. > You still need the journal of course, it's (unfortunately in some cases) a basic requirement in Ceph. I suppose what you meant is "don't need journal on SSDs anymore". And while that is true, this makes your slow storage at least twice as slow, which at some point (deep-scrub, data re-balancing, very busy cluster) is likely to make you wish you had those journal SSDs. > > If you really want to use SSD based OSDs, got at least with Giant, > > probably better even to wait for Hammer. > > Otherwise your performance will be nowhere near the investment you're > > making. > > Read up in the ML archives about SSD based clusters and their > > performance, as well as cache pools. > > > > Which brings us to the second point, cache pools are pretty pointless > > currently when it comes to performance. So unless you're planning to > > use EC pools, you will gain very little from them. > > So, ssd cache pool useless at all? > They're (currently) not performing all that well, ask people on the ML who're actually using them. This is a combination of Ceph currently being unable to fully utilize the full potential of SSDs in general and the cache pool code (having to promote/demote whole objects mainly) in particular. Both of these things are of course known to the Ceph developers and being improved, but right now I don't think they will give you what you expect from them. I would build a good, solid, classic Ceph cluster at this point in time and have a small cache pool for testing. Once that pool performs to your satisfaction, you can always grow it. Another reason to keep SSD based storage nodes separate. Christian -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |