17.02.2015 04:11, Christian Balzer пишет: > > Hello, > > re-adding the mailing list. > > On Mon, 16 Feb 2015 17:54:01 +0300 Mike wrote: > >> Hello >> >> 05.02.2015 08:35, Christian Balzer пишет: >>> >>> Hello, >>> >>>>> >>>>>> LSI 2308 IT >>>>>> 2 x SSD Intel DC S3700 400GB >>>>>> 2 x SSD Intel DC S3700 200GB >>>>> Why the separation of SSDs? >>>>> They aren't going to be that busy with regards to the OS. >>>> >>>> We would like to use 400GB SSD for a cache pool, and 200GB SSD for >>>> the journaling. >>>> >>> Don't, at least not like that. >>> First and foremost, SSD based OSDs/pools have different requirements, >>> especially when it comes to CPU. >>> Mixing your HDD and SSD based OSDs in the same chassis is a generally >>> a bad idea. >> >> Why? If we have for example SuperServer 6028U-TR4+ with proper >> configuration (4 x SSD DC S3700 for cache pool/8 x 6-8Tb SATA HDD for >> Cold storage/E5-2695V3 CPU/128Gb RAM), why it's still bad idea? It's >> something inside Ceph don't work well? >> > > Ceph in and by itself will of course work. > > But your example up there is total overkill on one hand and simply not > balanced on the other hand. > You'd be much better off (both performance and price wise) if you'd go > with something less powerful for a HDD storage node like this: > http://www.supermicro.com/products/system/2U/6027/SSG-6027R-E1R12T.cfm > with 2 400GB Intels in the back for journals and 16 cores total. > > While your SSD based storage nodes would be nicely dense by using > something like: > http://www.supermicro.com/products/system/2U/2028/SYS-2028TP-DC0TR.cfm > with 2 E5-2690 v3 per node (I'd actually rather prefer E5-2687W v3, but > those are running too hot). > Alternatively one of the 1U cases with up to 10 SSDs. > > Also maintaining a crush map that separates the SSD from HDD pools is made > a lot easier, less error prone by segregating nodes into SSD and HDD ones. > > There are several more reasons below. > > Yes this normal variants of configurations. But in this way you have 2 different nodes versus 1, it's require a more support inside company. In a whole setup you will be have for each MON, OSD, SSD-CACHE servers one configuration and another configurations for compute nodes. A lot of support, supplies, attention. That's why we still trying reduce amount of configuration for support. It's a balance support versus cost/speed/etc. >> For me cache pool it's 1-st fast small storage between big slow storage. >> > That's the idea, yes. > But besides the problems with performance I'm listing again below, that > "small" is another, very difficult to judge in advance problem. > By mixing your cache pool SSD OSDs into the HDD OSD chassis, you're > making yourself inflexible in that area (as in just add another SSD cache > pool node when needed). > Yes in some way inflexible, but I have one configuration not two and can grow up cluster simply add modes. >> You don't need journal anymore and if you need you can enlarge fast >> storage. >> > You still need the journal of course, it's (unfortunately in some cases) > a basic requirement in Ceph. > I suppose what you meant is "don't need journal on SSDs anymore". > And while that is true, this makes your slow storage at least twice as > slow, which at some point (deep-scrub, data re-balancing, very busy > cluster) is likely to make you wish you had those journal SSDs. > > Yes, journal on cold storage is need for re-balancing cluster if some node/hdd fail or promote/remove object from ssd cache. I remember a email in this mail list from one of inktank guys (sorry, didn't remember him full email and name), they wrote that "you no need journal if you use cache pool". >>> If you really want to use SSD based OSDs, got at least with Giant, >>> probably better even to wait for Hammer. >>> Otherwise your performance will be nowhere near the investment you're >>> making. >>> Read up in the ML archives about SSD based clusters and their >>> performance, as well as cache pools. >>> >>> Which brings us to the second point, cache pools are pretty pointless >>> currently when it comes to performance. So unless you're planning to >>> use EC pools, you will gain very little from them. >> >> So, ssd cache pool useless at all? >> > They're (currently) not performing all that well, ask people on the ML > who're actually using them. By now it's true I'm reading ML every day. > This is a combination of Ceph currently being unable to fully utilize the > full potential of SSDs in general and the cache pool code (having to > promote/demote whole objects mainly) in particular. > > Both of these things are of course known to the Ceph developers and being > improved, but right now I don't think they will give you what you expect > from them. > > I would build a good, solid, classic Ceph cluster at this point in time > and have a small cache pool for testing. > Once that pool performs to your satisfaction, you can always grow it. > Another reason to keep SSD based storage nodes separate. > > Christian > Thanks for answer! _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |