Hello, you may want to read up on the various high-density node threads and conversations here. You most certainly do NOT need high end-storage systems to create multi-petabyte storage systems with Ceph. If you were to use these chassis as a basis: https://www.supermicro.com.tw/products/system/4U/6048/SSG-6048R-E1CR60N.cfm [We (and surely others) urged Supermicro to provide a design like this] And fill them with 6TB HDDs, configure them as 5x 12HDD RAID6s, set your replication to 2 in Ceph, you will wind up with VERY reliable, resilient 1.2PB per rack (32U, leaving space for other bits and not melting the PDUs). Add fast SSDs or NVMes to this case for journals and you have decently performing mass storage. Need more IOPS for really hot data? Add a cache tier or dedicated SSD pools for special needs/customers. Alternatively, do "classic" Ceph with 3x replication or EC coding, but in either case (even more so with EC) you will need the most firebreathing CPUs available, so compared to the above design it may be a zero sum game cost wise, if not performance wise as well. This leaves you with 960TB in the same space when doing 3x replication. A middle of the road approach would be to use RAID1 or 10 based OSDs to bring down the computational needs in exchange for higher storage costs (effective 4x replication). This only gives you 720TB, alas it will be easier (and cheaper CPU cost wise) to achieve peak performance with this approach compared to the one above with 60 OSDs per node. Lastly, I give you this (and not being a fan of Fujitsu, mind): http://www.fujitsu.com/global/products/computing/storage/eternus-cd/ Christian On Mon, 30 May 2016 10:25:35 +0430 Jack Makenz wrote: > Forwarded conversation > Subject: Wasting the Storage capacity when using Ceph based On high-end > storage systems > ------------------------ > > From: *Jack Makenz* <jack.makenz@xxxxxxxxx> > Date: Sun, May 29, 2016 at 6:52 PM > To: ceph-community@xxxxxxxxxxxxxx > > > Hello All, > There are some serious problem about ceph that may waste storage capacity > when using high-end storage system(Hitachi, IBM, EMC, HP ,...) as > back-end for OSD hosts. > > Imagine in the real cloud we need *n Petabytes* of storage capacity that > commodity hardware's hard disks or OSD server's hard disks can't provide > this amount of storage capacity. thus we have to use storage systems as > back-end for OSD hosts(to implement OSD daemons ). > > But because almost all of these storage systems ( Regardless of their > brand) use Raid technology and also ceph replicate at least two copy of > each Object, lot's amount of storage capacity waste. > > So is there any solution to solve this problem/misunderstand ? > > Regards > Jack Makenz > > ---------- > From: *Nate Curry* <curry@xxxxxxxxxxxxx> > Date: Mon, May 30, 2016 at 5:50 AM > To: Jack Makenz <jack.makenz@xxxxxxxxx> > Cc: Unknown <ceph-community@xxxxxxxxxxxxxx> > > > I think that purpose of ceph is to get away from having to rely on high > end storage systems and to be provide the capacity to utilize multiple > less expensive servers as the storage system. > > That being said you should still be able to use the high end storage > systems with or without RAID enabled. You could do away with RAID > altogether and let Ceph handle the redundancy or you can have LUNs > assigned to hosts be put into use as OSDs. You could make it work > however but to get the most out of your storage with Ceph I think a > non-RAID configuration would be best. > > Nate Curry > > > _______________________________________________ > > Ceph-community mailing list > > Ceph-community@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-community-ceph.com > > > > > ---------- > From: *Doug Dressler* <darbymorrison@xxxxxxxxx> > Date: Mon, May 30, 2016 at 6:02 AM > To: Nate Curry <curry@xxxxxxxxxxxxx> > Cc: Jack Makenz <jack.makenz@xxxxxxxxx>, Unknown < > ceph-community@xxxxxxxxxxxxxx> > > > For non-technical reasons I had to run ceph initially using SAN disks. > > Lesson learned: > > Make sure deduplication is disabled on the SAN :-) > > > > ---------- > From: *Jack Makenz* <jack.makenz@xxxxxxxxx> > Date: Mon, May 30, 2016 at 9:05 AM > To: Nate Curry <curry@xxxxxxxxxxxxx>, ceph-community@xxxxxxxxxxxxxx > > > Thanks Nate, > But as i mentioned before , providing petabytes of storage capacity on > commodity hardware or enterprise servers is almost impossible, of course > that it's possible by installing hundreds of servers with 3 terabytes > hard disks, but this solution waste data center raise floor, power > consumption and also *money* :) -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Rakuten Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |