Hello, On Wed, 14 Oct 2015 09:25:41 +1000 Lindsay Mathieson wrote: > I'm adding a node (4 * WD RED 3TB) to our small cluster to bring it up to > replica 3. Can we assume from this node that the your current setup is something like 2 nodes with 4 drives each? > Given how much headache it has been managing multiple osd's Any particularities? > (including disk failures) They happen, but given your (assumed) cluster size they should pretty rare. >on my other nodes, I've decided to put all 4 > disks on the new node in a ZFS RAID 10 config with SSD SLOG & Cache with > just one OSD running on top of a ZFS filesystem mount. > Well, well, well... I assume you did read all the past threads about ZFS as backing FS for Ceph? Can your current cluster handle a deep scrub during average, peak time utilization without falling apart and getting unusably slow? > I only have one SSD available for cache and journaling, would I be better > off doing one the following? > > Option 1: > - Journal on the ZFS Filesystem (advantage - simplicity) > COW and journal... not so good, I'd imagine. > Option 2: > - Journal on a SSD partition, for a total of 3 (SLOG, Cache, Ceph > Journal). Advantage - possibly better ceph performance? > Probably. > Option 3 > - Something else? > The more OSDs, the better Ceph will perform (up to a point). Note that I have similar clusters to avoid dealing with disk failures, but unless you can afford to upgrade in a timely fashion and paying for what is effectively a 4 way replication you may want to reconsider your approach. If your cluster is really that lightly loaded that you can sustain performance with a reduced number of OSDs, I'd go with RAID1 OSDs and a replication of 2. So that adding the additional node and rebuilding the old ones will actually only slightly decrease your OSD count (from assumed 8 to 6). > Also - should I put the monitor on ZFS as well? > leveldb and COW, also probably not so good. Christian > If this works out, I'll probably migrate the other two nodes to a > similar setup. > > thanks, > -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |