According to http://ceph.com/docs/master/rados/operations/crush-map/, you should be able to construct a clever use of 'step take' and 'step choose' rules in your CRUSH map to force one copy to a particular bucket and allow the other two copies to be chosen elsewhere. I was looking for a way to have some locality like one copy in the current rack, one copy in the current row and a third copy somewhere else in the data center. I don't think there is an easy way to figure that out since the clients don't map to the CRUSH map. On Tue, Sep 2, 2014 at 9:54 AM, Jakes John <jakesjohn12345 at gmail.com> wrote: > Thanks Loic. > > > > On Mon, Sep 1, 2014 at 11:31 PM, Loic Dachary <loic at dachary.org> wrote: > >> Hi John, >> >> On 02/09/2014 05:29, Jakes John wrote:> Hi, >> > I have some general questions regarding the crush map. It would be >> helpful if someone can help me out by clarifying them. >> > >> > 1. I saw that a bucket 'host' is always created for the crush maps >> which are automatically generated by ceph. If I am manually creating >> crushmap, do I need to always add a bucket called ' host' ? As I was >> looking through the source code, I didn't see any need for this. If not >> necessary, can osd's of the same host be split into mulitple buckets? >> > >> > eg : Say host 1 has four osd's- osd.0,osd.1,osd.2, osd.3 >> > host 2 has four osd's- >> osd.4,osd.5,osd.6,osd.7 >> > >> > and create two buckets - >> > >> > > HostGroup bucket1- {osd.0, osd.1,osd.4,osd.5} >> > HostGroup bucket2-{osd.2,osd.3,osd.6,osd.7} where HostGroup is new >> bucket type instead of the default 'host' type. >> > >> > >> > Is this configuration possible or invalid? If this is possible, I can >> group SSD's of all hosts into 1 bucket and HDD's into other. >> >> What you describe seem possible but I'm not sure what problem you are >> trying to solve. The crush map described at >> >> >> http://ceph.com/docs/master/rados/operations/crush-map/#placing-different-pools-on-different-osds >> >> is not what you want ? >> > > No. I have seen this before. In the example, each of the hosts had either > only SSD's or only HDD's. In my example, host1 has two ssd's and two hdd's. > Similar is the case for host2. So, If I need to have a bucket for SSD's > only, I need to create a bucket as stated above. > > HostGroup bucket1- {osd.0, osd.1,osd.4,osd.5} > > Is this possible? > >> >> > 2. I have read in Ceph docs that same osd is not advised to be part of >> two buckets(two pools). >> >> A single OSD should be in a single bucket in the crush map indeed. But it >> is common for two OSD to be part of multiple pools. The pools are >> associated with a ruleset and each ruleset can choose in the same set of >> OSDs. >> >> > Is there any reason for it? But,I couldn't find this limitation in the >> source code. >> >> There is no limitation in the code but the crush function has been tested >> and used with a hierarchy where leaf nodes are not part of more than one >> bucket. >> >> > > >> Cheers >> >> > eg:osd.0 is in bucket1 and bucket2. >> > >> > Is this configuration possible or invalid? If this is possible, I have >> the flexibility to have group data which are written to different pools. >> > > So, it is possible right?. I have plans to have third replica to stay in a > particular rack for all pools.( common to all pools) > > >> > >> > 3. Is it possible to exclude or include a particular osd/host/rack in >> the crush mapping?. >> > >> > eg: I need to have third replica always in rack3 (a specified >> row/rack/host based on requirements) . First two can be chosen randomly >> > >> > If possible, how can I configure it? >> > >> > > Any ideas for 3,4,5 ? > >> > >> > 4. It is said that osd weights must be configured based on the storage. >> Say if I have SSD of 512 GB and HDD of 1 TB and if I configure .5 and 1 >> respectively, am I treating both SSD and HDD equally? How do I prioritize >> SSD over HDD? >> > >> > > > >> > 5. Continuing from 4), If i have mix of SSD's and HDD's in the same >> host, what are the best ways possible to utilize the SSD capabilities in >> the ceph cluster? >> > >> > >> > > > >> > Looking forward to your help, >> > >> > Thanks, >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > _______________________________________________ >> > ceph-users mailing list >> > ceph-users at lists.ceph.com >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > >> >> -- >> Lo?c Dachary, Artisan Logiciel Libre >> >> > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140902/bbb8f8d4/attachment.htm>
 |