Thx Sage It's clear now Best regards -----Message d'origine----- De : Sage Weil [mailto:sage@xxxxxxxxxxxx] Envoyé : jeudi 12 novembre 2015 16:01 À : CHEVALIER Ghislain IMT/OLPS Cc : ceph-devel@xxxxxxxxxxxxxxx Objet : RE: [CEPH][Crush][Tunables] issue when updating tunables On Thu, 12 Nov 2015, ghislain.chevalier@xxxxxxxxxx wrote: > Hi Sage, > Thanks for the reply > > You said > " You actually want straw_calc_version 1. This is just confusing output from the 'firefly' tunable detection... the straw_calc_version does not have any client dependencies." > > My objective is to have the most relevant tunables for a firefly platform. > > I didn't understand if : > - it's better to have straw_calc_version set to 1 but tunables_optimal will be automatically set to 0. > In other words are the following tunables OK ? > { "choose_local_tries": 0, > > "choose_local_fallback_tries": 0, > > "choose_total_tries": 50, > > "chooseleaf_descend_once": 1, > > "chooseleaf_vary_r": 1, > > "straw_calc_version": 0, > > "profile": "firefly", > > "optimal_tunables": 1, > > "legacy_tunables": 0, > > "require_feature_tunables": 1, > > "require_feature_tunables2": 1, > > "require_feature_tunables3": 1, > > "has_v2_rules": 0, > > "has_v3_rules": 0} It is best to also manually set straw_calc_version = 1. It won't say firefly, but it will still be compatible with firefly clients (that tunable only affects the mon behavior when updating the crush map). > - there's an issue with the tunables detection and update? Yes and no. We want 'ceph osd crush tunables firefly' to set options supported by all firefly deployments, and the initial firefly releases did not have the straw_calc_version = 1 support. In some cases switching it on can trigger some data movement the next time the crush map is adjusted, so we leave it off to be conservative. And we want the profile to match exactly what setting the profile sets. But it's confusing since it isn't 1:1 with what clients support. And if it is a fresh cluster you are better off with straw_calc_version = 1. (Same goes for old clusters, if you can tolerate a bit of initial rebalancing.) sage > > Best regards > > -----Message d'origine----- > De : Sage Weil [mailto:sage@xxxxxxxxxxxx] Envoyé : mardi 10 novembre > 2015 11:23 À : CHEVALIER Ghislain IMT/OLPS Cc : > ceph-devel@xxxxxxxxxxxxxxx Objet : Re: [CEPH][Crush][Tunables] issue > when updating tunables > > On Tue, 10 Nov 2015, ghislain.chevalier@xxxxxxxxxx wrote: > > Hi all, > > > > Context: > > Firefly 0.80.9 > > Ubuntu 14.04.1 > > Almost a production platform in an openstack environment > > 176 OSD (SAS and SSD), 2 crushmap-oriented storage classes , 8 > > servers in 2 rooms, 3 monitors on openstack controllers > > Usage: Rados Gateway for object service and RBD as back-end for > > Cinder and Glance > > > > The Ceph cluster was installed by Mirantis procedures (puppet/fuel/ceph-deploy): > > > > I noticed that tunables were curiously set. > > ceph osd crush show-tunables ==> > > { "choose_local_tries": 0, > > "choose_local_fallback_tries": 0, > > "choose_total_tries": 50, > > "chooseleaf_descend_once": 1, > > "chooseleaf_vary_r": 1, > > "straw_calc_version": 1, > > "profile": "unknown", > > "optimal_tunables": 0, > > "legacy_tunables": 0, > > "require_feature_tunables": 1, > > "require_feature_tunables2": 1, > > "require_feature_tunables3": 1, > > "has_v2_rules": 0, > > "has_v3_rules": 0} > > > > I tried to update them > > ceph osd crush tunables optimal ==> adjusted tunables profile to > > optimal > > > > But when checking > > ceph osd crush show-tunables ==> > > { "choose_local_tries": 0, > > "choose_local_fallback_tries": 0, > > "choose_total_tries": 50, > > "chooseleaf_descend_once": 1, > > "chooseleaf_vary_r": 1, > > "straw_calc_version": 1, > > "profile": "unknown", > > "optimal_tunables": 0, > > "legacy_tunables": 0, > > "require_feature_tunables": 1, > > "require_feature_tunables2": 1, > > "require_feature_tunables3": 1, > > "has_v2_rules": 0, > > "has_v3_rules": 0} > > > > Nothing has changed. > > > > I finally did > > ceph osd crush set-tunable straw_calc_version 0 > > You actually want straw_calc_version 1. This is just confusing output from the 'firefly' tunable detection... the straw_calc_version does not have any client dependencies. > > sage > > > > > > and > > ceph osd crush show-tunables ==> > > { "choose_local_tries": 0, > > "choose_local_fallback_tries": 0, > > "choose_total_tries": 50, > > "chooseleaf_descend_once": 1, > > "chooseleaf_vary_r": 1, > > "straw_calc_version": 0, > > "profile": "firefly", > > "optimal_tunables": 1, > > "legacy_tunables": 0, > > "require_feature_tunables": 1, > > "require_feature_tunables2": 1, > > "require_feature_tunables3": 1, > > "has_v2_rules": 0, > > "has_v3_rules": 0} > > > > It's OK > > > > My question: > > Does the "ceph osd crush tunables <profile>" command change all the requested parameters in order to set the tunables to the right profile? > > > > Brgds > > > > ____________________________________________________________________ > > __ ___________________________________________________ > > > > Ce message et ses pieces jointes peuvent contenir des informations > > confidentielles ou privilegiees et ne doivent donc pas etre > > diffuses, exploites ou copies sans autorisation. Si vous avez recu > > ce message par erreur, veuillez le signaler a l'expediteur et le > > detruire ainsi que les pieces jointes. Les messages electroniques > > etant susceptibles d'alteration, France Telecom - Orange decline > > toute responsabilite si ce message a ete altere, deforme ou > > falsifie. Merci > > > > This message and its attachments may contain confidential or > > privileged information that may be protected by law; they should not be distributed, used or copied without authorization. > > If you have received this email in error, please notify the sender and delete this message and its attachments. > > As emails may be altered, France Telecom - Orange shall not be liable if this message was modified, changed or falsified. > > Thank you. > > > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" > > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo > > info at http://vger.kernel.org/majordomo-info.html > > > > > > ______________________________________________________________________ > ___________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations > confidentielles ou privilegiees et ne doivent donc pas etre diffuses, > exploites ou copies sans autorisation. Si vous avez recu ce message > par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or > privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > > _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
