insofar as I can tell, yes. Everything indicates that they are in effect.
On Thu, Feb 23, 2017 at 7:14 PM, Brad Hubbard <bhubbard@xxxxxxxxxx> wrote:
> Is your change reflected in the current crushmap?
>
> On Fri, Feb 24, 2017 at 12:07 PM, Schlacta, Christ <aarcane@xxxxxxxxxxx> wrote:
>> ---------- Forwarded message ----------
>> From: Schlacta, Christ <aarcane@xxxxxxxxxxx>
>> Date: Thu, Feb 23, 2017 at 6:06 PM
>> Subject: Re: Upgrade Woes on suse leap with OBS ceph.
>> To: Brad Hubbard <bhubbard@xxxxxxxxxx>
>>
>>
>> So setting the above to 0 by sheer brute force didn't work, so it's
>> not crush or osd problem.. also, the errors still say mon0, so I
>> suspect it's related to communication between libceph in kernel and
>> the mon.
>>
>> aarcane@densetsu:/etc/target$ sudo ceph --cluster rk osd crush tunables hammer
>> adjusted tunables profile to hammer
>> aarcane@densetsu:/etc/target$ ceph --cluster rk osd crush show-tunables
>> {
>> "choose_local_tries": 0,
>> "choose_local_fallback_tries": 0,
>> "choose_total_tries": 50,
>> "chooseleaf_descend_once": 1,
>> "chooseleaf_vary_r": 1,
>> "chooseleaf_stable": 0,
>> "straw_calc_version": 1,
>> "allowed_bucket_algs": 54,
>> "profile": "hammer",
>> "optimal_tunables": 0,
>> "legacy_tunables": 0,
>> "minimum_required_version": "firefly",
>> "require_feature_tunables": 1,
>> "require_feature_tunables2": 1,
>> "has_v2_rules": 0,
>> "require_feature_tunables3": 1,
>> "has_v3_rules": 0,
>> "has_v4_buckets": 0,
>> "require_feature_tunables5": 0,
>> "has_v5_rules": 0
>> }
>>
>> aarcane@densetsu:/etc/target$ sudo rbd --cluster rk map rt1
>> rbd: sysfs write failed
>> In some cases useful info is found in syslog - try "dmesg | tail" or so.
>> rbd: map failed: (110) Connection timed out
>> aarcane@densetsu:~$ dmesg | tail
>> [10118.778868] libceph: mon0 10.0.0.67:6789 feature set mismatch, my
>> 40106b84a842a52 < server's e0106b84a846a52, missing a00000000004000
>> [10118.779597] libceph: mon0 10.0.0.67:6789 missing required protocol features
>> [10119.834634] libceph: mon0 10.0.0.67:6789 feature set mismatch, my
>> 40106b84a842a52 < server's e0106b84a846a52, missing a00000000004000
>> [10119.835174] libceph: mon0 10.0.0.67:6789 missing required protocol features
>> [10120.762983] libceph: mon0 10.0.0.67:6789 feature set mismatch, my
>> 40106b84a842a52 < server's e0106b84a846a52, missing a00000000004000
>> [10120.763707] libceph: mon0 10.0.0.67:6789 missing required protocol features
>> [10121.787128] libceph: mon0 10.0.0.67:6789 feature set mismatch, my
>> 40106b84a842a52 < server's e0106b84a846a52, missing a00000000004000
>> [10121.787847] libceph: mon0 10.0.0.67:6789 missing required protocol features
>> [10122.911117] libceph: mon0 10.0.0.67:6789 feature set mismatch, my
>> 40106b84a842a52 < server's e0106b84a846a52, missing a00000000004000
>> [10122.911872] libceph: mon0 10.0.0.67:6789 missing required protocol features
>> aarcane@densetsu:~$
>>
>>
>> On Thu, Feb 23, 2017 at 5:56 PM, Schlacta, Christ <aarcane@xxxxxxxxxxx> wrote:
>>> They're from the suse leap ceph team. They maintain ceph, and build
>>> up to date versions for suse leap. What I don't know is how to
>>> disable it. When I try, I get the following mess:
>>>
>>> aarcane@densetsu:/etc/target$ ceph --cluster rk osd crush set-tunable
>>> require_feature_tunables5 0
>>> Invalid command: require_feature_tunables5 not in straw_calc_version
>>> osd crush set-tunable straw_calc_version <int> : set crush tunable
>>> <tunable> to <value>
>>> Error EINVAL: invalid command
>>>
>>> On Thu, Feb 23, 2017 at 5:54 PM, Brad Hubbard <bhubbard@xxxxxxxxxx> wrote:
>>>> On Fri, Feb 24, 2017 at 11:00 AM, Schlacta, Christ <aarcane@xxxxxxxxxxx> wrote:
>>>>> aarcane@densetsu:~$ ceph --cluster rk osd crush show-tunables
>>>>> {
>>>>> "choose_local_tries": 0,
>>>>> "choose_local_fallback_tries": 0,
>>>>> "choose_total_tries": 50,
>>>>> "chooseleaf_descend_once": 1,
>>>>> "chooseleaf_vary_r": 1,
>>>>> "chooseleaf_stable": 1,
>>>>> "straw_calc_version": 1,
>>>>> "allowed_bucket_algs": 54,
>>>>> "profile": "jewel",
>>>>> "optimal_tunables": 1,
>>>>> "legacy_tunables": 0,
>>>>> "minimum_required_version": "jewel",
>>>>> "require_feature_tunables": 1,
>>>>> "require_feature_tunables2": 1,
>>>>> "has_v2_rules": 0,
>>>>> "require_feature_tunables3": 1,
>>>>> "has_v3_rules": 0,
>>>>> "has_v4_buckets": 0,
>>>>> "require_feature_tunables5": 1,
>>>>
>>>> I suspect setting the above to 0 would resolve the issue with the
>>>> client but there may be a reason why this is set?
>>>>
>>>> Where did those packages come from?
>>>>
>>>>> "has_v5_rules": 0
>>>>> }
>>>>>
>>>>> On Thu, Feb 23, 2017 at 4:45 PM, Brad Hubbard <bhubbard@xxxxxxxxxx> wrote:
>>>>>> On Thu, Feb 23, 2017 at 5:18 PM, Schlacta, Christ <aarcane@xxxxxxxxxxx> wrote:
>>>>>>> So I updated suse leap, and now I'm getting the following error from
>>>>>>> ceph. I know I need to disable some features, but I'm not sure what
>>>>>>> they are.. Looks like 14, 57, and 59, but I can't figure out what
>>>>>>> they correspond to, nor therefore, how to turn them off.
>>>>>>>
>>>>>>> libceph: mon0 10.0.0.67:6789 feature set mismatch, my 40106b84a842a42
>>>>>>> < server's e0106b84a846a42, missing a00000000004000
>>>>>>
>>>>>> http://cpp.sh/2rfy says...
>>>>>>
>>>>>> Bit 14 set
>>>>>> Bit 57 set
>>>>>> Bit 59 set
>>>>>>
>>>>>> Comparing this to
>>>>>> https://github.com/ceph/ceph/blob/master/src/include/ceph_features.h
>>>>>> shows...
>>>>>>
>>>>>> DEFINE_CEPH_FEATURE(14, 2, SERVER_KRAKEN)
>>>>>> DEFINE_CEPH_FEATURE(57, 1, MON_STATEFUL_SUB)
>>>>>> DEFINE_CEPH_FEATURE(57, 1, MON_ROUTE_OSDMAP) // overlap
>>>>>> DEFINE_CEPH_FEATURE(57, 1, OSDSUBOP_NO_SNAPCONTEXT) // overlap
>>>>>> DEFINE_CEPH_FEATURE(57, 1, SERVER_JEWEL) // overlap
>>>>>> DEFINE_CEPH_FEATURE(59, 1, FS_BTIME)
>>>>>> DEFINE_CEPH_FEATURE(59, 1, FS_CHANGE_ATTR) // overlap
>>>>>> DEFINE_CEPH_FEATURE(59, 1, MSG_ADDR2) // overlap
>>>>>>
>>>>>> $ echo "obase=16;ibase=16;$(echo e0106b84a846a42-a00000000004000|tr
>>>>>> '[a-z]' '[A-Z]')"|bc -qi
>>>>>> obase=16;ibase=16;E0106B84A846A42-A00000000004000
>>>>>> 40106B84A842A42
>>>>>>
>>>>>> So "me" (the client kernel) does not have the above features that are
>>>>>> present on the servers.
>>>>>>
>>>>>> Can you post the output of "ceph osd crush show-tunables"?
>>>>>>
>>>>>>>
>>>>>>> SuSE Leap 42.2 is Up to date as of tonight, no package updates available.
>>>>>>> All the ceph packages have the following version:
>>>>>>>
>>>>>>> 11.1.0+git.1486588482.ba197ae-72.1
>>>>>>>
>>>>>>> And the kernel has version:
>>>>>>>
>>>>>>> 4.4.49-16.1
>>>>>>>
>>>>>>> It was working perfectly before the upgrade.
>>>>>>>
>>>>>>> Thank you very much
>>>>>>> _______________________________________________
>>>>>>> ceph-users mailing list
>>>>>>> ceph-users@xxxxxxxxxxxxxx
>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Cheers,
>>>>>> Brad
>>>>
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>> Brad
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
> --
> Cheers,
> Brad
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
