Re: Lots of radosgw-admin commands fail after upgrade

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Jan 23, 2017 at 2:40 PM, Mustafa Muhammad
<mustafa1024m@xxxxxxxxx> wrote:
> On Mon, Jan 23, 2017 at 1:22 PM, Orit Wasserman <owasserm@xxxxxxxxxx> wrote:
>> On Mon, Jan 23, 2017 at 11:03 AM, Mustafa Muhammad
>> <mustafa1024m@xxxxxxxxx> wrote:
>>> On Mon, Jan 23, 2017 at 11:53 AM, Orit Wasserman <owasserm@xxxxxxxxxx> wrote:
>>>> On Mon, Jan 23, 2017 at 10:52 AM, Mustafa Muhammad
>>>> <mustafa1024m@xxxxxxxxx> wrote:
>>>>> On Mon, Jan 23, 2017 at 10:45 AM, Orit Wasserman <owasserm@xxxxxxxxxx> wrote:
>>>>>> On Mon, Jan 23, 2017 at 9:40 AM, Mustafa Muhammad
>>>>>> <mustafa1024m@xxxxxxxxx> wrote:
>>>>>>> On Sun, Jan 22, 2017 at 4:34 PM, Orit Wasserman <owasserm@xxxxxxxxxx> wrote:
>>>>>>>> On Sun, Jan 22, 2017 at 12:00 PM, Mustafa Muhammad
>>>>>>>> <mustafa1024m@xxxxxxxxx> wrote:
>>>>>>>>> On Sun, Jan 22, 2017 at 12:04 PM, Orit Wasserman <owasserm@xxxxxxxxxx> wrote:
>>>>>>>>>> On Sat, Jan 21, 2017 at 10:24 AM, Mustafa Muhammad
>>>>>>>>>> <mustafa1024m@xxxxxxxxx> wrote:
>>>>>>>>>>> Hello again :)
>>>>>>>>>>>
>>>>>>>>>>> It still doesn't work for me using 10.2.5:
>>>>>>>>>>>
>>>>>>>>>>> [root@monitor3 ~]# ceph -v
>>>>>>>>>>> ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367)
>>>>>>>>>>> [root@monitor3 ~]# radosgw-admin period update --commit
>>>>>>>>>>> 2017-01-21 11:06:20.659487 7f2ca18979c0  0 zonegroup default missing
>>>>>>>>>>> zone for master_zone=
>>>>>>>>>>> couldn't init storage provider
>>>>>>>>>>>
>>>>>>>>>>> I think I am hitting:
>>>>>>>>>>> http://tracker.ceph.com/issues/17364
>>>>>>>>>>>
>>>>>>>>>>> So I created new RPMs with this patch:
>>>>>>>>>>> https://github.com/ceph/ceph/pull/12315
>>>>>>>>>>>
>>>>>>>>>>> But now, it crashes when I try to update the period, I've attached the
>>>>>>>>>>> output and the details of my zonegroup, I also tried the just released
>>>>>>>>>>> Kraken, also crashes.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am working on a fix.
>>>>>>>>>> Will you be able to try it?
>>>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/ceph/ceph/pull/13054
>>>>>>>>
>>>>>>>> Good luck!
>>>>>>>
>>>>>>> This *kind of* worked, it doesn't crash anymore, and zonegroup now
>>>>>>> have master zone, but now I don't have my placement-targets anymore
>>>>>>> I used:
>>>>>>>
>>>>>>> radosgw-admin zone set --rgw-zone=default < new-default-zone.json
>>>>>>> radosgw-admin zonegroup set --rgw-zonegroup=default < new-default-zg.json
>>>>>>> radosgw-admin zonegroupmap set < new-zonegroupmap.json
>>>>>>>
>>>>>>
>>>>>> You don't need this and I suspect this it is the problem.
>>>>>> Can you try without this command?
>>>>>
>>>>> It worked fine, then after restarting the RGW containers, it was lost
>>>>> again, after some retries, I found that starting the older container
>>>>> (10.2.2 because I couldn't use anything newer before), is causing the
>>>>> revert.
>>>>>
>>>>> Now I only started 10.2.5 RGWs and everything works fine.
>>>>>
>>>>> Thank you very much, really appreciated.
>>>>>
>>>>
>>>> :)
>>>>
>>>> Can you open a tracker issue for the zonegroupmap command problem?
>>>> this way it will documented.
>>>
>>> Do you mean the "radosgw-admin zonegroupmap set  <
>>> new-zonegroupmap.json" or the whole issue? I can't verify that it was
>>> the cause, the older instances might be the cause, so not sure what to
>>> write in the issue?
>>>
>> For the crash I have already opened an issue
>> http://tracker.ceph.com/issues/18631.
>> I meant opening an issue for the "zonegroupmap set" that made the new
>> placement configuration disappear.
>> Please add in the description of the issue the commands you used and
>> if you can add the jsons files.
>
> I tested again, it was not the "zonegroupmap set" that caused this, it
> was the older RGWs, I retested "zonegroupmap set" and it didn't mess
> anything, but it also didn't change the map (I changed
> bucket_index_max_shards but it stayed the same), so I'll create two
> issues, one for the older RGW versions mess the map and another one
> for not changing after set operation.
>
Good catch
Thanks,
Orit

> Regards
> Mustafa
>
>>
>> Thanks,
>> Orit
>>> Regards
>>> Mustafa
>>>>
>>>>> Regards
>>>>> Mustafa
>>>>>
>>>>>>
>>>>>>> radosgw-admin zonegroup default --rgw-zonegroup=default
>>>>>>> radosgw-admin zone default --rgw-zone=default
>>>>>>> radosgw-admin period update --commit
>>>>>>>
>>>>>>> Is there something wrong I am doing? Can I update zonegroupmap
>>>>>>> directly (like I did) or should I only set zone and zonegroup, tried
>>>>>>> several things, still only getting:
>>>>>>>
>>>>>>>                 "placement_targets": [
>>>>>>>                     {
>>>>>>>                         "name": "default-placement",
>>>>>>>                         "tags": []
>>>>>>>                     }
>>>>>>>                 ]
>>>>>>>
>>>>>>>
>>>>>>> Regards
>>>>>>> Mustafa
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes, of course :)
>>>>>>>>>
>>>>>>>>> Thank you
>>>>>>>>>
>>>>>>>>> Mustafa
>>>>>>>>>
>>>>>>>>>> Orit
>>>>>>>>>>> What do you think, what should I do?
>>>>>>>>>>>
>>>>>>>>>>> Thanks a lot in advance
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Mustafa Muhammad
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux