When the exclusive-lock feature is used, any and all Ceph users used
for RBD purposes should be double-checked to ensure that they have
permission to blacklist clients. This would effect both librbd and
krbd, but only after a non-clean shutdown where the image is left in a
locked state by a dead client.
Prior to upgrading the monitors to the Luminous release, verify that
your RBD users have at least the following cap:
$ ceph auth get client.<RBD ID> 2>&1 | grep mon
caps mon = "allow r, allow command "osd blacklist""
Post upgrade to the Luminous release, you can future-proof your RBD
caps by updating to the following:
$ ceph auth get client.rbd_read_write
exported keyring for client.test
[client.rbd_read_write]
key = AQCyzbdZ13EVARAAt7EpNOt4C911Q3CEtBiCyw==
caps mon = "profile rbd"
caps osd = "profile rbd pool=xyz"
$ ceph auth get client.rbd_read_only
exported keyring for client.test
[client.rbd_read_only]
key = AQCyzbdZ13EVARAAt7EpNOt4C911Q3CEtBiCyw==
caps mon = "profile rbd"
caps osd = "profile rbd-read-only"
[1] http://docs.ceph.com/docs/master/rados/operations/user-management/#authorization-capabilities
On Tue, Sep 12, 2017 at 7:52 AM, Blair Bethwaite
<blair.bethwaite@xxxxxxxxx> wrote:
> You're the OP, so for that, thanks! Our upgrade plan (for Thursday
> this week) was modified today to include prep work to double-check the
> caps.
>
> On 12 September 2017 at 21:26, Nico Schottelius
> <nico.schottelius@xxxxxxxxxxx> wrote:
>>
>> Well, we basically needed to fix it, that's why did it :-)
>>
>>
>> Blair Bethwaite <blair.bethwaite@xxxxxxxxx> writes:
>>
>>> Great to see this issue sorted.
>>>
>>> I have to say I am quite surprised anyone would implement the
>>> export/import workaround mentioned here without *first* racing to this
>>> ML or IRC and crying out for help. This is a valuable resource, made
>>> more so by people sharing issues.
>>>
>>> Cheers,
>>>
>>> On 12 September 2017 at 07:22, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:
>>>> Yes -- the upgrade documentation definitely needs to be updated to add
>>>> a pre-monitor upgrade step to verify your caps before proceeding -- I
>>>> will take care of that under this ticket [1]. I believe the OpenStack
>>>> documentation has been updated [2], but let me know if you find other
>>>> places.
>>>>
>>>> [1] http://tracker.ceph.com/issues/21353
>>>> [2] http://docs.ceph.com/docs/master/rbd/rbd-openstack/#setup-ceph-client-authentication
>>>>
>>>> On Mon, Sep 11, 2017 at 5:16 PM, Nico Schottelius
>>>> <nico.schottelius@xxxxxxxxxxx> wrote:
>>>>>
>>>>> That indeed worked! Thanks a lot!
>>>>>
>>>>> The remaining question from my side: did we do anything wrong in the
>>>>> upgrade process and if not, should it be documented somewhere how to
>>>>> setup the permissions correctly on upgrade?
>>>>>
>>>>> Or should the documentation on the side of the cloud infrastructure
>>>>> software be updated?
>>>>>
>>>>>
>>>>>
>>>>> Jason Dillaman <jdillama@xxxxxxxxxx> writes:
>>>>>
>>>>>> Since you have already upgraded to Luminous, the fastest and probably
>>>>>> easiest way to fix this is to run "ceph auth caps client.libvirt mon
>>>>>> 'profile rbd' osd 'profile rbd pool=one'" [1]. Luminous provides
>>>>>> simplified RBD caps via named profiles which ensure all the correct
>>>>>> permissions are enabled.
>>>>>>
>>>>>> [1] http://docs.ceph.com/docs/master/rados/operations/user-management/#authorization-capabilities
>>>>>
>>>>> --
>>>>> Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
>>>>
>>>>
>>>>
>>>> --
>>>> Jason
>>>> _______________________________________________
>>>> ceph-users mailing list
>>>> ceph-users@xxxxxxxxxxxxxx
>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
>> --
>> Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
>
>
>
> --
> Cheers,
> ~Blairo
--
Jason
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
