On Mon, Feb 27, 2017 at 3:15 PM, Simon Weald <simon@xxxxxxxxxxxxxx> wrote:
> Hi Ilya
>
> On 27/02/17 13:59, Ilya Dryomov wrote:
>> On Mon, Feb 27, 2017 at 2:37 PM, Simon Weald <simon@xxxxxxxxxxxxxx> wrote:
>>> I've currently having some issues making some Jessie-based Xen hosts
>>> talk to a Trusty-based cluster due to feature mismatch errors. Our
>>> Trusty hosts are using 3.19.0-80 (the Vivid LTS kernel), and our Jessie
>>> hosts were using the standard Jessie kernel (3.16). Volumes wouldn't
>>> map, so I tried the kernel from jessie-backports (4.9.2-2~bpo8+1); still
>>> no joy. I then tried compiling the latest kernel in the 4.9 branch
>>> (4.9.12) from source with the Debian kernel config - still no joy. As I
>>> understand it there have been a lot of changes in krbd which I should
>>> have pulled in when building from source - am I missing something? Some
>>> info about the Xen hosts:
>>>
>>> root@xen-host:~# uname -r
>>> 4.9.12-internal
>>>
>>> root@xen-host:~# ceph -v
>>> ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367)
>>>
>>> root@xen-host:~# rbd map -p cinder
>>> volume-88188973-0f40-48a3-8a88-302d1cb5e093
>>> rbd: sysfs write failed
>>> RBD image feature set mismatch. You can disable features unsupported by
>>> the kernel with "rbd feature disable".
>>> In some cases useful info is found in syslog - try "dmesg | tail" or so.
>>> rbd: map failed: (6) No such device or address
>>>
>>> root@xen-host:~# dmesg | grep 'unsupported'
>>> [252723.885948] rbd: image volume-88188973-0f40-48a3-8a88-302d1cb5e093:
>>> image uses unsupported features: 0x38
>>>
>>> root@xen-host:~# rbd info -p cinder
>>> volume-88188973-0f40-48a3-8a88-302d1cb5e093
>>> rbd image 'volume-88188973-0f40-48a3-8a88-302d1cb5e093':
>>> size 1024 MB in 256 objects
>>> order 22 (4096 kB objects)
>>> block_name_prefix: rbd_data.c6bd3c5f705426
>>> format: 2
>>> features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
>>> flags:
>> object-map, fast-diff, deep-flatten are still unsupported.
>>
>>> Do
>>>
>>> $ rbd feature disable <image-name> deep-flatten,fast-diff,object-map,exclusive-lock
>>>
>>> to disable features unsupported by the kernel client. If you are using the
>>> kernel client, you should create your images with
>>>
>>> $ rbd create --size <size> --image-feature layering <image-name>
>>>
>>> or add
>>>
>>> rbd default features = 3
>>>
>>> to ceph.conf on the client side. (Setting rbd default features on the
>>> OSDs will have no effect.)
>> exclusive-lock is supported starting with 4.9. The above becomes
>>
>>> $ rbd feature disable <image-name> deep-flatten,fast-diff,object-map
>>> $ rbd create --size <size> --image-feature layering,exclusive-lock <image-name>
>>> rbd default features = 5
>> if you want it.
>>
>> Thanks,
>>
>> Ilya
>
>
> Ok, thanks, understood - I suspected it was still the kernel client
> which was causing it. As you may have guessed from the pool name, we are
> mapping volumes created by Openstack through to a separate platform - I
> would much rather not go altering features after Cinder has created them
> if at all possible, so this obviously rules krbd out. Which other rbd
> clients would you suggest we use? I've played with rbd-nbd, but it
> causes extra complexity as you can't query it for which volumes are
> mapped to which local nbd device. Additionally, I'm not sure if it'll
> have a significant performance overhead. I'd appreciate you thoughts!
If you add "rbd default features = 3" to the ceph.conf file that is fed
to cinder/glance on the openstack side, it will create images with just
layering enabled.
Thanks,
Ilya
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
