Hi Ilya On 27/02/17 13:59, Ilya Dryomov wrote: > On Mon, Feb 27, 2017 at 2:37 PM, Simon Weald <simon@xxxxxxxxxxxxxx> wrote: >> I've currently having some issues making some Jessie-based Xen hosts >> talk to a Trusty-based cluster due to feature mismatch errors. Our >> Trusty hosts are using 3.19.0-80 (the Vivid LTS kernel), and our Jessie >> hosts were using the standard Jessie kernel (3.16). Volumes wouldn't >> map, so I tried the kernel from jessie-backports (4.9.2-2~bpo8+1); still >> no joy. I then tried compiling the latest kernel in the 4.9 branch >> (4.9.12) from source with the Debian kernel config - still no joy. As I >> understand it there have been a lot of changes in krbd which I should >> have pulled in when building from source - am I missing something? Some >> info about the Xen hosts: >> >> root@xen-host:~# uname -r >> 4.9.12-internal >> >> root@xen-host:~# ceph -v >> ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367) >> >> root@xen-host:~# rbd map -p cinder >> volume-88188973-0f40-48a3-8a88-302d1cb5e093 >> rbd: sysfs write failed >> RBD image feature set mismatch. You can disable features unsupported by >> the kernel with "rbd feature disable". >> In some cases useful info is found in syslog - try "dmesg | tail" or so. >> rbd: map failed: (6) No such device or address >> >> root@xen-host:~# dmesg | grep 'unsupported' >> [252723.885948] rbd: image volume-88188973-0f40-48a3-8a88-302d1cb5e093: >> image uses unsupported features: 0x38 >> >> root@xen-host:~# rbd info -p cinder >> volume-88188973-0f40-48a3-8a88-302d1cb5e093 >> rbd image 'volume-88188973-0f40-48a3-8a88-302d1cb5e093': >> size 1024 MB in 256 objects >> order 22 (4096 kB objects) >> block_name_prefix: rbd_data.c6bd3c5f705426 >> format: 2 >> features: layering, exclusive-lock, object-map, fast-diff, deep-flatten >> flags: > object-map, fast-diff, deep-flatten are still unsupported. > >> Do >> >> $ rbd feature disable <image-name> deep-flatten,fast-diff,object-map,exclusive-lock >> >> to disable features unsupported by the kernel client. If you are using the >> kernel client, you should create your images with >> >> $ rbd create --size <size> --image-feature layering <image-name> >> >> or add >> >> rbd default features = 3 >> >> to ceph.conf on the client side. (Setting rbd default features on the >> OSDs will have no effect.) > exclusive-lock is supported starting with 4.9. The above becomes > >> $ rbd feature disable <image-name> deep-flatten,fast-diff,object-map >> $ rbd create --size <size> --image-feature layering,exclusive-lock <image-name> >> rbd default features = 5 > if you want it. > > Thanks, > > Ilya Ok, thanks, understood - I suspected it was still the kernel client which was causing it. As you may have guessed from the pool name, we are mapping volumes created by Openstack through to a separate platform - I would much rather not go altering features after Cinder has created them if at all possible, so this obviously rules krbd out. Which other rbd clients would you suggest we use? I've played with rbd-nbd, but it causes extra complexity as you can't query it for which volumes are mapped to which local nbd device. Additionally, I'm not sure if it'll have a significant performance overhead. I'd appreciate you thoughts! Thanks Simon -- PGP: http://www.simonweald.com/SimonWeald.asc https://pgp.mit.edu/pks/lookup?op=get&search=0x988E9858747ABE88 _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |