OK, it seems during the rbd creation with --stripe-count != 1 you have to follow the rule: stripe-unit * stripe-count = object-size So, EINVAL below was correct, but the message "librbd: STRIPINGV2 and format 2 or later required for non-default striping" is wrong and confusing. Now I have the problme that even with rbd.ko from kernel 3.13.5 this volume is not mountable: [root@rx37-0 ~]# rbd create --pool SSD-r2 --size 20480 --order 22 --image-format 2 --stripe-unit 1048576 --stripe-count 4 t2-1 [root@rx37-0 ~]# rbd -p SSD-r2 info t2-1 rbd image 't2-1': size 20480 MB in 5120 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.12722ae8944a format: 2 features: layering, striping stripe unit: 1024 kB stripe count: 4 [root@rx37-0 ~]# rbd -p SSD-r2 map t2-1 rbd: add failed: (22) Invalid argument [root@rx37-0 ~]# uname -a Linux rx37-0.primary.ceph-poc.fsc.net 3.13.5-1.el6.elrepo.x86_64 #1 SMP Sat Feb 22 21:34:07 EST 2014 x86_64 x86_64 x86_64 GNU/Linux [root@rx37-0 ~]# dmesg | tail rbd: image t2-1: unsupported stripe unit (got 1048576 want 4194304) Any hints ? Regards, -Dieter On Tue, Mar 11, 2014 at 04:37:46PM +0100, Jean-Charles LOPEZ wrote: > Hi Greg, > > but our default also has stripe-count = 1 so that no more than 1 stripe-unit is included in each order x object. > > So if you do --order 16?stripe-unit 65536 ?stripe-count 1 it then works > > I?m not sure if this is what you meant. > JC > > > > On Mar 11, 2014, at 08:32, Gregory Farnum <greg@xxxxxxxxxxx> wrote: > > > If the stripe size and object size are the same it's just chunking -- > > that's our default. Should work fine. > > -Greg > > Software Engineer #42 @ http://inktank.com | http://ceph.com > > > > > > On Tue, Mar 11, 2014 at 8:23 AM, Jean-Charles LOPEZ > > <jc.lopez@xxxxxxxxxxx> wrote: > >> Hi Dieter, > >> > >> you have a problem with your command. > >> > >> You set order = 16 so your RBD objects is going to be 65536 bytes > >> > >> Then you tell RBD that you stripe-unit is going to be 65536 which is the size of your full object. > >> > >> Either decrease the size of --stripe-unit to 8192 for example > >> Or increase order so that it is bigger than your stripe unit and contains a multiple of stripe-units (e.g. 21) > >> > >> And it will work without any problem > >> JC > >> > >> > >> > >> On Mar 11, 2014, at 07:22, Kasper Dieter <dieter.kasper@xxxxxxxxxxxxxx> wrote: > >> > >>> So, should I open a bug report ? > >>> > >>> STRIPINGV2 feature was added in Ceph v0.53, and I'm running v0.61 and using '--image-format 2' during 'rbd create' > >>> > >>> Regards, > >>> -Dieter > >>> > >>> > >>> On Tue, Mar 11, 2014 at 03:13:28PM +0100, Srinivasa Rao Ragolu wrote: > >>>> of course. rbd userland utilities provide you create images on RADOS as > >>>> block storage. > >>>> > >>>> On Tue, Mar 11, 2014 at 7:37 PM, Kasper Dieter > >>>> <[1]dieter.kasper@xxxxxxxxxxxxxx> wrote: > >>>> > >>>> I know, that format2 in rbd.ko is supported with kernel version 3.10 and > >>>> above. > >>>> > >>>> But, if I want to create an rbd-image > >>>> only the Ceph Userland services should be involved, shouldn't it ? > >>>> > >>>> -Dieter > >>>> > >>>> BTW the kernel version on the nodes hosting the OSDs processes is > >>>> 2.6.32-358.el6.x86_64 > >>>> but I can also boot with a 3.10.32 kernel. > >>>> > >>>> On Tue, Mar 11, 2014 at 02:57:05PM +0100, Srinivasa Rao Ragolu wrote: > >>>>> Please check the kernel version . Only kernel version 3.10 and > >>>> above are > >>>>> supported to create format type 2 images. > >>>>> > >>>>> On Tue, Mar 11, 2014 at 7:16 PM, Kasper Dieter > >>>>> <[1][2]dieter.kasper@xxxxxxxxxxxxxx> wrote: > >>>>> > >>>>> When using "rbd create ... --image-format 2" in some cases this > >>>> CMD is > >>>>> rejected by > >>>>> EINVAL with the message "librbd: STRIPINGV2 and format 2 or later > >>>>> required for non-default striping" > >>>>> But, in v0.61.9 "STRIPINGV2 and format 2" should be supported > >>>>> > >>>>> [root@rx37-3 ~]# rbd create --pool SSD-r2 --size 20480 --order 16 > >>>>> --image-format 2 --stripe-unit 65536 --stripe-count 4 t2 > >>>>> rbd: create error: (22) Invalid argument > >>>>> 2014-03-11 14:39:03.885185 7f15bc170760 -1 librbd: STRIPINGV2 and > >>>> format > >>>>> 2 or later required for non-default striping > >>>>> > >>>>> [root@rx37-3 ~]# ceph -v > >>>>> ceph version 0.61.9 (7440dcd135750839fa0f00263f80722ff6f51e90) > >>>>> > >>>>> Any hints ? > >>>>> > >>>>> Regards, > >>>>> -Dieter > >>>>> _______________________________________________ > >>>>> ceph-users mailing list > >>>>> [2][3]ceph-users@xxxxxxxxxxxxxx > >>>>> [3][4]http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>>>> > >>>>> References > >>>>> > >>>>> Visible links > >>>>> 1. mailto:[5]dieter.kasper@xxxxxxxxxxxxxx > >>>>> 2. mailto:[6]ceph-users@xxxxxxxxxxxxxx > >>>>> 3. [7]http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>>> > >>>> References > >>>> > >>>> Visible links > >>>> 1. mailto:dieter.kasper@xxxxxxxxxxxxxx > >>>> 2. mailto:dieter.kasper@xxxxxxxxxxxxxx > >>>> 3. mailto:ceph-users@xxxxxxxxxxxxxx > >>>> 4. http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>>> 5. mailto:dieter.kasper@xxxxxxxxxxxxxx > >>>> 6. mailto:ceph-users@xxxxxxxxxxxxxx > >>>> 7. http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>> _______________________________________________ > >>> ceph-users mailing list > >>> ceph-users@xxxxxxxxxxxxxx > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >> > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users@xxxxxxxxxxxxxx > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |