No difference:
[root@rx37-0 ~]# rbd map --pool SSD-r2 t2-1
rbd: add failed: (22) Invalid argument
[root@rx37-0 ~]# dmesg | tail
(…)
rbd: image t2-1: unsupported stripe unit (got 1048576 want 4194304)
The strace of ‘rbd map’ says:
open("/sys/bus/rbd/add", O_WRONLY) = 3
write(3, "192.168.113.13:6789,192.168.113.14:6789,192.168.113.15:6789 name=admin,key=client.admin SSD-r2 t2-1", 99) = -1 EINVAL (Invalid argument)
-Dieter
From: Michael J. Kidd [mailto:michael.kidd@xxxxxxxxxxx]
Sent: Wednesday, March 12, 2014 12:52 PM
To: Kasper, Dieter
Cc: Pankaj Laddha; Jean-Charles LOPEZ; Madhu Mohan Nelemane; ceph-users@xxxxxxxx
Subject: Re: rbd format 2 && stripe-count != 1 cannot be mapped with rbd.ko kernel 3.13.5
Try with --pool instead of -p
Sent from my mobile device. Please excuse brevity and typographical errors.
On Mar 12, 2014 5:51 AM, "Kasper Dieter" <dieter.kasper@xxxxxxxxxxxxxx> wrote:
OK,
it seems during the rbd creation with --stripe-count != 1
you have to follow the rule: stripe-unit * stripe-count = object-size
So, EINVAL below was correct,
but the message "librbd: STRIPINGV2 and format 2 or later required for non-default striping"
is wrong and confusing.
Now I have the problme that even with rbd.ko from kernel 3.13.5
this volume is not mountable:
[root@rx37-0 ~]# rbd create --pool SSD-r2 --size 20480 --order 22 --image-format 2 --stripe-unit 1048576 --stripe-count 4 t2-1
[root@rx37-0 ~]# rbd -p SSD-r2 info t2-1
rbd image 't2-1':
size 20480 MB in 5120 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.12722ae8944a
format: 2
features: layering, striping
stripe unit: 1024 kB
stripe count: 4
[root@rx37-0 ~]# rbd -p SSD-r2 map t2-1
rbd: add failed: (22) Invalid argument
[root@rx37-0 ~]# uname -a
Linux rx37-0.primary.ceph-poc.fsc.net 3.13.5-1.el6.elrepo.x86_64 #1 SMP Sat Feb 22 21:34:07 EST 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@rx37-0 ~]# dmesg | tail
rbd: image t2-1: unsupported stripe unit (got 1048576 want 4194304)
Any hints ?
Regards,
-Dieter
On Tue, Mar 11, 2014 at 04:37:46PM +0100, Jean-Charles LOPEZ wrote:
> Hi Greg,
>
> but our default also has stripe-count = 1 so that no more than 1 stripe-unit is included in each order x object.
>
> So if you do --order 16?stripe-unit 65536 ?stripe-count 1 it then works
>
> I?m not sure if this is what you meant.
> JC
>
>
>
> On Mar 11, 2014, at 08:32, Gregory Farnum <greg@xxxxxxxxxxx> wrote:
>
> > If the stripe size and object size are the same it's just chunking --
> > that's our default. Should work fine.
> > -Greg
> > Software Engineer #42 @ http://inktank.com | http://ceph.com
> >
> >
> > On Tue, Mar 11, 2014 at 8:23 AM, Jean-Charles LOPEZ
> > <jc.lopez@xxxxxxxxxxx> wrote:
> >> Hi Dieter,
> >>
> >> you have a problem with your command.
> >>
> >> You set order = 16 so your RBD objects is going to be 65536 bytes
> >>
> >> Then you tell RBD that you stripe-unit is going to be 65536 which is the size of your full object.
> >>
> >> Either decrease the size of --stripe-unit to 8192 for example
> >> Or increase order so that it is bigger than your stripe unit and contains a multiple of stripe-units (e.g. 21)
> >>
> >> And it will work without any problem
> >> JC
> >>
> >>
> >>
> >> On Mar 11, 2014, at 07:22, Kasper Dieter <dieter.kasper@xxxxxxxxxxxxxx> wrote:
> >>
> >>> So, should I open a bug report ?
> >>>
> >>> STRIPINGV2 feature was added in Ceph v0.53, and I'm running v0.61 and using '--image-format 2' during 'rbd create'
> >>>
> >>> Regards,
> >>> -Dieter
> >>>
> >>>
> >>> On Tue, Mar 11, 2014 at 03:13:28PM +0100, Srinivasa Rao Ragolu wrote:
> >>>> of course. rbd userland utilities provide you create images on RADOS as
> >>>> block storage.
> >>>>
> >>>> On Tue, Mar 11, 2014 at 7:37 PM, Kasper Dieter
> >>>> <[1]dieter.kasper@xxxxxxxxxxxxxx> wrote:
> >>>>
> >>>> I know, that format2 in rbd.ko is supported with kernel version 3.10 and
> >>>> above.
> >>>>
> >>>> But, if I want to create an rbd-image
> >>>> only the Ceph Userland services should be involved, shouldn't it ?
> >>>>
> >>>> -Dieter
> >>>>
> >>>> BTW the kernel version on the nodes hosting the OSDs processes is
> >>>> 2.6.32-358.el6.x86_64
> >>>> but I can also boot with a 3.10.32 kernel.
> >>>>
> >>>> On Tue, Mar 11, 2014 at 02:57:05PM +0100, Srinivasa Rao Ragolu wrote:
> >>>>> Please check the kernel version . Only kernel version 3.10 and
> >>>> above are
> >>>>> supported to create format type 2 images.
> >>>>>
> >>>>> On Tue, Mar 11, 2014 at 7:16 PM, Kasper Dieter
> >>>>> <[1][2]dieter.kasper@xxxxxxxxxxxxxx> wrote:
> >>>>>
> >>>>> When using "rbd create ... --image-format 2" in some cases this
> >>>> CMD is
> >>>>> rejected by
> >>>>> EINVAL with the message "librbd: STRIPINGV2 and format 2 or later
> >>>>> required for non-default striping"
> >>>>> But, in v0.61.9 "STRIPINGV2 and format 2" should be supported
> >>>>>
> >>>>> [root@rx37-3 ~]# rbd create --pool SSD-r2 --size 20480 --order 16
> >>>>> --image-format 2 --stripe-unit 65536 --stripe-count 4 t2
> >>>>> rbd: create error: (22) Invalid argument
> >>>>> 2014-03-11 14:39:03.885185 7f15bc170760 -1 librbd: STRIPINGV2 and
> >>>> format
> >>>>> 2 or later required for non-default striping
> >>>>>
> >>>>> [root@rx37-3 ~]# ceph -v
> >>>>> ceph version 0.61.9 (7440dcd135750839fa0f00263f80722ff6f51e90)
> >>>>>
> >>>>> Any hints ?
> >>>>>
> >>>>> Regards,
> >>>>> -Dieter
> >>>>> _______________________________________________
> >>>>> ceph-users mailing list
> >>>>> [2][3]ceph-users@xxxxxxxxxxxxxx
> >>>>> [3][4]http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>>>
> >>>>> References
> >>>>>
> >>>>> Visible links
> >>>>> 1. mailto:[5]dieter.kasper@xxxxxxxxxxxxxx
> >>>>> 2. mailto:[6]ceph-users@xxxxxxxxxxxxxx
> >>>>> 3. [7]http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>>
> >>>> References
> >>>>
> >>>> Visible links
> >>>> 1. mailto:dieter.kasper@xxxxxxxxxxxxxx
> >>>> 2. mailto:dieter.kasper@xxxxxxxxxxxxxx
> >>>> 3. mailto:ceph-users@xxxxxxxxxxxxxx
> >>>> 4. http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>> 5. mailto:dieter.kasper@xxxxxxxxxxxxxx
> >>>> 6. mailto:ceph-users@xxxxxxxxxxxxxx
> >>>> 7. http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>> _______________________________________________
> >>> ceph-users mailing list
> >>> ceph-users@xxxxxxxxxxxxxx
> >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>
> >> _______________________________________________
> >> ceph-users mailing list
> >> ceph-users@xxxxxxxxxxxxxx
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
