Re: rados + radosstriper puts fail with "large" input objects (mimic/nautilus, ec pool)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The placement groups are created, and the pool is completely functional for non-striper use. I can 'rados put' test objects in just fine to a pool created with any k value.

It's when the --striper option in 'rados put' is used to invoke the use of libradosstriper, it fails on pools where k is not a power of two.

Example working and non-working profiles are as follows (I also tried w=16 for the 10+3).

[root@ceph-dev-mon2 ~]# ceph osd erasure-code-profile get je-8-3-rfd-osd-echo-nodes
crush-device-class=hdd
crush-failure-domain=osd
crush-root=echo-nodes
jerasure-per-chunk-alignment=false
k=8
m=3
plugin=jerasure
technique=reed_sol_van
w=8

[root@ceph-dev-mon2 ~]# ceph osd erasure-code-profile get je-10-3-rfd-osd-echo-nodes-2
crush-device-class=hdd
crush-failure-domain=osd
crush-root=echo-nodes
jerasure-per-chunk-alignment=false
k=10
m=3
plugin=jerasure
technique=reed_sol_van
w=8

-----Original Message-----
From: Eugen Block <eblock@xxxxxx> 
Sent: 05 September 2019 09:02
To: ceph-users@xxxxxxx
Subject:  Re: rados + radosstriper puts fail with "large" input objects (mimic/nautilus, ec pool)

Hi,

I'm not sure if this could really be a bug. We have configured several ec pools with different profiles, many of them with k not a power of 2, e.g. k3 m7, k7 m11 on Luminous, k5 m1 in a Nautilus test cluster, I haven't had any issues regarding these profile yet.

Can you share the ruleset the pool is created with? Are all of the PGs actually created? You can check with

ceph pg ls-by-pool <POOL> | tr -s ' ' | cut -d " " --fields=1,17

Regards,
Eugen


Zitat von Thomas Byrne - UKRI STFC <tom.byrne@xxxxxxxxxx>:

>> It seems likely that jerasure ec pools (at least) are not compatible 
>> with libradosstriper if k is not a power of 2.
>
> Yes, that seems like the summary of the issue as it stands, I'm 
> intrigued to know what libradosstriper is doing compared to librados 
> that makes it only work on specific layouts, as in my mind these two 
> layers were effectively separate, but there must be something going on 
> here that I'm not aware of.
>
> This definitely seems like a bug, or at least incredibly confusing 
> undocumented behaviour.
>
> Cheers,
> Tom
>
> -----Original Message-----
> From: aoanla@xxxxxxxxx <aoanla@xxxxxxxxx>
> Sent: 04 September 2019 19:36
> To: ceph-users@xxxxxxx
> Subject:  Re: rados + radosstriper puts fail with "large" 
> input objects (mimic/nautilus, ec pool)
>
> Hi Tom:
>
> So, the failing ec code is also 10+2 here:
>
> ceph osd erasure-code-profile set glasgow-ec \
>    k=10 \
>    m=2 \
>    crush-failure-domain=osd
>
> Test EC codes which have succeeded, so far are:
>
> k=4, m=2 failure-domain=osd
> k=4, m=2 failure-domain=host
> k=8, m=2 failure-domain=osd
>
> EC codes which have failed
> k=6, m=2 failure-domain=osd
> k=10, m=2 failure-domain=osd
>
> It seems likely that jerasure ec pools (at least) are not compatible 
> with libradosstriper if k is not a power of 2.
>
> I suggest this be submitted as a bug?
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an 
> email to ceph-users-leave@xxxxxxx 
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an 
> email to ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux