Re: Questions about the EC pool

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



It should take ~25 seconds by default to detect a network failure, the
config option that controls this is "osd heartbeat grace" (default 20
seconds, but it takes a little longer for it to really detect the
failure).
Check ceph -w while performing the test.


Paul

-- 
Paul Emmerich

Looking for help with your Ceph cluster? Contact us at https://croit.io

croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90

On Fri, Nov 29, 2019 at 8:14 AM majia xiao <xiaomajia.st@xxxxxxxxx> wrote:
>
> Hello,
>
>
> We have a Ceph cluster (version 12.2.4) with 10 hosts, and there are 21 OSDs on each host.
>
>
>  An EC pool is created with the following commands:
>
>
> ceph osd erasure-code-profile set profile_jerasure_4_3_reed_sol_van \
>
>   plugin=jerasure \
>
>   k=4 \
>
>   m=3 \
>
>   technique=reed_sol_van \
>
>   packetsize=2048 \
>
>   crush-device-class=hdd \
>
>   crush-failure-domain=host
>
>
> ceph osd pool create pool_jerasure_4_3_reed_sol_van 2048 2048 erasure profile_jerasure_4_3_reed_sol_van
>
>
>
> Here are my questions:
>
> The EC pool is created using k=4, m=3, and crush-device-class=hdd, so we just disable the network interfaces of some hosts (using "ifdown" command) to verify the functionality of the EC pool while performing ‘rados bench’ command.
> However, the IO rate drops immediately to 0 when a single host goes offline, and it takes a long time (~100 seconds) for the IO rate becoming normal.
> As far as I know, the default value of min_size is k+1 or 5, which means that the EC pool can be still working even if there are two hosts offline.
> Is there something wrong with my understanding?
> According to our observations, it seems that the IO rate becomes normal when Ceph detects all OSDs corresponding to the failed host.
> Is there any way to reduce the time needed for Ceph to detect all failed OSDs?
>
>
>
> Thanks for any help.
>
>
> Best regards,
>
> Majia Xiao
>
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux