Re: Stretch cluster questions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

Just made some basic tests, feature works nicely as far as I have tested :)

I created 2 aditional pools each with a matching stretch rule:
- size=2/min=1 (not advised I know)
- size=6/min=3 (some kind of paranoid config)

When entering degraded stretch mode, the following changed where made automatically:
- size=4/min=2 -> size=4/min=1
- size=2/min=1 -> size=2/min=0 (!)
- size=6/min=3 -> size=6/min=1 (!)

Not really sure about what calc is performed here, but:
- It would be better to check not decrement min value below 1?
- Changing min=3 to min=2 would be better (safer)?

Also, when stretch bucket was back online and after recovery was complete:
- size=4/min=2 -> size=4/min=1 -> size=4/min=2
- size=2/min=1 -> size=2/min=0 -> size=2/min=2
- size=6/min=3 -> size=6/min=1 -> size=6/min=2

This time it seems recovery is setting min=2 as a fixed value.

Would it make sense that when entering degraded mode, min was set to min(round(size/2)-1, 1); and after recovery it was set to round(size/2)?

Otherwise awesome feature really! :-)

Cheers

El 5/5/22 a las 10:01, Eneko Lacunza escribió:

Hi Gregory

Thanks for your confirmation. I hope I can start some tests today.

Cheers

El 5/5/22 a las 5:19, Gregory Farnum escribió:
On Wed, May 4, 2022 at 1:25 AM Eneko Lacunza <elacunza@xxxxxxxxx> wrote:

    Hi Gregory,

    El 3/5/22 a las 22:30, Gregory Farnum escribió:
    On Mon, Apr 25, 2022 at 12:57 AM Eneko Lacunza<elacunza@xxxxxxxxx>  <mailto:elacunza@xxxxxxxxx>  wrote:
    We're looking to deploy a stretch cluster for a 2-CPD deployment. I have
    read the following docs:
    https://docs.ceph.com/en/latest/rados/operations/stretch-mode/#stretch-clusters

    I have some questions:

    - Can we have multiple pools in a stretch cluster?
    Yes.

    - Can we have multiple different crush rules in a stretch cluster? I'm
    asking this because the command for stretch mode activation asks for a
    rule...
    Right, so what happens there is any pool with a default rule gets
    switched to the specified CRUSH rule. That doesn't stop you from
    changing the rule after engaging stretch mode, or giving it a
    non-default rule ahead of time. You just have to be careful to make
    sure it satisfies the stretch mode rules about placing across data
    centers.
    So the only purpose of the "stretch_rule" param to
    "enable_stretch_mode" is replacing default replicated rule?

    If so, when stretch mode is activated, the described special
    behaviour applies to all crush rules/pools:

    - the OSDs will only take PGs active when they peer across data
    centers (or whatever other CRUSH bucket type you specified),
    assuming both are alive

    - Pools will increase in size from the default 3 to 4, expecting
    2 copies in each site (for size=2 pools will it increase to 4 too?)

    Is this accurate?


Yep! AFAIK most clusters just use the default pool for replication, barring mixed media types. *shrug*



    We want to have different purpose pools on this Ceph cluster:

    - Important VM disks, with 2 copies in each DC (SSD class)
    - Ephemeral VM disks, with just 2 copies overall (SSD class)
    - Backup data in just one DC (HDD class).

    Objective of the 2-DC deployment is disaster recovery, HA isn't
    required, but I'll take it if deployment is reasonable :-) .
    I'm leery of this for the reasons described in the docs — if you don't
    have 2 replicas per site, you lose data availability every time an OSD
    goes down for any reason (or else you have a window while recovery
    happens where the data is not physically available in both sites,
    which rather negates the purpose).

    Is this because the following: "the OSDs will only take PGs
    active when they peer across data centers (or whatever other
    CRUSH bucket type you specified), assuming both are alive"?



    This conclusion wasn't obvious to me, but after your reply and a
    third read, now seems expected :-)


Right!



    Thanks for your comments. I'll be doing some tests, will write
    back if I find something unexpected (to me?) :-)

    Cheers

    Eneko Lacunza
    Zuzendari teknikoa | Director técnico
    Binovo IT Human Project

    Tel. +34 943 569 206 |https://www.binovo.es
    Astigarragako Bidea, 2 - 2º  <https://www.google.com/maps/search/Astigarragako+Bidea,+2+-+2%C2%BA?entry=gmail&source=g>  izda. Oficina 10-11, 20180 Oiartzun

    https://www.youtube.com/user/CANALBINOVO
    https://www.linkedin.com/company/37269706/



     EnekoLacunza

Director Técnico | Zuzendari teknikoa

Binovo IT Human Project

	943 569 206 <tel:943 569 206>

	elacunza@xxxxxxxxx <mailto:elacunza@xxxxxxxxx>

	binovo.es <//binovo.es>

	Astigarragako Bidea, 2 - 2 izda. Oficina 10-11, 20180 Oiartzun

	
youtube <https://www.youtube.com/user/CANALBINOVO/>	
	linkedin <https://www.linkedin.com/company/37269706/>	
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux