Hi,
Thank you Eugen for pointing it out. Yes, OSDs are deployed by
cephadm with drive_group. It seems that the orch module simplifies
the process to make it easier for users.
yes it does, you can also manage your OSDs via dashboard which makes
it also easier for some users.
When replacing an osd, there will be no PG remapping, and backfill
will restore the data on the new disk, right?
That depends on how you decide to go through the replacement process.
Usually without your intervention (e.g. setting the appropriate OSD
flags) the remapping will happen after an OSD goes down and out.
In case of restoring a host with multiple OSDs, like WAL/DB SSD
needs to be replaced, I see two options.
1) Keep cluster in degraded state and rebuild all OSDs.
2) Mark those OSDs out so PGs are rebalanced, rebuild OSDs, and
bring them back in to rebalance PGs again.
These are basically your options, yes.
The key here is how much time backfilling and rebalancing will take?
The intention is to not keep cluster in degraded state for too long.
I assume they are similar, because either of them is to copy the
same amount of data?
If that's true, then option #2 is pointless.
Could anyone share such experiences, like how long time it takes
to recover how much data on what kind of networking/computing env?
No, option 2 is not pointless, it helps you prevent a degraded state.
Having a small cluster or crush rules that only allow few failed OSDs
it could be dangerous taking out an entire node, risking another
failure and potential data loss. It highly depends on your specific
setup and if you're willing to take the risk during rebuild of a node.
The recovery/backfill speed is also depeneding on the size of OSDs,
the object sizes, amount of data, etc. You would probably need to
search the mailing list for examples from someone sharing their
experience or so, I don't have captured such statistics.
Regards,
Eugen
Zitat von Tony Liu <tonyliu0592@xxxxxxxxxxx>:
Thank you Eugen for pointing it out. Yes, OSDs are deployed by
cephadm with drive_group. It seems that the orch module simplifies
the process to make it easier for users.
When replacing an osd, there will be no PG remapping, and backfill
will restore the data on the new disk, right?
In case of restoring a host with multiple OSDs, like WAL/DB SSD
needs to be replaced, I see two options.
1) Keep cluster in degraded state and rebuild all OSDs.
2) Mark those OSDs out so PGs are rebalanced, rebuild OSDs, and
bring them back in to rebalance PGs again.
The key here is how much time backfilling and rebalancing will take?
The intention is to not keep cluster in degraded state for too long.
I assume they are similar, because either of them is to copy the
same amount of data?
If that's true, then option #2 is pointless.
Could anyone share such experiences, like how long time it takes
to recover how much data on what kind of networking/computing env?
Thanks!
Tony
-----Original Message-----
From: Eugen Block <eblock@xxxxxx>
Sent: Wednesday, November 25, 2020 1:49 AM
To: ceph-users@xxxxxxx
Subject: Re: replace osd with Octopus
Hi,
assuming you deployed with cephadm since you're mentioning Octopus
there's a brief section in [1]. The basis for the OSD deployment is the
drive_group configuration. If nothing has changed in your setup and you
replace an OSD cephadm will detect the available disk and match it with
the drive_group config. If there's enough space on the SSD too, it will
redeploy the OSD.
The same goes for your second case: you'll need to remove all OSDs from
that host, zap the devices, replace the SSD and then cephadm will deploy
the entire host. That's the simple case. If redeploying all OSDs on that
host is not an option you'll probably have to pause orchestrator in
order to migrate devices yourself to prevent to much data movement.
Regards,
Eugen
[1] https://docs.ceph.com/en/latest/mgr/orchestrator/#replace-an-osd
Zitat von Tony Liu <tonyliu0592@xxxxxxxxxxx>:
> Hi,
>
> I did some search about replacing osd, and found some different steps,
> probably for different release?
> Is there recommended process to replace an osd with Octopus?
> Two cases here:
> 1) replace HDD whose WAL and DB are on a SSD.
> 1-1) failed disk is replaced by the same model.
> 1-2) working disk is replaced by bigger one.
> 2) replace the SSD holding WAL and DB for multiple HDDs.
>
>
> Thanks!
> Tony
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
