Hi Eugen,
On 3/11/25 16:48, Eugen Block wrote:
Hi Daniel,
the first thing to mention is, while min_size 2/size 3 is good, having
only three nodes leaves the cluster without any options to recover in
case of a node failure. So it 's recommended to use at least four nodes.
What exactly do you mean with "without any options to recover"? From my
understanding, with min_size=2 I can still operate the cluster with 2
healthy nodes in read-write I/O during recovery of the third node.
Even when another node failed, I still have one node in read-only mode
while recovery node 2 and 3. Sure, this is not optimal, but it's okay
for my use-case.
You have to be aware that the hybrid rule only gives you performance
advantages for read requests (from primary the OSD). Writing is only
completed when all replicas have acked the write, so your clients will
be waiting for the HDDs to ack.
The 5 TB are not wasted if you have other pools utilizing HDDs.
Regards,
Eugen
Zitat von Daniel Vogelbacher <daniel@xxxxxxxxxxxxxx>:
Hi,
I want to setup a 3-node Ceph cluster with fault domain configured to
"host".
Each node should be equipped with:
6x SAS3 HDD 12TB
1x SAS3 SSD 7TB (should be extended to 2x7 later)
The ceph configuration should be size=3, min_size=2. All nodes are
connected with 2x10Gbit (LACP).
I want to use different CRUSH rules for different pools. CephFS and
low priority/IO VMs stored on RBD should use only HDD drives with
default replication CRUSH rule.
For high priority VMs, I want to create another RBD data pool which
uses a modified CRUSH replication rule:
|# Hybrid storage policy rule hybrid { ruleset 2 type replicated step
take ssd step chooseleaf firstn 1 type host step emit step take hdd
step chooseleaf firstn -1 type host step emit } |
|For pools using this hybrid rule, PGs are stored on one SSD
(primary) and two HDD (secondary) devices. But these have different
sizes in my hardware setup. What happens with the remaining disk
space (12-7=5) 5GB on the secondary devices? Is it just unusable, or
will ceph use it for other pools with default replication? In any
case, I don't bother about these 5GB, just want to know how it works.
For the above setup, can you recommend any important configuration
settings and should I modify the OSD weighting? Thanks. |-- Best
regards / Mit freundlichen Grüßen
Daniel Vogelbacher
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
--
Best regards / Mit freundlichen Grüßen
Daniel Vogelbacher
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
