My question is,
Is it possible to create a 3 copies ruleset where first copy is stored
on class nvme and all other copies is stored on class hdd? At the same
time making sure that the step to store on class hdd is not placed in
the same datacenter as the first copy on class nvme?
Below is a simplified ceph setup of a Hybrid solution where one copy is
stored on one NVMe drive and two HDD drives. Advantage is great read
performance and cost savings. Disadvantages is low write performance.
Still the write performance is good thanks to rockdb on Intel Optane
disks in HDD servers.
I have six servers in this example.
Only NVMe drives on storage101, storage102 and storage103.
Only HDD drives on storage201, storage202 and storage203.
All servers is connected at 40 Gbit public network and 40 Gbit cluster
network. Backbone is 100 Gbit.
root default
├── datacenter Alfa/
│ ├── host Storage101
│ │ ├── OSD 1TB NVMe
│ │ ├── OSD 1TB NVMe
│ │ └── OSD 1TB NVMe
│ └── host Storage201
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ └── OSD 10TB HDD
│
├── datacenter Bravo/
│ ├── host Storage102
│ │ ├── OSD 1TB NVMe
│ │ ├── OSD 1TB NVMe
│ │ └── OSD 1TB NVMe
│ └── host Storage202
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ ├── OSD 10TB HDD
│ └── OSD 10TB HDD
│
└── datacenter Charlie/
├── host Storage101
│ ├── OSD 1TB NVMe
│ ├── OSD 1TB NVMe
│ └── OSD 1TB NVMe
└── host Storage201
├── OSD 10TB HDD
├── OSD 10TB HDD
├── OSD 10TB HDD
├── OSD 10TB HDD
├── OSD 10TB HDD
└── OSD 10TB HDD
rule hybrid {
id 1
type replicated
min_size 1
max_size 10
step take default class nvme
step chooseleaf firstn 1 type datacenter
step emit
step take default class hdd
step chooseleaf firstn -1 type datacenter
step emit
}
Above rule works but has the problem that storing in class hdd still can
choose the datacenter where the first copy is stored on class nvme. With
this setup, failure in one datacenter will make the ceph cluster loose
qourum.
Is it possible to create a rule so all -1 copies is stored in another
osd class AND datacenter?
Regards,
Niklas
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
