On 6/19/22 23:23, Christian Wuerdig wrote:
On Sun, 19 Jun 2022 at 02:29, Satish Patel <satish.txt@xxxxxxxxx> wrote:
Greeting folks,
We are planning to build Ceph storage for mostly cephFS for HPC workload
and in future we are planning to expand to S3 style but that is yet to be
decided. Because we need mass storage, we bought the following HW.
15 Total servers and each server has a 12x18TB HDD (spinning disk) . We
understand SSD/NvME would be best fit but it's way out of budget.
I hope you have extra HW on hand for Monitor and MDS servers
^^ this. It also depends on the uptime guarantees you have to provide
(if any). Are the HPC users going to write large files? Or loads of
small files? The more metadata operations the busier the MDSes will be,
but if it's mainly large files the load on them will be much lower.
Ceph recommends using a faster disk for wal/db if the data disk is slow and
in my case I do have a slower disk for data.
Question:
1. Let's say if i want to put a NvME disk for wal/db then what size i
should buy.
The official recommendation is to budget 4% of OSD size for WAL/DB - so in
your case that would be 720GB per OSD. Especially if you want to go to S3
later you should stick closer to that limit since RGW is a heavy meta data
user.
CephFS can be metadata heavy also, depending on work load. You can
co-locate the S3 service on this cluster later on, but from an
operational perspective this might not be preferred: you can tune the
hardware / configuration for each use case. Easier to troubleshoot,
independent upgrade cycle, etc.
Gr. Stefan
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
