Hey Cephers,
We have a CephFS use case with the following workload requirements with around 10 Billion small sized files each around 1 to 10 kb
Workloads might be very write intensive. For example when we need to perform backup recovery that writes hundreds of millions of files or large-scale imports that also happen from time to time.
Hierarchy is the following: CephFS CSI Kubernetes volumes that can reach a billion or more files each.
We expect up to 100,000 files/directories.
Overall it would consist of 3000 CephFS volumes each of size about 20 to 30 TB based on PVC each having about 200 million files.
While having a discussion with Dan he mentioned one fear about one FS with 10B files is that it would be impractical to scrub -- it would take too long.
And suggested that we should split this into several small clusters for this particular reason alone.
We are looking forward to documentation, case studies and MDS tuning, configuration references as well as examples if anyone has any knowledge or suggestions about such a workload.
We also want to understand if there are any limitations as well if anyone has tested such a kind of workload in a Ceph cluster.
Kind regards,
Gaurav
_______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
