Re: MDS performance with 10 billion small sized files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



As I understand it the workload you describe is way more suitable for object (S3) storage than for file. These numbers were also tested successfully with RGW.  
Regards,

Josh


On Thu, Dec 1, 2022 at 3:07 PM Gaurav Sitlani <sitlanigaurav7@xxxxxxxxx> wrote:

Hey Cephers,


We have a CephFS use case with the following workload requirements with around 10 Billion small sized files each around 1 to 10 kb 

  • Workloads might be very write intensive. For example when we need to perform backup recovery that writes hundreds of millions of files or large-scale imports that also happen from time to time.

  • Hierarchy is the following: CephFS CSI Kubernetes volumes that can reach a billion or more files each.

  • We expect up to 100,000 files/directories.


Overall it would consist of 3000 CephFS volumes each of size about 20 to 30 TB based on PVC each having about 200 million files.


While having a discussion with Dan he mentioned one fear about one FS with 10B files is that it would be impractical to scrub -- it would take too long.

And suggested that we should split this into several small clusters for this particular reason alone.


We are looking forward to documentation, case studies and MDS tuning, configuration references as well as examples if anyone has any knowledge or suggestions about such a workload.


We also want to understand if there are any limitations as well if anyone has tested such a kind of workload in a Ceph cluster.


Kind regards,

Gaurav

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx

[Index of Archives]     [CEPH Users]     [Ceph Devel]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux