Willi; ZFS on RBD seems like a waste, and overkill. A redundant storage solution on top of a redundant storage solution? You can have multiple file systems within CephFS, the thing to note is that each CephFS MUST have a SEPARATE active MDS. For failover, each should have a secondary MDS, and these also need to be separate (preferably running in standby-replay mode). Each MDS instance can only handle one responsibility, for one file system. Each file system also uses 2 pools; one for metadata (think filenames, file properties, and the directory tree), and one for the file data itself. The containerization present by default in Octopus should make running many MDSs easier. We run 3 CephFS file systems from our primary cluster. This uses 6 MDSs, and 6 pools. We assigned the metadata pools to our SSDs (using CRUSH rules) for performance. You might also work with your users on switching to an Object Storage paradigm (think S3), as RadosGW has some nice disaster recovery features. Thank you, Dominic L. Hilsbos, MBA Director - Information Technology Perform Air International, Inc. DHilsbos@xxxxxxxxxxxxxx www.PerformAir.com -----Original Message----- From: Willi Schiegel [mailto:willi.schiegel@xxxxxxxxx] Sent: Sunday, January 26, 2020 4:24 AM To: ceph-users@xxxxxxx Subject: General question CephFS or RBD Hello All, I have a HW RAID based 240 TB data pool with about 200 million files for users in a scientific institution. Data sizes range from tiny parameter files for scientific calculations and experiments to huge images of brain scans. There are group directories, home directories, Windows roaming profile directories organized in ZFS pools on Solaris operating systems, exported via NFS and Samba to Linux, macOS, and Windows clients. I would like to switch to CephFS because of the flexibility and expandability but I cannot find any recommendations for which storage backend would be suitable for all the functionality we have. Since I like the features of ZFS like immediate snapshots of very large data pools, quotas for each file system within hierarchical data trees and dynamic expandability by simply adding new disks or disk images without manual resizing would it be a good idea to create RBD images, map them onto the file servers and create zpools on the mapped images? I know that ZFS best works with raw disks but maybe a RBD image is close enough to a raw disk? Or would CephFS be the way to go? Can there be multiple CephFS pools for the group data folders and for the user's home directory folders for example or do I have to have everything in one single file space? Maybe someone can share his or her field experience? Thank you very much. Best regards Willi _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |