Hi folks, In lieu of a prod ready Cephfs I'm wondering what others in the user community are doing for file-serving out of Ceph clusters (if at all)? We're just about to build a pretty large cluster - 2PB for file-based NAS and another 0.5PB rgw. For the rgw component we plan to dip our toes in and use an EC backing pool with a ~25TB (usable) 10K SAS + SSD cache tier. For the file storage we're looking at mounting RBDs (out of a standard 3-replica pool for now) on a collection of presentation nodes, which will use ZFS to stripe together those RBD vdevs into a zpool which we can then carve datasets out of for access from NFS & CIFS clients. Those presentation servers will have some PCIe SSD in them for ZIL and L2ARC devices, and clients will be split across them depending on what ID domain they are coming from. Presentation server availability issues will be handled by mounting the relevant zpool on a spare server, so it won't be HA from a client perspective, but I can't see a way to getting this with an RBD backend. Wondering what the collective wisdom has to offer on such a setup... -- Cheers, ~Blairo
 |