Quoting Matt Larson (larsonmattr@xxxxxxxxx): > > The MDS servers are not planned to have SSD drives. The storage > servers would have HD's and 1 nVME SSD drive that could hold metadata > volumes. Be careful with that. One thing we have learned (the hard way) is that the impact of a lot of OMAP data (key-value info used by the MDS ... and RGW for indexes for that matter) condensed on a limited amount of OSDs might hurt performance. Ideally you would want to spread it out accross as many (flash) drives as you can. Hundreds of thousands to a couple million files / directories will probably be not be an issue. Are you planning on storing (processed) datasets on CephFS for the long term as well? In that case a lot of files / directories will matter (over time). If budget / hardware allows buy more NVMe ... and partition them in equal parts (benchmarks show that 4 would be optimal). And put the WAL/DB of the HDs (spinners) on them. Be careful though, if one NVMe dies the HDs that depend on them are going down with it. It's a tradeoff between efficiency and cost / performance. Gr. Stefan -- | BIT BV https://www.bit.nl/ Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / info@xxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |