M.2 was not designed for hot swap, and Icydock's solution is a bit outside specification. I really like the new Supermicro box (610P) that has 12 spinning disks and then 6 NVMs. 2 of them in 2.5"x7mm format and 4 of them in the new E1.S format. E1.S is practically next gen hot plug M.2 Ján Senko Proton Technologies AG Lead Storage Engineer ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Monday, September 13th, 2021 at 20:23, Reed Dier <reed.dier@xxxxxxxxxxx> wrote: > I've been eyeing a similar icydock product (https://www.icydock.com/goods.php?id=309 https://www.icydock.com/goods.php?id=309) for make M.2 drives more serviceable. > > While M.2 isn't ideal, if you have a 2U/4U box with a ton of available slots in the back, you could use these with some Micron 7300 MAX or like M.2's for WAL/DB. > > In theory would make identifying failed M.2 easier/quicker, and allow hot-servicing, rather than say an on-motherboard slot, requiring a full server pull to service. > > Curious if anyone has experience with it yet. > > Reed > > > On Sep 9, 2021, at 12:36 PM, Mark Nelson mnelson@xxxxxxxxxx wrote: > > > > I don't think the bigger tier 1 enterprise vendors have really jumped on, but I've been curious to see if anyone would create a dense hotswap m.2 setup (possibly combined with traditional 3.5" HDD bays). The only vendor I've really seen even attempt something like this is icydock: > > > > https://www.icydock.com/goods.php?id=287 > > > > 8 NVMe m.2 devices in a single 5.25" bay. They also have another version that does 6 m.2 in 2x3.5". You could imagine that one of the tier 1 enterprise vendors could probably do something similar on the back of a traditional 12-bay 2U 3.5" chassis. Stick in some moderately sized high write endurance m.2 devices and you're looking at something like 2 OSD DB/WAL per NVMe. As it is, 6:1 with 2x2.5" seems to be pretty typical and isn't terrible if you use decent drives. > > > > Mark > > > > On 9/9/21 12:04 PM, David Orman wrote: > > > > > Exactly, we minimize the blast radius/data destruction by allocating > > > > > > more devices for DB/WAL of smaller size than less of larger size. We > > > > > > encountered this same issue on an earlier iteration of our hardware > > > > > > design. With rotational drives and NVMEs, we are now aiming for a 6:1 > > > > > > ratio based on our CRUSH rules/rotational disk sizing/nvme > > > > > > sizing/server sizing/EC setup/etc. > > > > > > Make sure to use write-friendly NVMEs for DB/WAL and the failures > > > > > > should be much fewer and further between. > > > > > > On Thu, Sep 9, 2021 at 9:11 AM Janne Johansson icepic.dz@xxxxxxxxx wrote: > > > > > > > Den tors 9 sep. 2021 kl 16:09 skrev Michal Strnad michal.strnad@xxxxxxxxx: > > > > > > > > > When the disk with DB died > > > > > > > > > > it will cause inaccessibility of all depended OSDs (six or eight in our > > > > > > > > > > environment), > > > > > > > > > > How do you do it in your environment? > > > > > > > > > > Have two ssds for 8 OSDs, so only half go away when one ssd dies. > > > > > > > > -- > > > > > > > > May the most significant bit of your life be positive. > > > > > > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > > > > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |