Den tis 25 maj 2021 kl 02:51 skrev Tuffli, Chuck <chuck.tuffli@xxxxxxx>: > I found a warning in Red Hat documentation regarding OSD and RBD: > Ceph Block Devices must be deployed on separate nodes from the Ceph Monitor and OSD nodes. Running kernel clients and kernel server daemons on the same node can lead to kernel deadlocks. [1] > It was hard to tell from the documentation if this is still true as some Rook documentation [2] mentioned a solution. Is having an RBD on the same node as the OSD's still a no-no or is it OK with the conditions listed in the Rook docs (i.e. patched kernel + no XFS)? TIA! I think it warns against a machine hosting data on an OSD while at the same time mounting an RBD which has parts served by the OSD it serves. If you end up with a situation where both parts require lots of memory (during OSD recovery for instance) and the RBD (or the filesystem on top of it) also needs memory, stuff can wedge so both suffer. The same would go for running a mon on the same machine as the OSD, if the OSD memory requirements pushes the machine into swap for instance, then OSD recovery would be slower but probably be "more or less ok", but if the mon process gets swapped out at the same time, it will most certainly not be a good member of the mon quorum and that will potentially lead to even worse issues than a slow OSD recovering. If all things are perfect, you have endless ram and cpu and iops and so on then this might not apply to you, but if you are in the lower 50% of machine resources like half the ceph admins out here, then it might just be a good idea to consider the above warning. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |