On 08/27/2012 11:17 AM, Stephen Perkins wrote:
Are there any thoughts on placing all Ceph nodes as single virtual machines running on top of an ESXi hypervisor? What I mean by this is that each brick runs ESXi and then only runs one virtual machine.
This sounds like it's going to have a bunch of overhead, but if the performance under ESXi is acceptable to you, it might make sense. I'm not aware of anyone testing OSDs on ESXi, so you could run into performance issues that don't appear on real hardware.
Are there any advantages of running the MON and MDS servers as independent virtual machines on the same physical brick as an OSD virtual machine (rather than just running the processes)? Multi-port Ethernet systems can segregate traffic between the instances...
If your kernel/glibc doesn't support syncfs(2), running the monitor in a virtual machine can be beneficial since it won't affect the OSD when calling sync(2). I don't see many other advantages, since the monitors are pretty light-weight. It might be a disadvantage to run the MDS in vm, since it will make a large chunk of memory unusable by the OSD, even after the MDS stops using some of it, and vice versa. You may prefer to be able to control the amount of memory available to each independently, but I don't see much advantage there. The OSD will require more memory during recovery than during normal usage, so running the MDS and OSD as processes lets them use the memory the other doesn't need at the moment. The MDS effectively serves as a cache, so having lots of memory for it is important for metadata-heavy workloads. Josh -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
