It's kind of a philosophical question. Technically there's nothing that
prevents you from putting ceph and the hypervisor on the same boxes.
It's a question of whether or not potential cost savings are worth
increased risk of failure and contention. You can minimize those things
through various means (cgroups, ristricting NUMA nodes, etc). What is
more difficult is isolating disk IO contention (say if you want local
SSDs for VMs), memory bus and QPI contention, network contention, etc.
If the VMs are working really hard you can restrict them to their own
socket, and you can even restrict memory usage to the local socket, but
what about remote socket network or disk IO? (you will almost certainly
want these things on the ceph socket) I wonder as well about increased
risk of hardware failure with the increased load, but I don't have any
statistics.
I'm guessing if you spent enough time at it you could make it work
relatively well, but at least personally I question how beneficial it
really is after all of that. If you are going for cost savings, I
suspect efficient compute and storage node designs will be nearly as
good with much less complexity.
Mark
On 03/26/2015 07:11 AM, Wido den Hollander wrote:
On 26-03-15 12:04, Stefan Priebe - Profihost AG wrote:
Hi Wido,
Am 26.03.2015 um 11:59 schrieb Wido den Hollander:
On 26-03-15 11:52, Stefan Priebe - Profihost AG wrote:
Hi,
in the past i rwad pretty often that it's not a good idea to run ceph
and qemu / the hypervisors on the same nodes.
But why is this a bad idea? You save space and can better use the
ressources you have in the nodes anyway.
Memory pressure during recovery *might* become a problem. If you make
sure that you don't allocate more then let's say 50% for the guests it
could work.
mhm sure? I've never seen problems like that. Currently i ran each ceph
node with 64GB of memory and each hypervisor node with around 512GB to
1TB RAM while having 48 cores.
Yes, it can happen. You have machines with enough memory, but if you
overprovision the machines it can happen.
Using cgroups you could also prevent that the OSDs eat up all memory or CPU.
Never seen an OSD doing so crazy things.
Again, it really depends on the available memory and CPU. If you buy big
machines for this purpose it probably won't be a problem.
Stefan
So technically it could work, but memorey and CPU pressure is something
which might give you problems.
Stefan
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
