On 10/02/2013 10:45 AM, Oliver Daudey wrote:
Hey Robert,
On 02-10-13 14:44, Robert van Leeuwen wrote:
Hi,
I'm running a test setup with Ceph (dumpling) and Openstack (Grizzly) using libvirt to "patch" the ceph disk directly to the qemu instance.
I'm using SL6 with the patched qemu packages from the Ceph site (which the latest version is still cuttlefish):
http://www.ceph.com/packages/ceph-extras
When I turn on client caching strange things start to happen:
I run filebench to test the performance.
During the filebench the virtual machine will have intermittently really slow network connections:
I'm talking here about ping reply's taking 30 SECONDS so effectively losing the network.
This is what I set in the ceph client:
[client]
rbd cache = true
rbd cache writethrough until flush = true
Anyone else noticed this behaviour before or have some troubleshooting tips?
I noticed exactly the same thing when trying RBD-caching on libvirt for
some KVM-instances (in combination with writeback-caching in
libvirt/KVM, as recommended). Even with moderate disk-access, it did
exactly what you described. Had to disable the caching again because of
this.
Using KVM 1.1.2 with libvirt 0.9.12, patched to auto-enable RBD-caching
on RBD with "cache=writeback", which is how I used it. AFAIK, this
patch made it into the later official versions. Haven't really started
debugging this yet.
The behavior you both are seeing is fixed by making flush requests
asynchronous in the qemu driver. This was fixed upstream in qemu 1.4.2
and 1.5.0. If you've installed from ceph-extras, make sure you're using
the .async rpms [1] (we should probably remove the non-async ones at
this point).
The cuttlefish qemu rpms should work fine with dumpling. They're only
separate from the bobtail ones to be able to use newer functions in
librbd.
Josh
[1]
http://www.ceph.com/packages/ceph-extras/rpm/centos6/x86_64/qemu-kvm-0.12.1.2-2.355.el6.2.cuttlefish.async.x86_64.rpm
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
