On 03/19/2013 07:06 PM, Josh Durgin wrote:
On 03/18/2013 07:53 AM, Wolfgang Hennerbichler wrote:
On 03/13/2013 06:38 PM, Josh Durgin wrote:
Anyone seeing this problem, could you try the wip-rbd-cache-aio branch?
Hi,
just compiled and tested it out, unfortunately there's no big change:
ceph --version
ceph version 0.58-375-ga4a6075 (a4a60758b7a10d51419c1961bb12f26b87672fd5)
libvirt-config:
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<source protocol='rbd' name='rbd/debian-template:rbd_cache=1'>
<host name='10.32.1.177' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07'
function='0x0'/>
</disk>
within the VM:
cat /dev/zero > /bigfile
again trashes the virtual machine.
reading seems quick(er) though (i can't go back to bobtail without some
hassles on this machine (leveldb on monitor & such), so this is just a
guess):
dd if=/dev/vda of=/dev/null
... 82.0 MB/s
One thing that I've noticed is that the virsh start command returns very
quickly, this used to take longer on bobtail.
Thanks,
Josh
you're welcome. I'm really eager helping you out now, I've got a
testbed, if you apply a patch in git I can probably test within 24 hours.
Wolfgang
Thanks for trying it out. Yesterday I managed to reliably reproduce the
problem, but I'm still looking for the root cause. It seems there's
some subtle interaction causing the issue, since even when the cache is
write-through or flushes data immediately (i.e. not acting as a write
cache), the problem persists. I'll update
http://tracker.ceph.com/issues/3737 when I've got more information.
Josh
The issue was that the qemu rbd driver was blocking the main qemu
thread when flush was called, since it was using a synchronous flush.
Fixing this involves patches to librbd to add an asynchronous flush,
and a patch to qemu to use it.
The ceph patches are in the next and master branches, and will be
backported to bobtail. The patch for qemu is at [1] if anyone wants to
try it out.
Josh
[1] http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg05367.html
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
