I presume that since QEMU 1.2+ sets the default cache mode to writeback if not otherwise specified, and since giant sets rbd_cache to true if not otherwise specified, then the result should be to cache? We have a fair number of VMs running on hosts where we don't specify either explicitly, and I've always had the feeling that its _not_ caching... saving a small text file in the VM (e.g. with vim) always seems to take much longer than it should - but I wonder if that's because vim is doing an fsync(). If I understand the QEMU docs correctly, cache=unsafe would immediately ack the guest's fsync() - at the risk of data losss if the QEMU process crashes. On 09/06/15 06:59, Alexandre DERUMIER wrote: > oops, sorry, my bad, I had wrong settings when testing. > > you are right, remove rbd_cache from ceph.conf is enough to remove overloading > > > > host conf : no value : guest cache=writeback : result : cache > host conf : rbd_cache=false : guest cache=writeback : result : nocache (wrong) > host conf : rbd_cache=true : guest cache=writeback : result : cache > host conf : no value : guest cache=none : result : nocache > host conf : rbd_cache=false : guest cache=none : result : no cache > host conf : rbd_cache=true : guest cache=none : result : cache (wrong) > > > ----- Mail original ----- > De: "aderumier" <aderumier@xxxxxxxxx> > À: "Jason Dillaman" <dillaman@xxxxxxxxxx> > Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> > Envoyé: Mardi 9 Juin 2015 06:33:49 > Objet: Re: rbd cache + libvirt > > previous matrix was with ceph < giant > > > with ceph =>giant, rbd_cache=true by default, so cache=none not working if a ceph.conf exist. > > > host conf : no value : guest cache=writeback : result : cache > host conf : rbd_cache=false : guest cache=writeback : result : nocache (wrong) > host conf : rbd_cache=true : guest cache=writeback : result : cache > host conf : no value : guest cache=none : result : cache (wrong) > host conf : rbd_cache=false : guest cache=none : result : no cache > host conf : rbd_cache=true : guest cache=none : result : cache (wrong) > > > ----- Mail original ----- > De: "aderumier" <aderumier@xxxxxxxxx> > À: "Jason Dillaman" <dillaman@xxxxxxxxxx> > Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> > Envoyé: Mardi 9 Juin 2015 06:23:06 > Objet: Re: rbd cache + libvirt > >>> In the short-term, you can remove the "rbd cache" setting from your ceph.conf > > That's not true, you need to remove the ceph.conf file. > Removing rbd_cache is not enough or default rbd_cache=false will apply. > > > I have done tests, here the result matrix > > > host ceph.conf : no rbd_cache : guest cache=writeback : result : nocache (wrong) > host ceph.conf : rbd_cache=false : guest cache=writeback : result : nocache (wrong) > host ceph.conf : rbd_cache=true : guest cache=writeback : result : cache > host ceph.conf : no rbd_cache : guest cache=none : result : nocache > host ceph.conf : rbd_cache=false : guest cache=none : result : no cache > host ceph.conf : rbd_cache=true : guest cache=none : result : cache (wrong) > > > > ----- Mail original ----- > De: "Jason Dillaman" <dillaman@xxxxxxxxxx> > À: "Andrey Korolyov" <andrey@xxxxxxx> > Cc: "Josh Durgin" <jdurgin@xxxxxxxxxx>, "aderumier" <aderumier@xxxxxxxxx>, "ceph-users" <ceph-users@xxxxxxxxxxxxxx> > Envoyé: Lundi 8 Juin 2015 22:29:10 > Objet: Re: rbd cache + libvirt > >> On Mon, Jun 8, 2015 at 10:43 PM, Josh Durgin <jdurgin@xxxxxxxxxx> wrote: >>> On 06/08/2015 11:19 AM, Alexandre DERUMIER wrote: >>>> >>>> Hi, >>>>>> >>>>>> looking at the latest version of QEMU, >>>> >>>> >>>> It's seem that it's was already this behaviour since the add of rbd_cache >>>> parsing in rbd.c by josh in 2012 >>>> >>>> >>>> http://git.qemu.org/?p=qemu.git;a=blobdiff;f=block/rbd.c;h=eebc3344620058322bb53ba8376af4a82388d277;hp=1280d66d3ca73e552642d7a60743a0e2ce05f664;hb=b11f38fcdf837c6ba1d4287b1c685eb3ae5351a8;hpb=166acf546f476d3594a1c1746dc265f1984c5c85 >>>> >>>> >>>> I'll do tests on my side tomorrow to be sure. >>> >>> >>> It seems like we should switch the order so ceph.conf is overridden by >>> qemu's cache settings. I don't remember a good reason to have it the >>> other way around. >>> >>> Josh >>> >> >> Erm, doesn`t this code *already* represent the right priorities? >> Cache=none setting should set a BDRV_O_NOCACHE which is effectively >> disabling cache in a mentioned snippet. >> > > Yes, the override is applied (correctly) based upon your QEMU cache settings. However, it then reads your configuration file and re-applies the "rbd_cache" setting based upon what is in the file (if it exists). So in the case where a configuration file has "rbd cache = true", the override of "rbd cache = false" derived from your QEMU cache setting would get wiped out. The long term solution would be to, as Josh noted, switch the order (so long as there wasn't a use-case for applying values in this order). In the short-term, you can remove the "rbd cache" setting from your ceph.conf so that QEMU controls it (i.e. it cannot get overridden when reading the configuration file) or use a different ceph.conf for a drive which requires different cache settings from the default configuration's settings. > > Jason > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |