This rbd_cache is only applicable to librbd not with the kernel rbd. Hope you are trying with librbd based env. If not, then the caching effect you are seeing is the filesystem cache. Thanks & Regards Somnath -----Original Message----- From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Bruce McFarland Sent: Monday, February 02, 2015 3:39 PM To: Mykola Golub; Udo Lembke Cc: ceph-users@xxxxxxxx; Prashanth Nednoor Subject: Re: RBD caching on 4K reads??? I'm still missing something. I can check on the monitor to see that the running config on the cluster has rbd cache = false [root@essperf13 ceph]# ceph --admin-daemon /var/run/ceph/ceph-mon.essperf13.asok config show | grep rbd "debug_rbd": "0\/5", "rbd_cache": "false", Since rbd caching is a client setting I've added the following to the rbd client /etc/ceph/ceph.conf [global] log file = /var/log/ceph/rbd.log rbd cache = false rbd readahead max bytes = 0 "should be" disabled if rbd cache = false, but I'm paranoid [client] admin socket = /var/run/ceph/rbd-$pid.asok I never see a rbd*asok file in /var/run/ceph. I started the rbd driver on the client without /var/run/ceph directory and then see: 2015-02-02 14:40:30.254509 7f81888257c0 -1 asok(0x7f8189182390) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/rbd-1716.asok': (2) No such file or directory When I attempt to map the rbd image to the client device with "rbd map". Once I create /var/run/ceph these messages don't occur. So it appears that the admin sockets are being created, but only for the duration of the command. I still see the effects of rbd caching if I run fio/vdbench with 4K random reads, but I have not been able to create a persistant rbd admin socket so that I can dump the running configuration and/or change it at run time. Any ideas on what I've overlooked? Any pointers to documentation on the [client] section of ceph.conf? rbd admin sockets? Nothing at ceph.com/docs on either topic. Thanks, Bruce -----Original Message----- From: Mykola Golub [mailto:to.my.trociny@xxxxxxxxx] Sent: Sunday, February 01, 2015 1:24 PM To: Udo Lembke Cc: Bruce McFarland; ceph-users@xxxxxxxx; Prashanth Nednoor Subject: Re: RBD caching on 4K reads??? On Fri, Jan 30, 2015 at 10:09:32PM +0100, Udo Lembke wrote: > Hi Bruce, > you can also look on the mon, like > ceph --admin-daemon /var/run/ceph/ceph-mon.b.asok config show | grep > cache rbd cache is a client setting, so you have to check this connecting to the client admin socket. Its location is defined in ceph.conf, [client] section, "admin socket" parameter. -- Mykola Golub _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |