Thanks for your prompt reply.
what is the symptom of "out-of-sync"?
The folder does not receive any updates for days, i.e. I don't see any new subfolders and files that are present on (actively writing) clients. It happened on two clients so far, each running different versions of ceph-fuse and OS/userspace. One of the clients is on the same switch as the MDS. It appears to me that both clients stopped receiving updates virtually simultaneously, i.e. they have the same (out-of-sync) view on the folder.
which version of ceph-mds/ceph-fuse are you using?
MDS: ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299) / CentOS Linux release 7.2.1511
client1: ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299) / CentOS Linux release 7.2.1511
client2: ceph version 0.94.5 (9764da52395923e0b32908d83a9f7304401fee43) / Fedora release 23 (Twenty Three)
you enable client debug by "--debug_client=20' option
Thanks. I've already remounted the clients, but once the issue is back I will do some debugging.
And last but not least, writing a file to the folder, i.e. touch test, triggers synchronization.
Kind regards,
Mykola
On Wed, Dec 30, 2015 at 7:49 AM, Yan, Zheng <ukernel@xxxxxxxxx> wrote:
On Wed, Dec 30, 2015 at 5:59 AM, Mykola Dvornik <mykola.dvornik@xxxxxxxxx> wrote:Hi guys, I have 16 OSD/1MON/1MDS ceph cluster serving CephFS. The FS is mounted on 11 clients using ceph-fuse. In some cases there are multiple ceph-fuse processes per client, each with its own '-r' option. The problem is that some of the clients get significantly out-of-sync. Flushing caches, 'touching' things, etc. does not help.what is the symptom of "out-of-sync"?ceph -s reports 'Client XXX failing to respond to cache pressure'. Although I've increased "mds_cache_size" to "1000000", perf dump mds still reports inodes exceeding the inode_max. I cannot confirm yet, but it appears to me that the out-of-sync issue started to appear since the mds_cache_size increase.which version of ceph-mds/ceph-fuse are you using?mds log does not have anything suspicious in it. So is there any way to debug ceph-fuse?you enable client debug by "--debug_client=20' option, youRegards, -- Mykola _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
