Hi,
On 09/14/2016 12:43 PM, Dennis Kramer (DT) wrote:
Hi Goncalo,
Thank you. Yes, i have seen that thread, but I have no near full osds
and my mds cache size is pretty high.
You can use the daemon socket on the mds server to get an overview of
the current cache state:
ceph daemon mds.XXX perf dump
The message itself indicates that the mds is in fact trying to convince
clients to release capabilities, probably because it is running out of
cache.
The 'session ls' command on the daemon socket lists all current ceph
clients and the number capabilities for each client. Depending on your
workload / applications you might be surprised how many capabilities are
assigned to individual nodes...
From the client side of view the error means that there's either a bug
in the client, or an application is keeping a large number of files open
(e.g. do you run mlocate on the clients?)
If you use the kernel based client re-mounting won't help, since the
internal state is keep the same (afaik). In case of the ceph-fuse client
the ugly way to get rid off the mount point is a lazy / forced umount
and killing the ceph-fuse process if necessary. Processes with open file
handles will complain afterwards.
Before using rude ways to terminate the client session i would propose
to look for rogue applications on the involved host. We had a number of
problems with multithreaded applications and concurrent file access on
the past (both with ceph-fuse from hammer and kernel based clients).
lsof or other tools might help locating the application.
Regards,
Burkhard
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
