Re: cephfs mds millions of caps

[Webert de Souza Lima <webert.boss@xxxxxxxxx>]

I have upgraded the kernel on a client node (one that has close-to-zero traffic) used for tests.

   {

      "reconnecting" : false,

      "id" : 1620266,

      "num_leases" : 0,

      "inst" : "client.1620266 10.0.0.111:0/3921220890",

      "state" : "open",

      "completed_requests" : 0,

      "num_caps" : 1402490,

      "client_metadata" : {

         "kernel_version" : "4.4.0-104-generic",

         "hostname" : "suppressed",

         "entity_id" : "admin"

      },

      "replay_requests" : 0

   },

still 1.4M caps used.

is upgrading the client kernel enough ?

Regards,

Webert Lima

DevOps Engineer at MAV Tecnologia

Belo Horizonte - Brasil

IRC NICK - WebertRLZ

On Fri, Dec 15, 2017 at 11:16 AM, Webert de Souza Lima <webert.boss@xxxxxxxxx> wrote:

So,

On Fri, Dec 15, 2017 at 10:58 AM, Yan, Zheng <ukernel@xxxxxxxxx> wrote:

300k are ready quite a lot. opening them requires long time. does you
mail server really open so many files?

Yes, probably. It's a commercial solution. A few thousand domains, dozens of thousands of users and god knows how any mailboxes.

From the daemonperf you can see the write workload is high, so yes, too much files opening (dovecot mdbox stores multiple e-mails per file, split into many files).

I checked 4.4 kernel, it includes the code that trim cache when mds recovers.

Ok, all nodes are running 4.4.0-75-generic. The fix might have been included in a newer version.
I'll upgrade it asap.

 

Regards,

Webert Lima

DevOps Engineer at MAV Tecnologia

Belo Horizonte - Brasil

IRC NICK - WebertRLZ

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com