On Tue, Nov 24, 2015 at 10:26 PM, Mike Miller <millermike287@xxxxxxxxx> wrote: > Hi, > > in my cluster with 16 OSD daemons and more than 20 million files on cephfs, > the memory usage on MDS is around 16 GB. It seems that 'mds cache size' has > no real influence on the memory usage of the MDS. > > Is there a formula that relates 'mds cache size' directly to memory > consumption on the MDS? The dominant factor should be the number of inodes in cache, although there are other things too. Depending on version I think it was ~2KB of memory for each inode+dentry at last count. > In the documentation (and other posts on the mailing list) it is said that > the MDS needs 1 GB per daemon. I am observing that the MDS uses almost > exactly 1 GB per OSD daemon (I have 16 OSD and 16 GB memory usage on the > MDS). Is this the correct formula? > > Or is it 1 GB per MDS daemon? It's got nothing to do with the number of OSDs. I'm not sure where 1GB per MDS came from, although you can certainly run a reasonable low-intensity cluster on that. > > In my case, the standard 'mds cache size 100000' makes MDS crash and/or the > cephfs is unresponsive. Larger values for 'mds cache size' seem to work > really well. Right. You need the total cache size of your MDS "cluster" (which is really just 1) to be larger than your working set size or you'll have trouble. Similarly if you have any individual directories which are a significant portion of your total cache it might cause issues. -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |