On Wed, Sep 19, 2012 at 1:52 PM, Gregory Farnum <greg@xxxxxxxxxxx> wrote: > On Wed, Sep 19, 2012 at 1:48 PM, Sage Weil <sage@xxxxxxxxxxx> wrote: >> On Wed, 19 Sep 2012, Tren Blackburn wrote: >>> Hey List; >>> >>> I'm in the process of rsyncing in about 7TB of data to Ceph across >>> approximately 58565475 files (okay, so I guess that's not so >>> approximate). It's only managed to copy a small portion of this so far >>> (about 35GB) and the server that currently is the mds master shows the >>> ceph-mds process closing in on 2GB of RAM used. I'm getting a little >>> nervous about this steep increase. >>> >>> fern ceph # ps wwaux | grep ceph-mds >>> root 29943 3.7 0.9 2473884 1915468 ? Ssl 08:42 11:22 >>> /usr/bin/ceph-mds -i 1 --pid-file /var/run/ceph/mds.1.pid -c >>> /etc/ceph/ceph.conf >>> >>> Thank in advance! >> >> That's not necessarily a problem. The mds memory usage is controlled via >> the 'mds cache size' knob which defaults to 100,000. You might try >> halving that and restarting the MDS. > > Last time we counted those inodes were ~1KB each, so getting 2GB of > memory used is an awful lot. > > Tren, what does your tree look like — are there very large directories? Greg: It's difficult to tell you that. I'm rsyncing 2 volumes from our filers. Each base directory on each filer mount has approximate 213 directories, and then each directory under that has approximately anywhere from 3000 - 5000 directories (very loose approximation here, 850,000 directories per filer mount), and then each of those directories contains files. We have many many files here. We're doing this to see how CephFS handles lots of files. We are coming from MooseFS which its master metalogger process eats lots of ram, so we're hoping that Ceph is a bit lighter on us. Sage: The memory the MDS is using is only a cache? There should be no problem restarting the MDS server while activity is going on? I should probably change the limit for the non-active MDS servers first, and then the active one and hope it fails over cleanly? Thank you both for your assistance! t. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
