Re: [LSF/MM TOPIC] Better handling of negative dentries

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Mar 16, 2022 at 01:52:23PM +1100, Dave Chinner wrote:
> On Wed, Mar 16, 2022 at 10:07:19AM +0800, Gao Xiang wrote:
> > On Tue, Mar 15, 2022 at 01:56:18PM -0700, Roman Gushchin wrote:
> > > 
> > > > On Mar 15, 2022, at 12:56 PM, Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:
> > > > 
> > > > The number of negative dentries is effectively constrained only by memory
> > > > size.  Systems which do not experience significant memory pressure for
> > > > an extended period can build up millions of negative dentries which
> > > > clog the dcache.  That can have different symptoms, such as inotify
> > > > taking a long time [1], high memory usage [2] and even just poor lookup
> > > > performance [3].  We've also seen problems with cgroups being pinned
> > > > by negative dentries, though I think we now reparent those dentries to
> > > > their parent cgroup instead.
> > > 
> > > Yes, it should be fixed already.
> > > 
> > > > 
> > > > We don't have a really good solution yet, and maybe some focused
> > > > brainstorming on the problem would lead to something that actually works.
> > > 
> > > I’d be happy to join this discussion. And in my opinion it’s going beyond negative dentries: there are other types of objects which tend to grow beyond any reasonable limits if there is no memory pressure.
> > 
> > +1, we once had a similar issue as well, and agree that is not only
> > limited to negative dentries but all too many LRU-ed dentries and inodes.
> 
> Yup, any discussion solely about managing buildup of negative
> dentries doesn't acknowledge that it is just a symptom of larger
> problems that need to be addressed.

Yes, but let's not make this _so_ broad a discussion that it becomes
unsolvable.  Rather, let's look for a solution to this particular problem
that can be adopted by other caches that share a similar problem.

For example, we might be seduced into saying "this is a slab problem"
because all the instances we have here allocate from slab.  But slab
doesn't have enough information to solve the problem.  Maybe the working
set of the current workload really needs 6 million dentries to perform
optimally.  Maybe it needs 600.  Slab can't know that.  Maybe slab can
play a role here, but the only component which can know the appropriate
size for a cache is the cache itself.

I think the logic needs to be in d_alloc().  Before it calls __d_alloc(),
it should check ... something ... to see if it should try to shrink
the LRU list.  The devil is in what that something should be.  I'm no
expert on the dcache; do we just want to call prune_dcache_sb() for
every 1/1000 time?  Rely on DCACHE_REFERENCED to make sure that we're
not over-pruning the list?  If so, what do we set nr_to_scan to?  1000 so
that we try to keep the dentry list the same size?  1500 so that it
actually tries to shrink?

I don't feel like I know enough to go further here.  But it feels better
than what we currently do -- calling all the shrinkers from deep in
the page allocator.



[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux