On Thu, Sep 12, 2024 at 01:36:45PM +0200, Mateusz Guzik wrote: > I have to note what to do with a dentry after unlink is merely a subset > of the general problem of what to do about negative entries. I had a > look at it $elsewhere some years back and as one might suspect userspace > likes to do counterproductive shit. For example it is going to stat a > non-existent path 2-3 times and then open(..., O_CREAT) on it. > > I don't have numbers handy and someone(tm) will need to re-evaluate, but > crux of the findings was as follows: > - there is a small subset of negative entries which keep getting tons of > hits > - a sizeable count literally does not get any hits after being created > (aka wastes memory) > - some negative entries get 2-3 hits and get converted into a positive > entry afterwards (see that stat shitter) > - some flip flop with deletion/creation > > So whatever magic mechanism, if it wants to mostly not get in the way in > terms of performance, will have to account for the above. > > I ended up with a kludge where negative entries hang out on some number > of LRU lists and get promoted to a hot list if they manage to get some > number of hits. The hot list is merely a FIFO and entries there no > longer count any hits. Removal from the cold LRU also demotes an entry > from the hot list. This all reminds me of that paper you pointed me at. https://arxiv.org/pdf/1512.00727 Summary for the impatient: Use 10% of the memory for a "not yet proven to be useful" entries, and use multiple Bloom filters to decide which ones are sufficiently useful to be added to the "more permanent" cache. I don't think it solves every problem our current dcache implementation has, but I'm 90% sure it'll be a huge improvement.
