On Tue, Feb 27, 2024 at 02:59:33PM -0800, Paul E. McKenney wrote: > On Tue, Feb 27, 2024 at 09:19:47PM +0200, Amir Goldstein wrote: > > On Tue, Feb 27, 2024 at 8:56 PM Paul E. McKenney <paulmck@xxxxxxxxxx> wrote: > > > > > > Hello! > > > > > > Recent discussions [1] suggest that greater mutual understanding between > > > memory reclaim on the one hand and RCU on the other might be in order. > > > > > > One possibility would be an open discussion. If it would help, I would > > > be happy to describe how RCU reacts and responds to heavy load, along with > > > some ways that RCU's reactions and responses could be enhanced if needed. > > > > Adding fsdevel as this should probably be a cross track session. > > The more, the merrier! ;-) Starting to look at the practicalities of freeing pages via rcu. Where to stick the rcu_head might be a problem. I think by the time we're in free_unref_page() enough of struct page isn't in use that we can just... throw it in the struct page union somewhere (ok, actually there's one already for ZONE_DEVICE pages; we just have to make sure it doesn't collide with anything - probably just compound page metadata). But it'd be more efficient if we weren't using linked lists for this. It's premature to ask for this before I've tested or profiled anything, but I just wanted to throw it out there that we might want to look using a dequeue for pending frees (and perhaps other things?). Willy and I were just chatting about this and I think the conclusion we're coming to is that we'll probably want to try just RCU freeing _everything_ at first, the reason being that anonymous pages are probably going to get included and that may end up being most of the allocations and frees - so why not just do everything. Meaning - _lots _of memory cycling through RCU, not bounded by device bandwidth like I initially thought. Another relevant topic: https://kernelnewbies.org/MatthewWilcox/Memdescs The plan has been to eventually get an actual enum/type tag for our different page types, and that would be really helpful for system visibility/introspection in general, and in particular for this project where we're going to want to be able to put numbers on different things for performance analysis and then later probably excluding things from RCU freeing in a sane standard way (slab, at least). I wonder if anyone wants to solve this. The problem with doing it now is that we've got no more page flags on 32 bit, but it'd be _very_ helpful in future cleanups if we could figure out a way to make it hapen. That'd help with another wrinkle, which is non compound higher order pages. Those are tricky to handle, because how do we know the order from the RCU callback? If we just don't RCU free them, that's sketchy without the page type enum because we have no way to have a master definition/list of "these types of pages are RCU freed, these aren't". Possibly we can just fake-compound them - store the order like it is in compound pages but don't initialize every page like happens for compound pages.
