On Wed, Jun 07, 2023 at 08:35:05PM -0700, Hugh Dickins wrote:
> My current thinking (but may be proved wrong) is along the lines of:
> why does something on its way to being freed need to be on any list
> than the rcu_head list? I expect the current answer is, that the
> other half is allocated, so the page won't be freed; but I hope that
> we can put it back on that list once we're through with the rcu_head.
I was having the same thought. It is pretty tricky, but if this was
made into some core helper then PPC and S390 could both use it and PPC
would get a nice upgrade to have the S390 frag re-use instead of
leaking frags.
Broadly we have three states:
all frags free
at least one frag free
all frags used
'all frags free' should be returned to the allocator
'at least one frag free' should have the struct page on the mmu_struct's list
'all frags used' should be on no list.
So if we go from
all frags used -> at least one frag free
Then we put it on the RCU then the RCU puts it on the mmu_struct list
If we go from
at least one frag free -> all frags free
Then we take it off the mmu_struct list, put it on the RCU, and RCU
frees it.
Your trick to put the list_head for the mm_struct list into the frag
memory looks like the right direction. So 'at least one frag free' has
a single already RCU free'd frag hold the list head pointer. Thus we
never use the LRU and the rcu_head is always available.
The struct page itself can contain the actual free frag bitmask.
I think if we split up the memory used for pt_frag_refcount we can get
enough bits to keep track of everything. With only 2-4 frags we should
be OK.
So we track this data in the struct page:
- Current RCU free TODO bitmask - if non-zero then a RCU is already
triggered
- Next RCU TODO bitmaks - If an RCU is already triggrered then we
accumulate more free'd frags here
- Current Free Bits - Only updated by the RCU callback
?
We'd also need to store the mmu_struct pointer in the struct page for
the RCU to be able to add/remove from the mm_struct list.
I'm not sure how much of the work can be done with atomics and how
much would need to rely on spinlock inside the mm_struct.
It feels feasible and not so bad. :)
Figure it out and test it on S390 then make power use the same common
code, and we get full RCU page table freeing using a reliable rcu_head
on both of these previously troublesome architectures :) Yay
Jason
