==============================
State Of The Page, August 2022
==============================
I thought I'd write down where we are with struct page and where
we're going, just to make sure we're all (still?) pulling in a similar
direction.
Destination
===========
For some users, the size of struct page is simply too large. At 64
bytes per 4KiB page, memmap occupies 1.6% of memory. If we can get
struct page down to an 8 byte tagged pointer, it will be 0.2% of memory,
which is an acceptable overhead.
struct page {
unsigned long mem_desc;
};
Types of memdesc
----------------
This is very much subject to change as new users present themselves.
Here are the current ones in-plan:
- Undescribed. Instead of the rest of the word being a pointer,
there are 2^28 subtypes available:
- Unmappable. Typically device drivers allocating private memory.
- Reserved. These pages are not allocatable.
- HWPoison
- Offline (eg balloon)
- Guard (see debug_pagealloc)
- Slab
- Anon Folio
- File Folio
- Buddy (ie free -- also for PCP?)
- Page Table
- Vmalloc
- Net Pool
- Zsmalloc
- Z3Fold
- Mappable. Typically device drivers mapping memory to userspace
That implies 4 bits needed for the tag, so all memdesc allocations
must be 16-byte aligned. That is not an undue burden. Memdescs
must also be TYPESAFE_BY_RCU if they are mappable to userspace or
can be stored in a file's address_space.
It may be worth distinguishing between vmalloc-mappable and
vmalloc-unmappable to prevent some things being mapped to userspace
inadvertently.
Contents of a memdesc
---------------------
At least initially, the first word of a memdesc must be identical to the
current page flags. That allows various functions (eg set_page_dirty())
to work on any kind of page without needing to know whether it's a device
driver page, a vmalloc page, anon or file folio.
Similarly, both anon and file folios must have the list_head in the
same place so they can be placed on the same LRU list. Whether anon
and file folios become separate types is still unclear to me.
Mappable
--------
All pages mapped to userspace must have:
- A refcount
- A mapcount
Preferably in the same place in the memdesc so we can handle them without
having separate cases for each type of memdesc. It would be nice to have
a pincount as well, but that's already an optional feature.
I propose:
struct mappable {
unsigned long flags; /* contains dirty flag */
atomic_t _refcount;
atomic_t _mapcount;
};
struct folio {
union {
unsigned long flags;
struct mappable m;
};
...
};
Memdescs which should never be mapped to userspace (eg slab, page tables,
zsmalloc) do not need to contain such a struct.
Mapcount
--------
While discussed above, handling mapcount is tricky enough to need its
own section. Since folios can be mapped unaligned, we may need to
increment mapcount once per page table entry that refers to it. This
is different from how THPs are handled today (one refcount per page
plus a compound_mapcount for how many times the entire THP is mapped).
So splitting a PMD entry results in incrementing mapcount by
(PTRS_PER_PMD - 1).
If the mapcount is raised to dangerously high levels, we can split
the page. This should not happen in normal operation.
Extended Memdescs
-----------------
One of the things we're considering is that maybe a filesystem will
want to have private data allocated with its folios. Instead of hanging
extra stuff off folio->private, they could embed a struct folio inside
a struct ext4_folio.
Buddy memdesc
-------------
I need to firm up a plan for this. Allocating memory in order to free
memory is generally a bad idea, so we either have to coopt the contents
of other memdescs (and some allocations don't have memdescs!) or we
need to store everything we need in the remainder of the unsigned long.
I'm not yet familiar enough with the page allocator to have a clear
picture of what is needed.
Where are we?
=============
v5.17:
- Slab was broken out from struct page in 5.17 (thanks to Vlastimil).
- XFS & iomap mostly converted from pages to folios
- Block & page cache mostly have the folio interfaces in place
v5.18:
- Large folio (multiple page) support added for filesystems that opt in
- File truncation converted to folios
- address_space_operations (aops) ->set_page_dirty converted to ->dirty_folio
- Much of get_user_page() converted to folios
- rmap_walk() converted to folios
v5.19:
- Most aops now converted to folios
- More folio conversions in migration, shmem, swap, vmscan
v6.0:
- aops->migratepage became migrate_folio
- isolate_page and putback_page removed from aops
- More folio conversions in migration, shmem, swap, vmscan
Todo
====
Well, most of the above!
- Individual filesystems need converting from pages to folios
- Zsmalloc, z3fold, page tables, netpools need to be split from
struct page into their own types
- Anywhere referring to page->... needs to be converted to folio
or some other type.
Help with any of this gratefully appreciated. Especially if you're the
maintainer of a thing and want to convert it yourself. I'd rather help
explain the subtleties of folios / mappables / ... to you than try
to figure out the details of your code to convert it myself (and get
it wrong). Please contact me to avoid multiple people working on
the same thing.
