On Mon, Jun 10, 2024 at 11:36 AM Usama Arif <usamaarif642@xxxxxxxxx> wrote: > > > On 10/06/2024 18:57, Yosry Ahmed wrote: > > On Mon, Jun 10, 2024 at 5:18 AM Usama Arif <usamaarif642@xxxxxxxxx> wrote: > >> Approximately 10-20% of pages to be swapped out are zero pages [1]. > >> Rather than reading/writing these pages to flash resulting > >> in increased I/O and flash wear, a bitmap can be used to mark these > >> pages as zero at write time, and the pages can be filled at > >> read time if the bit corresponding to the page is set. > >> With this patch, NVMe writes in Meta server fleet decreased > >> by almost 10% with conventional swap setup (zswap disabled). > >> > >> [1]https://lore.kernel.org/all/20171018104832epcms5p1b2232e2236258de3d03d1344dde9fce0@epcms5p1/ > >> > >> Signed-off-by: Usama Arif <usamaarif642@xxxxxxxxx> > >> --- > >> include/linux/swap.h | 1 + > >> mm/page_io.c | 92 +++++++++++++++++++++++++++++++++++++++++++- > >> mm/swapfile.c | 21 +++++++++- > >> 3 files changed, 111 insertions(+), 3 deletions(-) > >> > >> diff --git a/include/linux/swap.h b/include/linux/swap.h > >> index a11c75e897ec..e88563978441 100644 > >> --- a/include/linux/swap.h > >> +++ b/include/linux/swap.h > >> @@ -299,6 +299,7 @@ struct swap_info_struct { > >> signed char type; /* strange name for an index */ > >> unsigned int max; /* extent of the swap_map */ > >> unsigned char *swap_map; /* vmalloc'ed array of usage counts */ > >> + unsigned long *zeromap; /* vmalloc'ed bitmap to track zero pages */ > >> struct swap_cluster_info *cluster_info; /* cluster info. Only for SSD */ > >> struct swap_cluster_list free_clusters; /* free clusters list */ > >> unsigned int lowest_bit; /* index of first free in swap_map */ > >> diff --git a/mm/page_io.c b/mm/page_io.c > >> index a360857cf75d..2cac1e11fb85 100644 > >> --- a/mm/page_io.c > >> +++ b/mm/page_io.c > >> @@ -172,6 +172,82 @@ int generic_swapfile_activate(struct swap_info_struct *sis, > >> goto out; > >> } > >> > >> +static bool is_folio_page_zero_filled(struct folio *folio, int i) > >> +{ > >> + unsigned long *data; > >> + unsigned int pos, last_pos = PAGE_SIZE / sizeof(*data) - 1; > >> + bool ret = false; > >> + > >> + data = kmap_local_folio(folio, i * PAGE_SIZE); > >> + if (data[last_pos]) > >> + goto out; > >> + for (pos = 0; pos < PAGE_SIZE / sizeof(*data); pos++) { > >> + if (data[pos]) > >> + goto out; > >> + } > >> + ret = true; > >> +out: > >> + kunmap_local(data); > >> + return ret; > >> +} > >> + > >> +static bool is_folio_zero_filled(struct folio *folio) > >> +{ > >> + unsigned int i; > >> + > >> + for (i = 0; i < folio_nr_pages(folio); i++) { > >> + if (!is_folio_page_zero_filled(folio, i)) > >> + return false; > >> + } > >> + return true; > >> +} > > Is there any benefit to iterating on the folio in pages (i.e. have > > both is_folio_zero_filled() and is_folio_page_zero_filled())? Why > > don't we just kmap the entire folio and check it all at once? > > Is there an API to kmap an entire folio? I thought kmap_local_folio() takes in a 'size' parameter for some reason, my bad. > > I could just do data = page_address(&folio->page) in above and iterate > through folio_nr_pages(folio) * PAGE_SIZE, and do it all in one > function, but this just looks much nicer and much more readable? Yeah if we need to map each page individually, the current code is better. > > In other places in the kernel, the folio iteration is through pages as well: > > https://elixir.bootlin.com/linux/latest/source/arch/csky/abiv2/cacheflush.c#L29 > > https://elixir.bootlin.com/linux/latest/source/arch/mips/mm/cache.c#L162 > > and others as well. [..] > >> @@ -515,8 +600,11 @@ void swap_read_folio(struct folio *folio, bool synchronous, > >> psi_memstall_enter(&pflags); > >> } > >> delayacct_swapin_start(); > >> - > >> - if (zswap_load(folio)) { > >> + if (swap_zeromap_folio_test(folio)) { > >> + folio_zero_fill(folio); > >> + folio_mark_uptodate(folio); > >> + folio_unlock(folio); > > We don't currently support swapping in large folios, but it is a work > > in progress, and this will break once we have it. > > swap_zeromap_folio_test() will return false even if parts of the folio > > are in fact zero-filled. Then, we will go read those from disk swap, > > essentially corrupting data. > > So yes, with this patch I tested swap out of large zero folio, but when > swapping in it was page by page. My assumption was that swap_read_folio > (when support is added) would only pass a large folio that was earlier > swapped out as a large folio. So if a zero filled large folio was > swapped out, the zeromap will be set for all the pages in that folio, > and at large folio swap in (when it is supported), it will see that all > the bits in the zeromap for that folio are set, and will just > folio_zero_fill. > > If even a single page in large folio has non-zero data then zeromap will > not store it and it will go to either zswap or disk, and at read time as > all the bits in zeromap are not set, it will still goto either zswap or > disk, so I think this works? > > Is my assumption wrong that only large folios can be swapped in only if > they were swapped out as large? I think this code works in that case. I think the assumption is incorrect. I think we would just check if contiguous PTEs have contiguous swap entries and swapin the folio as a large folio in this case. It is likely that the swap entries are contiguous because it was swapped out as a large folio, but I don't think it's guaranteed. For example, here is a patch that implements large swapin support for the synchronous swapin case, and I think it just checks that the PTEs have contiguous swap entries: https://lore.kernel.org/linux-mm/20240304081348.197341-6-21cnbao@xxxxxxxxx/ This makes a lot of sense because otherwise you'd have to keep track of how the folios were composed at the time of swapout, to be able to swap the same folios back in.
