On Thu, Jan 12, 2023 at 11:31:49AM +0100, Vlastimil Babka wrote: > On 12/26/22 09:44, Lorenzo Stoakes wrote: > > This brings mlock in line with the folio batches declared in mm/swap.c and > > makes the code more consistent across the two. > > > > The existing mechanism for identifying which operation each folio in the > > batch is undergoing is maintained, i.e. using the lower 2 bits of the > > struct folio address (previously struct page address). This should continue > > to function correctly as folios remain at least system word-aligned. > > > > All invoctions of mlock() pass either a non-compound page or the head of a > > THP-compound page and no tail pages need updating so this functionality > > works with struct folios being used internally rather than struct pages. > > > > In this patch the external interface is kept identical to before in order > > to maintain separation between patches in the series, using a rather > > awkward conversion from struct page to struct folio in relevant functions. > > > > However, this maintenance of the existing interface is intended to be > > temporary - the next patch in the series will update the interfaces to > > accept folios directly. > > > > Signed-off-by: Lorenzo Stoakes <lstoakes@xxxxxxxxx> > > Acked-by: Vlastimil Babka <vbabka@xxxxxxx> > > with some nits: > > > -static struct lruvec *__munlock_page(struct page *page, struct lruvec *lruvec) > > +static struct lruvec *__munlock_folio(struct folio *folio, struct lruvec *lruvec) > > { > > - int nr_pages = thp_nr_pages(page); > > + int nr_pages = folio_nr_pages(folio); > > bool isolated = false; > > > > - if (!TestClearPageLRU(page)) > > + if (!folio_test_clear_lru(folio)) > > goto munlock; > > > > isolated = true; > > - lruvec = folio_lruvec_relock_irq(page_folio(page), lruvec); > > + lruvec = folio_lruvec_relock_irq(folio, lruvec); > > > > - if (PageUnevictable(page)) { > > + if (folio_test_unevictable(folio)) { > > /* Then mlock_count is maintained, but might undercount */ > > - if (page->mlock_count) > > - page->mlock_count--; > > - if (page->mlock_count) > > + if (folio->mlock_count) > > + folio->mlock_count--; > > + if (folio->mlock_count) > > goto out; > > } > > /* else assume that was the last mlock: reclaim will fix it if not */ > > > > munlock: > > - if (TestClearPageMlocked(page)) { > > - __mod_zone_page_state(page_zone(page), NR_MLOCK, -nr_pages); > > - if (isolated || !PageUnevictable(page)) > > + if (folio_test_clear_mlocked(folio)) { > > + zone_stat_mod_folio(folio, NR_MLOCK, -nr_pages); > > AFAIK the 1:1 replacement would be __zone_stat_mod_folio(), this is stronger > thus not causing a bug, but unneccessary? I used this rather than __zone_stat_mod_folio() as this is what mlock_folio() does and I wanted to maintain consistency with that function. However, given we were previously user the weaker page version of this function, I agree that we should do the same with the folio, will change! > > > + if (isolated || !folio_test_unevictable(folio)) > > __count_vm_events(UNEVICTABLE_PGMUNLOCKED, nr_pages); > > else > > __count_vm_events(UNEVICTABLE_PGSTRANDED, nr_pages); > > } > > > > - /* page_evictable() has to be checked *after* clearing Mlocked */ > > - if (isolated && PageUnevictable(page) && page_evictable(page)) { > > - del_page_from_lru_list(page, lruvec); > > - ClearPageUnevictable(page); > > - add_page_to_lru_list(page, lruvec); > > + /* folio_evictable() has to be checked *after* clearing Mlocked */ > > + if (isolated && folio_test_unevictable(folio) && folio_evictable(folio)) { > > + lruvec_del_folio(lruvec, folio); > > + folio_clear_unevictable(folio); > > + lruvec_add_folio(lruvec, folio); > > __count_vm_events(UNEVICTABLE_PGRESCUED, nr_pages); > > } > > out: > > if (isolated) > > - SetPageLRU(page); > > + folio_set_lru(folio); > > return lruvec; > > } > > > > /* > > - * Flags held in the low bits of a struct page pointer on the mlock_pvec. > > + * Flags held in the low bits of a struct folio pointer on the mlock_fbatch. > > */ > > #define LRU_PAGE 0x1 > > #define NEW_PAGE 0x2 > > Should it be X_FOLIO now? > > > -static inline struct page *mlock_lru(struct page *page) > > +static inline struct folio *mlock_lru(struct folio *folio) > > { > > - return (struct page *)((unsigned long)page + LRU_PAGE); > > + return (struct folio *)((unsigned long)folio + LRU_PAGE); > > } > > > > -static inline struct page *mlock_new(struct page *page) > > +static inline struct folio *mlock_new(struct folio *folio) > > { > > - return (struct page *)((unsigned long)page + NEW_PAGE); > > + return (struct folio *)((unsigned long)folio + NEW_PAGE); > > } > > > > /* > > - * mlock_pagevec() is derived from pagevec_lru_move_fn(): > > - * perhaps that can make use of such page pointer flags in future, > > - * but for now just keep it for mlock. We could use three separate > > - * pagevecs instead, but one feels better (munlocking a full pagevec > > - * does not need to drain mlocking pagevecs first). > > + * mlock_folio_batch() is derived from folio_batch_move_lru(): perhaps that can > > + * make use of such page pointer flags in future, but for now just keep it for > > ^ folio? > > > + * mlock. We could use three separate folio batches instead, but one feels > > + * better (munlocking a full folio batch does not need to drain mlocking folio > > + * batches first). > > */ > > -static void mlock_pagevec(struct pagevec *pvec) > > +static void mlock_folio_batch(struct folio_batch *fbatch) > Ack on all remaining comments also, will spin a v4, thanks for the review!