On Tue, May 11, 2021 at 10:47:02PM +0100, Matthew Wilcox (Oracle) wrote:
> We also waste a lot of instructions ensuring that we're not looking at
> a tail page. Almost every call to PageFoo() contains one or more hidden
> calls to compound_head(). This also happens for get_page(), put_page()
> and many more functions. There does not appear to be a way to tell gcc
> that it can cache the result of compound_head(), nor is there a way to
> tell it that compound_head() is idempotent.
I instrumented _compound_head() on a test VM:
+++ b/include/linux/page-flags.h
@@ -179,10 +179,13 @@ enum pageflags {
#ifndef __GENERATING_BOUNDS_H
+extern atomic_t chcc;
+
static inline unsigned long _compound_head(const struct page *page)
{
unsigned long head = READ_ONCE(page->compound_head);
+ atomic_inc(&chcc);
if (unlikely(head & 1))
return head - 1;
return (unsigned long)page;
which means it catches both calls to compound_head() and page_folio().
Between patch 8/96 in folio_v9 and patch 96/96, the number of calls in
an idle VM went down from almost 7k/s to just over 5k/s; about 25%.
