On Tue, Jun 28, 2022 at 04:21:55PM -0700, Darrick J. Wong wrote:
> On Wed, Jun 29, 2022 at 08:17:57AM +1000, Dave Chinner wrote:
> > On Tue, Jun 28, 2022 at 02:18:24PM +0100, Matthew Wilcox wrote:
> > > On Tue, Jun 28, 2022 at 12:31:55PM +0100, Matthew Wilcox wrote:
> > > > On Tue, Jun 28, 2022 at 12:27:40PM +0100, Matthew Wilcox wrote:
> > > > > On Tue, Jun 28, 2022 at 05:31:20PM +1000, Dave Chinner wrote:
> > > > > > So using this technique, I've discovered that there's a dirty page
> > > > > > accounting leak that eventually results in fsx hanging in
> > > > > > balance_dirty_pages().
> > > > >
> > > > > Alas, I think this is only an accounting error, and not related to
> > > > > the problem(s) that Darrick & Zorro are seeing. I think what you're
> > > > > seeing is dirty pages being dropped at truncation without the
> > > > > appropriate accounting. ie this should be the fix:
> > > >
> > > > Argh, try one that actually compiles.
> > >
> > > ... that one's going to underflow the accounting. Maybe I shouldn't
> > > be writing code at 6am?
> > >
> > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > > index f7248002dad9..4eec6ee83e44 100644
> > > --- a/mm/huge_memory.c
> > > +++ b/mm/huge_memory.c
> > > @@ -18,6 +18,7 @@
> > > #include <linux/shrinker.h>
> > > #include <linux/mm_inline.h>
> > > #include <linux/swapops.h>
> > > +#include <linux/backing-dev.h>
> > > #include <linux/dax.h>
> > > #include <linux/khugepaged.h>
> > > #include <linux/freezer.h>
> > > @@ -2439,11 +2440,15 @@ static void __split_huge_page(struct page *page, struct list_head *list,
> > > __split_huge_page_tail(head, i, lruvec, list);
> > > /* Some pages can be beyond EOF: drop them from page cache */
> > > if (head[i].index >= end) {
> > > - ClearPageDirty(head + i);
> > > - __delete_from_page_cache(head + i, NULL);
> > > + struct folio *tail = page_folio(head + i);
> > > +
> > > if (shmem_mapping(head->mapping))
> > > shmem_uncharge(head->mapping->host, 1);
> > > - put_page(head + i);
> > > + else if (folio_test_clear_dirty(tail))
> > > + folio_account_cleaned(tail,
> > > + inode_to_wb(folio->mapping->host));
> > > + __filemap_remove_folio(tail, NULL);
> > > + folio_put(tail);
> > > } else if (!PageAnon(page)) {
> > > __xa_store(&head->mapping->i_pages, head[i].index,
> > > head + i, 0);
> > >
> >
> > Yup, that fixes the leak.
> >
> > Tested-by: Dave Chinner <dchinner@xxxxxxxxxx>
>
> Four hours of generic/522 running is long enough to conclude that this
> is likely the fix for my problem and migrate long soak testing to my
> main g/522 rig and:
>
> Tested-by: Darrick J. Wong <djwong@xxxxxxxxxx>
>
Just based on Willy's earlier comment.. what I would probably be a
little careful/curious about here is whether the accounting fix leads to
an indirect behavior change that does impact reproducibility of the
corruption problem. For example, does artificially escalated dirty page
tracking lead to increased reclaim/writeback activity than might
otherwise occur, and thus contend with the fs workload? Clearly it has
some impact based on Dave's balance_dirty_pages() problem reproducer,
but I don't know if it extends beyond that off the top of my head. That
might make some sense if the workload is fsx, since that doesn't
typically stress cache/memory usage the way a large fsstress workload or
something might.
So for example, interesting questions might be... Do your corruption
events happen to correspond with dirty page accounting crossing some
threshold based on available memory in your test environment? Does
reducing available memory affect reproducibility? Etc.
Brian
> --D
>
> > Cheers,
> >
> > Dave.
> > --
> > Dave Chinner
> > david@xxxxxxxxxxxxx
>
