On Sat, 2024-10-12 at 16:50 -0700, Yang Shi wrote:
> On Sat, Oct 12, 2024 at 1:24 PM Avi Kivity <avi@xxxxxxxxxxxx> wrote:
> >
> > On Sat, 2024-10-12 at 13:05 -0700, Yang Shi wrote:
> > > On Sat, Oct 12, 2024 at 8:38 AM Avi Kivity <avi@xxxxxxxxxxxx>
> > > wrote:
> > > >
> > > > On Fri, 2024-10-11 at 15:29 -0700, Yang Shi wrote:
> > > > > On Wed, Oct 9, 2024 at 9:04 AM Avi Kivity <avi@xxxxxxxxxxxx>
> > > > > wrote:
> > > > > >
> > > > > > On Linux 6.10.10 with CONFIG_READ_ONLY_THP_FOR_FS=y,
> > > > > > madvise(MADV_COLLAPSE) on program text fails with EINVAL.
> > > > > >
> > > > > > To reproduce, compile the reproducer with
> > > > > >
> > > > > > clang -g -o text-hugepage text-hugepage.c \
> > > > > > -fuse-ld=lld \
> > > > > > -Wl,-zcommon-page-size=2097152 -Wl,-zmax-page-
> > > > > > size=2097152
> > > > > > \
> > > > > > -Wl,-z,separate-loadable-segments
> > > > > >
> > > > > > and run:
> > > > >
> > > > > Didn't clang make the page cache dirty?
> > > > >
> > > > > Having sync between clang and the execution made the problem
> > > > > go
> > > > > away
> > > > > for me.
> > > > >
> > > >
> > > > I see it even with sync (and msync just before the madvise
> > > > calls).
> > >
> > > Did you stop khugepaged? It may race with MADV_COLLAPSE. If it
> > > failed
> > > due to race with khugepaged, you should see -EAGAIN instead of
> > > -EINVAL.
> >
> >
> > I did not, but I don't imagine I hit the race in all my attempts.
> >
> > >
> > > I did the below commands in a loop for 1000 times, it never
> > > failed (I
> > > modified the test program a little bit to print out failure if
> > > MADV_COLLAPSE returns failure). I had khugepaged stopped and ran
> > > the
> > > test on v6.12-rc1 kernel on my AmpereOne machine.
> > >
> > > rm text-hugepage
> > > clang -g -o text-hugepage text-hugepage.c -fuse-ld=lld
> > > -Wl,-zcommon-page-size=2097152 -Wl,-zmax-page-size=2097152
> > > -Wl,-z,separate-loadable-segments
> > > sync
> > > ./text-hugepage
> > >
> > > >
> > > >
> > > > Tracing shows this (last lines before syscall exit):
> > > >
> > > > > hpage_collapse_scan_file() {
> > > > > __rcu_read_lock();
> > > > > __rcu_read_unlock();
> > > > > }
> > >
> > > It meant collapse_file() was not called at all.
> > > hpage_collapse_scan_file() failed. A couple of reasons may fail
> > > it,
> > > for example, refcount is not expected, not on lru, etc. You can
> > > trace
> > > huge_memory:mm_khugepaged_scan_file to get more information about
> > > the
> > > failure.
> >
> >
> > text-hugepage-689146 [023] 200457.073794:
> > mm_khugepaged_scan_file:
> > mm=0xffff92fc512aac00, scan_pfn=0x5a4310, filename=text-hugepage,
> > present=0, swap=0, result=page_compound
>
> Aha, it is because v6.10 doesn't support collapse non-PMD order large
> folios. It has been fixed in v6.12-rc1. The patch series is:
> https://lore.kernel.org/all/cover.1724140601.git.baolin.wang@xxxxxxxx
> ibaba.com/
>
> The subject says "shmem", but it actually works for regular files
> too.
Thanks a lot. I will retest when 6.12 reaches Fedora testing.
