On 5 Mar 2025, at 17:38, Hugh Dickins wrote: > On Wed, 5 Mar 2025, Zi Yan wrote: >> On 5 Mar 2025, at 16:03, Hugh Dickins wrote: >>> >>> Beyond checking that, I didn't have time yesterday to investigate >>> further, but I'll try again today (still using last weekend's mm.git). >> >> I am trying to replicate your runs locally. Can you clarify your steps >> of “kernel builds on huge tmpfs while swapping to SSD”? Do you impose >> a memory limit so that anonymous memory is swapped to SSD or make tmpfs >> swap to SSD? > > Yeah, my heart sank a bit when I saw Andrew (with good intention) asking > you to repeat my testing. > > We could spend weeks going back and forth on that, and neither of us has > weeks to spare. > > "To fulfil contractual obligations" I'll mail you the tarfile I send > out each time I'm asked for this; but I haven't updated that tarfile > in four years, whereas I'm frequently tweaking things to match what's > needed (most recently and relevantly, I guess enabling 64kB hugepages > for anon and shmem in addition to the PMD-sized). > > Please don't waste much of your time over trying to replicate what > I'm doing: just give the scripts a glance, as a source for "oh, > I could exercise something like that in my testing too" ideas. > > Yes, I limit physical memory by booting with mem=1G, and also apply > lower memcg v1 limits. > > I made a point of saying "SSD" there because I'm not testing zram or > zswap at all, whereas many others are testing those rather than disk. > > swapoff, and ext4 on loop0 on tmpfs, feature in what I exercise, but are > NOT relevant to the corruption I'm seeing here - that can occur before > any swapoff, and it's always on the kernel build in tmpfs: the parallel > build in ext4 on loop0 on tmpfs completes successfully. Thanks for the scripts. I kinda replicate your setup as follows: 1. boot a VM with 1GB memory and 8 cores; 2. mount a tmpfs with huge=always and 200GB; 3. clone the mainline kernel and use x86_64 defconfig (my gcc 14 gives errors during the old kernel builds), this takes about 2GB space, so some of tmpfs is already swapped to SSD; 4. create a new cgroupv2 and set memory.high to 700MB to induce memory swap during kernel compilation; 5. run “while true; do echo 1 | sudo tee /proc/sys/vm/compact_memory >/dev/null; done” to trigger compaction all the time; 6. build the kernel with make -j20. I ran the above on mm-everything-2025-03-05-03-54 plus the xarray fix v3, folio_split() with your fixes, and Minimize xa_node allocation during xarry split patches. The repo is at: https://github.com/x-y-z/linux-dev/tree/shmem_fix-mm-everything-2025-03-05-03-54. It has ran over night for 30 kernel builds and no crash happened so far. I wonder if you can give my repo a shot. I just boosted khugepaged like you did and see no immediate crash. But I will let it run for longer. Thanks. Best Regards, Yan, Zi
