Hi Alexander, On Mon, 24 Jul 2023 at 17:40, Naresh Kamboju <naresh.kamboju@xxxxxxxxxx> wrote: > > On Mon, 24 Jul 2023 at 15:50, Alexander Potapenko <glider@xxxxxxxxxx> wrote: > > > > On Sat, Jul 22, 2023 at 6:37 PM Linus Torvalds > > <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote: > > > > > > [ Removed the stable reviewers, bringing in the kfence people ] > > > > > > See > > > > > > https://lore.kernel.org/lkml/CA+G9fYvgy22wiY=c3wLOrCM6o33636abhtEynXhJkqxJh4ca0A@xxxxxxxxxxxxxx/ > > > > > > for the original report. The warning was introduced in 8f0b36497303 > > > ("mm: kfence: fix objcgs vector allocation"), and Google doesn't find > > > any other cases of this. > > > > > > Anybody? > > > > > > Linus > > > > > > > > > > > NOTE: > > > > The following kernel warning was noticed while booting qemu-arm64 > > > > with these configs enabled on stable rc 6.4.5-rc1. > > > > > > > > CONFIG_ARM64_64K_PAGES=y > > > > CONFIG_KFENCE=y > > > > Is there a full config somewhere? > > Please find build details > - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/ > - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/config > - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/vmlinux.xz > - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/System.map > - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/Image.gz > > > > > > > This crash is not easily reproducible. > > > > CONFIG_KFENCE_SAMPLE_INTERVAL=10 > > CONFIG_KFENCE_NUM_OBJECTS=2048 > > > > might improve reproducibility. > > The above test have following Kconfigs enabled. > > CONFIG_HAVE_ARCH_KASAN=y > CONFIG_HAVE_ARCH_KASAN_SW_TAGS=y > CONFIG_HAVE_ARCH_KASAN_HW_TAGS=y > CONFIG_HAVE_ARCH_KASAN_VMALLOC=y > CONFIG_CC_HAS_KASAN_GENERIC=y > CONFIG_CC_HAS_KASAN_SW_TAGS=y > CONFIG_CC_HAS_WORKING_NOSANITIZE_ADDRESS=y > # CONFIG_KASAN is not set > CONFIG_HAVE_ARCH_KFENCE=y > CONFIG_KFENCE=y > CONFIG_KFENCE_SAMPLE_INTERVAL=100 > CONFIG_KFENCE_NUM_OBJECTS=255 > # CONFIG_KFENCE_DEFERRABLE is not set > CONFIG_KFENCE_STRESS_TEST_FAULTS=0 > > > > > > > > > > > boot logs: > > > > -------- > > > > [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510] > > > > [ 0.000000] Linux version 6.4.5-rc1 (tuxmake@tuxmake) > > > > (aarch64-linux-gnu-gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils > > > > for Debian) 2.40) #1 SMP PREEMPT @1689957802 > > > > [ 0.000000] random: crng init done > > > > [ 0.000000] Machine model: linux,dummy-virt > > > > ... > > > > [ 0.006821] kfence: initialized - using 33554432 bytes for 255 > > > > objects at 0x(____ptrval____)-0x(____ptrval____) > > > > ... > > > > [ 7.726994] ------------[ cut here ]------------ > > > > [ 7.727704] WARNING: CPU: 1 PID: 1 at mm/kfence/core.c:1097 > > > > __kfence_free+0x84/0xc8 > > > ... > > > > [ 7.746478] Call trace: > > > > [ 7.746776] __kfence_free+0x84/0xc8 > > > > [ 7.747134] __slab_free+0x490/0x508 > > > > [ 7.748063] __kmem_cache_free+0x2b4/0x2d0 > > > > [ 7.748377] kfree+0x78/0x140 > > > > [ 7.748638] single_release+0x40/0x60 > > > > [ 7.750664] __fput+0x78/0x260 > > > > [ 7.751065] ____fput+0x18/0x30 > > > > [ 7.752086] task_work_run+0x80/0xe0 > > > > [ 7.753122] do_notify_resume+0x200/0x1398 > > > > [ 7.754292] el0_svc+0xec/0x100 > > > > [ 7.754573] el0t_64_sync_handler+0xf4/0x120 > > > > [ 7.755559] el0t_64_sync+0x190/0x198 > > > > It would be interesting to see the contents of > > /sys/kernel/debug/kfence/objects together with the object address. > > Would it be possible to boot the kernel with no_hash_pointers and add > > a line printing the object address: > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > index dad3c0eb70a01..23f27f6cb18cf 100644 > > --- a/mm/kfence/core.c > > +++ b/mm/kfence/core.c > > @@ -1094,7 +1094,10 @@ void __kfence_free(void *addr) > > struct kfence_metadata *meta = addr_to_metadata((unsigned long)addr); > > > > #ifdef CONFIG_MEMCG > > - KFENCE_WARN_ON(meta->objcg); > > + if (meta->objcg) { > > + pr_err("ADDR: %px\n", addr); > > + KFENCE_WARN_ON(1); > > + } > > #endif > > /* > > * If the objects of the cache are SLAB_TYPESAFE_BY_RCU, defer freeing > > > > , and then dump /sys/kernel/debug/kfence/objects? > > > > This testing is running on CI loops and however, I will try to reproduce > this locally. I have applied the above debug patch and tested in a loop but the issues did not reproduce yet. - Naresh
