On 21.11.19 12:17, Christian Borntraeger wrote: > Folks, > > I do get errors like the following when running a new testcase in our KVM CI. > The test basically unloads kvm, reloads with with hpage=1 (enable huge page > support for guests on s390) start a guest with libvirt and hugepages, shut the > guest down and unload the kvm module. It also crashes without large pages. The trigger is really that the time between "guest is going away" and rmmod kvm is really short. i > > WARNING: CPU: 8 PID: 208 at lib/percpu-refcount.c:108 percpu_ref_exit+0x50/0x58 > Modules linked in: kvm(-) xt_CHECKSUM xt_MASQUERADE bonding xt_tcpudp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ip6table_na> > CPU: 8 PID: 208 Comm: kworker/8:1 Not tainted 5.2.0+ #66 > Hardware name: IBM 2964 NC9 712 (LPAR) > Workqueue: events sysfs_slab_remove_workfn > Krnl PSW : 0704e00180000000 0000001529746850 (percpu_ref_exit+0x50/0x58) > R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 > Krnl GPRS: 00000000ffff8808 0000001529746740 000003f4e30e8e18 0036008100000000 > 0000001f00000000 0035008100000000 0000001fb3573ab8 0000000000000000 > 0000001fbdb6de00 0000000000000000 0000001529f01328 0000001fb3573b00 > 0000001fbb27e000 0000001fbdb69300 000003e009263d00 000003e009263cd0 > Krnl Code: 0000001529746842: f0a0000407fe srp 4(11,%r0),2046,0 > 0000001529746848: 47000700 bc 0,1792 > #000000152974684c: a7f40001 brc 15,152974684e > >0000001529746850: a7f4fff2 brc 15,1529746834 > 0000001529746854: 0707 bcr 0,%r7 > 0000001529746856: 0707 bcr 0,%r7 > 0000001529746858: eb8ff0580024 stmg %r8,%r15,88(%r15) > 000000152974685e: a738ffff lhi %r3,-1 > Call Trace: > ([<000003e009263d00>] 0x3e009263d00) > [<00000015293252ea>] slab_kmem_cache_release+0x3a/0x70 > [<0000001529b04882>] kobject_put+0xaa/0xe8 > [<000000152918cf28>] process_one_work+0x1e8/0x428 > [<000000152918d1b0>] worker_thread+0x48/0x460 > [<00000015291942c6>] kthread+0x126/0x160 > [<0000001529b22344>] ret_from_fork+0x28/0x30 > [<0000001529b2234c>] kernel_thread_starter+0x0/0x10 > Last Breaking-Event-Address: > [<000000152974684c>] percpu_ref_exit+0x4c/0x58 > ---[ end trace b035e7da5788eb09 ]--- > > I have bisected this to > # first bad commit: [f0a3a24b532d9a7e56a33c5112b2a212ed6ec580] mm: memcg/slab: rework non-root kmem_cache lifecycle management > > unmounting /sys/fs/cgroup/memory/ before the test makes the problem go away so > it really seems to be related to the new percpu-refs from this patch. > > Any ideas? > > Christian >