On Tue, 7 Nov 2023 at 19:51, Mark Brown <broonie@xxxxxxxxxx> wrote: > > On Tue, Nov 07, 2023 at 06:43:25PM +0530, Naresh Kamboju wrote: > > > # # SVE-VL-64-0: Expected > > [3904000039044000390480003904c0003904000139044001390480013904c0013904000239044002390480023904c0023904000339044003390480033904c003] > > <> > > You've elided *lots* of error reports from the actual test which suggest > that there is substantial memory corruption, it looks like tearing part > way through loading or saving the values - the start of the vectors > looks fine but at some point they get what looks like a related process' > data, eg: > > # # SVE-VL-64-0: Expected [3904000039044000390480003904c0003904000139044001390480013904c0013904000239044002390480023904c0023904000339044003390480033904c003] > # # SVE-VL-64-0: Got [3904000039044000390480003904c000390480003904c00039040001390440013904000139044001390480013904c001390480013904c0013904000239044002] > > This only appears to affect SVE and SME, I didn't spot any FPSIMD > corruption but then that is the smallest case (and I didn't notice any > VL 16 cases either). It looks like the corruption is on the first thing > we check each time (either register 0 or the highest ZA.H vector for > ZA), all the values do look lke they were plausibly generated by > fp-stress test programs. > > Then we get what looks like memory corruption: > > > # # SVE-VL-256-<1>[ 88.160313] Unable to handle kernel paging > > request at virtual address 00550f0344550f02 > > > <4>[ 88.195706] Call trace: > > <4>[ 88.196098] percpu_ref_get_many > > (include/linux/percpu-refcount.h:174 (discriminator 2) > > include/linux/percpu-refcount.h:204 (discriminator 2)) > > <4>[ 88.196815] refill_obj_stock (mm/memcontrol.c:3339 (discriminator 2)) > > <4>[ 88.197367] obj_cgroup_uncharge (mm/memcontrol.c:3406) > > <4>[ 88.197835] kmem_cache_free (include/linux/mm.h:1630 > > include/linux/mm.h:1849 include/linux/mm.h:1859 mm/slab.h:208 > > mm/slab.h:572 mm/slub.c:3804 mm/slub.c:3831) > > <4>[ 88.198407] put_pid.part.0 (kernel/pid.c:118) > > <4>[ 88.198870] delayed_put_pid (kernel/pid.c:127) > > <4>[ 88.200527] rcu_core (arch/arm64/include/asm/preempt.h:13 > > (discriminator 1) kernel/rcu/tree.c:2146 (discriminator 1) > > kernel/rcu/tree.c:2403 (discriminator 1)) > > This all seems very surprising, especially given that AFAICT there are > no changes in stable-6.6-rc for arch/arm64. We do not see on the mainline and next. Is this reported problems on stable-rc 6.6 and 6.5 are due to running latest kselftest on older kernels ? -- # # SSVE-VL-32-1: Mismatch: PID=641, iteration=0, reg=0 # # SSVE-VL-128-1: Got [<junk>] # # SSVE-VL-256-1: Got [<junk>] Unable to handle kernel paging request at virtual address 00740f0322740f02 0<1>[ 89.400173] Mem abort info: <1>[ 89.400844] ESR = 0x0000000096000004 <1>[ 89.401974] EC = 0x25: DABT (current EL), IL = 32 bits <1>[ 89.403399] SET = 0, FnV = 0 <1>[ 89.404421] EA = 0, S1PTW = 0 <1>[ 89.405317] FSC = 0x04: level 0 translation fault <1>[ 89.406545] Data abort info: <1>[ 89.407493] ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000 <1>[ 89.408785] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 <1>[ 89.410001] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 <1>[ 89.411485] [00740f0322740f02] address between user and kernel address ranges <0>[ 89.413851] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP <4>[ 89.415573] Modules linked in: crct10dif_ce sm3_ce sm3 sha3_ce sha512_ce sha512_arm64 fuse drm dm_mod ip_tables x_tables <4>[ 89.419561] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.5.11-rc1 #1 <4>[ 89.420795] Hardware name: linux,dummy-virt (DT) <4>[ 89.422676] pstate: 624000c9 (nZCv daIF +PAN -UAO +TCO -DIT -SSBS BTYPE=--) <4>[ 89.424344] pc : refill_obj_stock+0x6c/0x250 <4>[ 89.426324] lr : refill_obj_stock+0x6c/0x250 <trim> <4>[ 89.447170] Call trace: <4>[ 89.447756] refill_obj_stock+0x6c/0x250 <4>[ 89.449033] obj_cgroup_uncharge+0x20/0x38 <4>[ 89.450457] kmem_cache_free+0xf8/0x500 <4>[ 89.451066] delayed_put_pid+0x50/0xb0 <4>[ 89.452401] rcu_core+0x3cc/0x950 <4>[ 89.453369] rcu_core_si+0x1c/0x30 <4>[ 89.454465] __do_softirq+0x118/0x438 <4>[ 89.455738] run_ksoftirqd+0x40/0xf8 <4>[ 89.456893] smpboot_thread_fn+0x1d0/0x248 <4>[ 89.457969] kthread+0xfc/0x1a0 <4>[ 89.459171] ret_from_fork+0x10/0x20 <0>[ 89.460445] Code: aa1603e0 97fffef8 aa0003f4 97f6cbf6 (f9400269) <4>[ 89.462028] ---[ end trace 0000000000000000 ]--- <0>[ 89.463494] Kernel panic - not syncing: Oops: Fatal exception in interrupt <2>[ 89.465046] SMP: stopping secondary CPUs <0>[ 89.466327] Kernel Offset: 0x2dabffa00000 from 0xffff800080000000 <0>[ 89.467385] PHYS_OFFSET: 0x40000000 <0>[ 89.468131] CPU features: 0x00000000,68f167a1,cce6773f <0>[ 89.469850] Memory Limit: none <0>[ 89.470836] ---[ end Kernel panic - not syncing: Oops: Fatal exception in interrupt ]--- Links: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.5.y/build/v6.5.10-89-g73d52afec3ab/testrun/20963504/suite/log-parser-test/tests/ https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.5.y/build/v6.5.10-89-g73d52afec3ab/testrun/20963709/suite/log-parser-test/tests/ https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.5.y/build/v6.5.10-89-g73d52afec3ab/testrun/20963709/suite/log-parser-test/test/check-kernel-oops/log - Naresh
