(cc linux-mm)
On Sun, 12 Feb 2023 08:27:32 -0800 Guenter Roeck <linux@xxxxxxxxxxxx> wrote:
> Hi,
>
> I am having problems booting from nvme in linux-next. Bisect points to 06e5887cdcc11
> ("Merge branch 'mm-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm").
> The problem was first seen on February 8.
>
> Some of the log messages as well as the bisect log are attached. Note that the problem
> is not seen in all architectures. The list below may not be complete since several
> architectures/platforms fail to boot for other reasons.
>
> Guenter
>
> ---
>
> # bad: [6ba8a227fd19d19779005fb66ad7562608e1df83] Add linux-next specific files for 20230210
> # good: [4ec5183ec48656cec489c49f989c508b68b518e3] Linux 6.2-rc7
> git bisect start 'HEAD' 'v6.2-rc7'
> # good: [94613f0efc69ed41f9229ef5c294db3ec37145da] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
> git bisect good 94613f0efc69ed41f9229ef5c294db3ec37145da
> # good: [19e62c715fe70dae4582c2874ed3e66715d09af6] Merge branch 'rcu/next' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> git bisect good 19e62c715fe70dae4582c2874ed3e66715d09af6
> # good: [5d8b7ecef7f4a681b6e5538db59ff26c389c0ab6] Merge branch 'for-next' of https://gitlab.com/peda-linux/mux.git
> git bisect good 5d8b7ecef7f4a681b6e5538db59ff26c389c0ab6
> # good: [c349bf6ec83903b20fe570c5609b9a864a64e09c] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/krisman/unicode.git
> git bisect good c349bf6ec83903b20fe570c5609b9a864a64e09c
> # good: [5a06a9f17454df38f35672be522ff5eb9b4277d2] selftest: add testing unsharing and counting ksm zero page
> git bisect good 5a06a9f17454df38f35672be522ff5eb9b4277d2
> # good: [f5d115a7b06e5661ed5218ffa9a2644c4ff1c135] Merge branch 'mm-nonmm-unstable' into mm-everything
> git bisect good f5d115a7b06e5661ed5218ffa9a2644c4ff1c135
> # good: [9a863a6a51394bff480c959b713874c090a8f5c6] mm/gup: make locked never NULL in the internal GUP functions
> git bisect good 9a863a6a51394bff480c959b713874c090a8f5c6
> # good: [80fe99787924df7e6048204be7403a2d0b48918c] Merge branch 'for-next/kspp' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git
> git bisect good 80fe99787924df7e6048204be7403a2d0b48918c
> # good: [bed9e516f1183faa0e484479701cc669efd9049a] Merge branch 'vfio-no-iommu' into iommufd.git for-next
> git bisect good bed9e516f1183faa0e484479701cc669efd9049a
> # bad: [06e5887cdcc111d64caa632c39e00a81340f0262] Merge branch 'mm-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> git bisect bad 06e5887cdcc111d64caa632c39e00a81340f0262
> # good: [223ec6ab265ead0b319bc2f15d0d1be05078a74b] mm/memremap.c: fix outdated comment in devm_memremap_pages
> git bisect good 223ec6ab265ead0b319bc2f15d0d1be05078a74b
> # good: [b942a520d9e43bc31f0808d2f2267a1ddba75518] bcache: Replace zero-length arrays with DECLARE_FLEX_ARRAY() helper
> git bisect good b942a520d9e43bc31f0808d2f2267a1ddba75518
> # good: [3592abd2f44dad9d7232b09e85e8dd1b7fdf91e2] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd.git
> git bisect good 3592abd2f44dad9d7232b09e85e8dd1b7fdf91e2
> # first bad commit: [06e5887cdcc111d64caa632c39e00a81340f0262] Merge branch 'mm-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
>
> ---
> alpha:
>
> Unable to handle kernel paging request at virtual address 0000000000000008
> swapper(0): Oops 1
> pc = [<fffffc0000439100>] ra = [<fffffc00004390e0>] ps = 0007 Tainted: G N
> pc is at dma_pool_free+0x60/0xf0
> ra is at dma_pool_free+0x40/0xf0
> v0 = 0000000000000007 t0 = 0000000000000000 t1 = fffffc00012037c0
> t2 = 0000000000000001 t3 = fffffc0002ac1100 t4 = fffffc00000400a0
> t5 = fffffc00000400a0 t6 = 0000000000000000 t7 = fffffc00011b0000
> s0 = fffffc0001e6ae40 s1 = 0000000000000000 s2 = 0000000000000007
> s3 = 0000000000000000 s4 = fffffc0001e6ae50 s5 = 0000000000000005
> s6 = fffffc0001eb7932
> a0 = fffffc0001e6ae50 a1 = 0000000000000000 a2 = 0000000000000000
> a3 = 0000000000000002 a4 = 0000000000000000 a5 = 000000000000005a
> t8 = 0000000000000000 t9 = fffffc0001b67248 t10= 0000000000000000
> t11= 0000000000000140 pv = fffffc000037b400 at = 0000000000000001
> gp = fffffc000123cf78 sp = fffffc00011b3b80
> Disabling lock debugging due to kernel taint
> Trace:
> [<fffffc0000a0e1dc>] nvme_unmap_data+0xac/0x220
> [<fffffc0000a0e350>] nvme_pci_complete_batch+0x0/0xe0
> [<fffffc0000a0fda8>] nvme_poll_cq+0x4b8/0x510
> [<fffffc0000a0e350>] nvme_pci_complete_batch+0x0/0xe0
> [<fffffc0000a119f4>] nvme_irq+0x34/0xd0
> [<fffffc00003b076c>] tick_handle_periodic+0x8c/0xc0
> [<fffffc00003857a0>] __handle_irq_event_percpu+0x80/0x1c0
> [<fffffc00003859e8>] handle_irq_event+0x68/0x110
> [<fffffc000038af24>] handle_level_irq+0xe4/0x220
> [<fffffc0000316020>] handle_irq+0x90/0x100
> [<fffffc0000325204>] clipper_srm_device_interrupt+0x24/0x40
> [<fffffc00003161c0>] do_entInt+0x130/0x1e0
> [<fffffc0000311510>] ret_from_sys_call+0x0/0x10
> [<fffffc00003736b8>] do_idle+0x88/0x1e0
> [<fffffc0000df7460>] cpu_idle_poll.isra.0+0x0/0x70
> [<fffffc0000df74a0>] cpu_idle_poll.isra.0+0x40/0x70
> [<fffffc0000373bd8>] cpu_startup_entry+0x28/0x40
> [<fffffc0000df7640>] rest_init+0x100/0x104
> [<fffffc000031001c>] _stext+0x1c/0x20
> [<fffffc0000310000>] _stext+0x0/0x20
>
> ---
> mips (all variants):
>
> CPU 0 Unable to handle kernel paging request at virtual address 00000004, epc == 802d9eb8, ra == 802d9ea4
> Oops[#1]:
> CPU: 0 PID: 0 Comm: swapper/0 Tainted: G N 6.2.0-rc7-next-20230210 #1
> $ 0 : 00000000 00000001 00000000 00000000
> $ 4 : 8260e888 00000000 810ee780 00000001
> $ 8 : 00000000 813dba24 05345ded 0000020e
> $12 : ffffffff 00000001 8198835c 00000000
> $16 : 8260e880 00000000 00000000 01244800
> $20 : 8260e888 820afec8 808b6c24 a2668290
> $24 : 00000002 810f0000
> $28 : 810c4000 820afe18 81030000 802d9ea4
> Hi : 000656d0
> Lo : 4b877998
> epc : 802d9eb8 dma_pool_free+0x48/0xa0
> ra : 802d9ea4 dma_pool_free+0x34/0xa0
> Status: 1000a402 KERNEL EXL
> Cause : 0080000c (ExcCode 03)
> BadVA : 00000004
> PrId : 00019300 (MIPS 24Kc)
> Modules linked in:
> Process swapper/0 (pid: 0, threadinfo=d5273c62, task=cb66b3ba, tls=00000000)
> Stack : ffffffff fffffffe ffffffff ffffffff 821e0dc0 01244800 820b1000 811b61c0
> 810eaa0c 808b6af0 811b61c0 8275c000 820afec8 808b6c24 00000000 801c92f8
> 82604700 821e0dc0 00000001 811b61c0 810eaa0c 808b7b34 810d3b3c 8123447c
> 00000002 8123447c 00000000 80d6660c 81030000 810d3b2c 810f0000 00000000
> 810d3ac0 811b01e0 811b0200 0000000b 00000001 82650c80 00000000 808b86c0
> ...
> Call Trace:
> [<802d9eb8>] dma_pool_free+0x48/0xa0
> [<808b6af0>] nvme_unmap_data+0x84/0x1b8
> [<808b7b34>] nvme_poll_cq+0x444/0x558
> [<808b86c0>] nvme_irq+0x34/0x84
> [<801b8020>] __handle_irq_event_percpu+0x280/0x434
> [<801b8284>] handle_irq_event+0x50/0xbc
> [<801bdf18>] handle_level_irq+0xf8/0x218
> [<801b6eec>] generic_handle_domain_irq+0x2c/0x44
> [<801b6eec>] generic_handle_domain_irq+0x2c/0x44
> [<80d734d8>] do_domain_IRQ+0x20/0x2c
> [<80795b88>] plat_irq_dispatch+0x64/0xcc
> [<801039a0>] handle_int+0x160/0x16c
> [<80d679b0>] r4k_wait+0xc/0x2c
> [<80d67e38>] default_idle_call+0x98/0x384
> [<80189670>] do_idle+0xfc/0x1cc
> [<80189b60>] cpu_startup_entry+0x2c/0x34
> [<80d68310>] rest_init+0x118/0x1c0
> [<811bc788>] arch_post_acpi_subsys_init+0x0/0x8
>
>
> ---
> sparc64:
>
> 5.070356] Unable to handle kernel NULL pointer dereference
> [ 5.071034] tsk->{mm,active_mm}->context = 0000000000000001
> [ 5.071519] tsk->{mm,active_mm}->pgd = fffff800048b0000
> [ 5.071812] \|/ ____ \|/
> [ 5.071812] "@'/ .. \`@"
> [ 5.071812] /_| \__/ |_\
> [ 5.071812] \__U_/
> [ 5.072549] init(1): Oops [#1]
> [ 5.073008] CPU: 0 PID: 1 Comm: init Tainted: G N 6.2.0-rc7-next-20230210 #1
> [ 5.073599] TSTATE: 0000004480e01605 TPC: 000000000061bdb0 TNPC: 000000000061bdb4 Y: 00000000 Tainted: G N
> [ 5.074154] TPC: <dma_pool_free+0x30/0xa0>
> [ 5.074778] g0: 0265a89d01ced61d g1: 0000000000000000 g2: 0000000000000000 g3: 000000000000000a
> [ 5.075311] g4: fffff800041669a0 g5: fffff8001dca0000 g6: fffff80004168000 g7: 0000000080b583eb
> [ 5.075821] o0: 000000000000000e o1: 000000000000000e o2: 0000000000000006 o3: 0000000000000002
> [ 5.076265] o4: 0000000000000000 o5: 000000000000000e sp: fffff8001fdff191 ret_pc: 000000000061bd90
> [ 5.076716] RPC: <dma_pool_free+0x10/0xa0>
> [ 5.076978] l0: 0000000001b61a58 l1: 000000000000000e l2: 000000000208c278 l3: 0000000000000002
> [ 5.077421] l4: 81b8e1e5a4e0c637 l5: fffff80004167420 l6: 000000000000401e l7: 0000000000000001
> [ 5.077862] i0: fffff800045bae90 i1: 000000000000000e i2: 0000000000000000 i3: 0000000000000002
> [ 5.078299] i4: 0000000000000000 i5: fffff800045bae80 i6: fffff8001fdff241 i7: 0000000000b53e54
> [ 5.078746] I7: <nvme_unmap_data+0x74/0x140>
> [ 5.079031] Call Trace:
> [ 5.079384] [<0000000000b53e54>] nvme_unmap_data+0x74/0x140
> [ 5.079752] [<0000000000b54338>] nvme_handle_cqe+0x338/0x3e0
> [ 5.080053] [<0000000000b5460c>] nvme_irq+0x8c/0x180
> [ 5.080316] [<00000000004e7564>] __handle_irq_event_percpu+0x204/0x3a0
> [ 5.080647] [<00000000004e7774>] handle_irq_event+0x34/0xc0
> [ 5.080934] [<00000000004ec750>] handle_fasteoi_irq+0x90/0x220
> [ 5.081231] [<00000000004e6128>] generic_handle_irq+0x28/0x40
> [ 5.081526] [<0000000000f4d12c>] handler_irq+0xac/0x100
> [ 5.081848] [<00000000004274b0>] sys_call_table+0x760/0x970
> [ 5.082191] [<000000000090196c>] __blk_mq_run_hw_queue+0x16c/0x2a0
> [ 5.082646] [<000000000090c0dc>] blk_mq_sched_insert_requests+0xdc/0x5c0
> [ 5.082993] [<00000000009053c0>] blk_mq_flush_plug_list+0xe0/0x700
> [ 5.083311] [<00000000008f6280>] __blk_flush_plug+0xc0/0x100
> [ 5.083602] [<00000000008f66c4>] blk_finish_plug+0x24/0x40
> [ 5.083886] [<00000000005b48c4>] read_pages+0x164/0x280
> [ 5.084159] [<00000000005b4af8>] page_cache_ra_unbounded+0x118/0x1a0
