On Mon, 2019-04-01 at 21:29 +-0300, Leon Romanovsky wrote: +AD4 On Mon, Apr 01, 2019 at 02:45:54PM -0300, Jason Gunthorpe wrote: +AD4 +AD4 On Mon, Apr 01, 2019 at 10:36:05AM -0700, syzbot wrote: +AD4 +AD4 +AD4 Hello, +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 syzbot found the following crash on: +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 HEAD commit: e3ecb83e Add linux-next specific files for 20190401 +AD4 +AD4 +AD4 git tree: linux-next +AD4 +AD4 +AD4 console output: https://syzkaller.appspot.com/x/log.txt?x+AD0-13bc36cd200000 +AD4 +AD4 +AD4 kernel config: https://syzkaller.appspot.com/x/.config?x+AD0-db6c9f2bfeb91a99 +AD4 +AD4 +AD4 dashboard link: https://syzkaller.appspot.com/bug?extid+AD0-2e3e485d5697ea610460 +AD4 +AD4 +AD4 compiler: gcc (GCC) 9.0.0 20181231 (experimental) +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 Unfortunately, I don't have any reproducer for this crash yet. +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 IMPORTANT: if you fix the bug, please add the following tag to the commit: +AD4 +AD4 +AD4 Reported-by: syzbot+-2e3e485d5697ea610460+AEA-syzkaller.appspotmail.com +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 WARNING: CPU: 1 PID: 7 at drivers/infiniband/core/cma.c:4674 +AD4 +AD4 +AD4 cma+AF8-exit+AF8-net+-0x327/0x390 drivers/infiniband/core/cma.c:4674 +AD4 +AD4 +AD4 Kernel panic - not syncing: panic+AF8-on+AF8-warn set ... +AD4 +AD4 +AD4 +AD4 Matt: This is why the WARN+AF8-ON(+ACE-xa+AF8-empty()) is so valuable. Magically +AD4 +AD4 syzkaller can find something in this code is buggy. +AD4 +AD4 +AD4 +AD4 Mellanox is also showing a different testing failure over the weekend +AD4 +AD4 (use after free or something) from your 'cma: Convert portspace IDRs +AD4 +AD4 to XArray' +AD4 +AD4 This is what I see in my environment. +AD4 +AD4 +AFs 72.725596+AF0 +AD4 +AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9 +AD4 +AFs 72.726017+AF0 BUG: KASAN: use-after-free in cma+AF8-check+AF8-port+-0x86a/0xa20 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.726263+AF0 Read of size 8 at addr ffff888069fde998 by task ucmatose/387 +AD4 +AFs 72.726460+AF0 +AD4 +AFs 72.726550+AF0 CPU: 3 PID: 387 Comm: ucmatose Not tainted 5.1.0-rc2+- +ACM-253 +AD4 +AFs 72.726751+AF0 Hardware name: QEMU Standard PC (Q35 +- ICH9, 2009), BIOS +AD4 rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014 +AD4 +AFs 72.727119+AF0 Call Trace: +AD4 +AFs 72.727210+AF0 dump+AF8-stack+-0x7c/0xc0 +AD4 +AFs 72.727342+AF0 print+AF8-address+AF8-description+-0x6c/0x23c +AD4 +AFs 72.727505+AF0 ? cma+AF8-check+AF8-port+-0x86a/0xa20 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.727666+AF0 kasan+AF8-report.cold.3+-0x1c/0x35 +AD4 +AFs 72.727805+AF0 ? cma+AF8-check+AF8-port+-0x86a/0xa20 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.727977+AF0 ? cma+AF8-check+AF8-port+-0x86a/0xa20 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.728138+AF0 cma+AF8-check+AF8-port+-0x86a/0xa20 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.728306+AF0 rdma+AF8-bind+AF8-addr+-0x11bc/0x1b00 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.728465+AF0 ? find+AF8-held+AF8-lock+-0x33/0x1c0 +AD4 +AFs 72.728597+AF0 ? cma+AF8-ndev+AF8-work+AF8-handler+-0x180/0x180 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.728756+AF0 ? wait+AF8-for+AF8-completion+-0x3d0/0x3d0 +AD4 +AFs 72.728928+AF0 ucma+AF8-bind+-0x120/0x160 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.729089+AF0 ? ucma+AF8-resolve+AF8-addr+-0x1a0/0x1a0 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.729256+AF0 ucma+AF8-write+-0x1f8/0x2b0 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.729409+AF0 ? ucma+AF8-open+-0x260/0x260 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.729571+AF0 vfs+AF8-write+-0x157/0x460 +AD4 +AFs 72.729688+AF0 ksys+AF8-write+-0xb8/0x170 +AD4 +AFs 72.729828+AF0 ? +AF8AXw-ia32+AF8-sys+AF8-read+-0xb0/0xb0 +AD4 +AFs 72.729954+AF0 ? trace+AF8-hardirqs+AF8-off+AF8-caller+-0x5b/0x160 +AD4 +AFs 72.730107+AF0 ? do+AF8-syscall+AF8-64+-0x18/0x3c0 +AD4 +AFs 72.730243+AF0 do+AF8-syscall+AF8-64+-0x95/0x3c0 +AD4 +AFs 72.730363+AF0 entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe +AD4 +AFs 72.730508+AF0 RIP: 0033:0x7f6f1758fff8 +AD4 +AFs 72.730624+AF0 Code: 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 +AD4 00 f3 0f 1e fa 48 8d 05 25 77 0d 00 8b 00 85 c0 75 17 b8 01 00 00 00 0f +AD4 05 +ADw-48+AD4 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 41 54 49 89 d4 +AD4 55 +AD4 +AFs 72.731146+AF0 RSP: 002b:00007fff99f99088 EFLAGS: 00000246 ORIG+AF8-RAX: 0000000000000001 +AD4 +AFs 72.731365+AF0 RAX: ffffffffffffffda RBX: 00007fff99f99090 RCX: 00007f6f1758fff8 +AD4 +AFs 72.731579+AF0 RDX: 0000000000000090 RSI: 00007fff99f99090 RDI: 0000000000000003 +AD4 +AFs 72.731814+AF0 RBP: 0000564942bd8ec0 R08: 0000564942bd9180 R09: 0000000000000000 +AD4 +AFs 72.732043+AF0 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 +AD4 +AFs 72.732262+AF0 R13: 0000000000000001 R14: 0000000000000000 R15: 00005649413cc470 +AD4 +AFs 72.732494+AF0 +AD4 +AFs 72.732572+AF0 Allocated by task 381: +AD4 +AFs 72.732692+AF0 +AF8AXw-kasan+AF8-kmalloc.constprop.5+-0xc1/0xd0 +AD4 +AFs 72.732857+AF0 cma+AF8-alloc+AF8-port+-0x4d/0x160 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.733006+AF0 rdma+AF8-bind+AF8-addr+-0x14e7/0x1b00 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.733153+AF0 ucma+AF8-bind+-0x120/0x160 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.733299+AF0 ucma+AF8-write+-0x1f8/0x2b0 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.733452+AF0 vfs+AF8-write+-0x157/0x460 +AD4 +AFs 72.733569+AF0 ksys+AF8-write+-0xb8/0x170 +AD4 +AFs 72.733675+AF0 do+AF8-syscall+AF8-64+-0x95/0x3c0 +AD4 +AFs 72.733800+AF0 entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe +AD4 +AFs 72.733956+AF0 +AD4 +AFs 72.734029+AF0 Freed by task 381: +AD4 +AFs 72.734133+AF0 +AF8AXw-kasan+AF8-slab+AF8-free+-0x12e/0x180 +AD4 +AFs 72.734284+AF0 kfree+-0xed/0x290 +AD4 +AFs 72.734399+AF0 rdma+AF8-destroy+AF8-id+-0x6b6/0x9e0 +AFs-rdma+AF8-cm+AF0 +AD4 +AFs 72.734559+AF0 ucma+AF8-close+-0x110/0x300 +AFs-rdma+AF8-ucm+AF0 +AD4 +AFs 72.734701+AF0 +AF8AXw-fput+-0x25a/0x740 +AD4 +AFs 72.734832+AF0 task+AF8-work+AF8-run+-0x10e/0x190 +AD4 +AFs 72.734959+AF0 do+AF8-exit+-0x85e/0x29e0 +AD4 +AFs 72.735071+AF0 do+AF8-group+AF8-exit+-0xf0/0x2e0 +AD4 +AFs 72.735182+AF0 get+AF8-signal+-0x2e0/0x17e0 +AD4 +AFs 72.735304+AF0 do+AF8-signal+-0x94/0x1570 +AD4 +AFs 72.735424+AF0 exit+AF8-to+AF8-usermode+AF8-loop+-0xfa/0x130 +AD4 +AFs 72.735612+AF0 do+AF8-syscall+AF8-64+-0x327/0x3c0 +AD4 +AFs 72.735756+AF0 entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe +AD4 +AFs 72.735941+AF0 +AD4 +AFs 72.736033+AF0 The buggy address belongs to the object at ffff888069fde990 +AD4 +AFs 72.736033+AF0 which belongs to the cache kmalloc-32 of size 32 +AD4 +AFs 72.736414+AF0 The buggy address is located 8 bytes inside of +AD4 +AFs 72.736414+AF0 32-byte region +AFs-ffff888069fde990, ffff888069fde9b0) +AD4 +AFs 72.736777+AF0 The buggy address belongs to the page: +AD4 +AFs 72.736940+AF0 page:ffffea0001a7f780 count:1 mapcount:0 mapping:ffff88806bc03980 index:0x0 +AD4 +AFs 72.737171+AF0 flags: 0x4000000000000200(slab) +AD4 +AFs 72.737295+AF0 raw: 4000000000000200 dead000000000100 dead000000000200 ffff88806bc03980 +AD4 +AFs 72.737525+AF0 raw: 0000000000000000 0000000000550055 00000001ffffffff 0000000000000000 +AD4 +AFs 72.737786+AF0 page dumped because: kasan: bad access detected +AD4 +AFs 72.737948+AF0 +AD4 +AFs 72.738019+AF0 Memory state around the buggy address: +AD4 +AFs 72.738164+AF0 ffff888069fde880: fb fb fc fc fb fb fb fb fc fc fb fb fb fb fc fc +AD4 +AFs 72.738396+AF0 ffff888069fde900: fb fb fb fb fc fc fb fb fb fb fc fc fb fb fb fb +AD4 +AFs 72.738627+AF0 +AD4-ffff888069fde980: fc fc fb fb fb fb fc fc fb fb fb fb fc fc fb fb +AD4 +AFs 72.738869+AF0 +AF4 +AD4 +AFs 72.738999+AF0 ffff888069fdea00: fb fb fc fc fb fb fb fb fc fc fb fb fb fb fc fc +AD4 +AFs 72.739213+AF0 ffff888069fdea80: fb fb fb fb fc fc fb fb fb fb fc fc fb fb fb fb +AD4 +AFs 72.739431+AF0 +AD4 +AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9 +AD4 +AFs 72.739667+AF0 Disabling lock debugging due to kernel taint This is what I encountered while running blktests: nvmet: adding nsid 1 to subsystem nvme-test +AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9 BUG: KASAN: use-after-free in cma+AF8-check+AF8-port+-0x28/0x400 +AFs-rdma+AF8-cm+AF0 Read of size 8 at addr ffff8880ba96f818 by task ln/10510 CPU: 5 PID: 10510 Comm: ln Not tainted 5.1.0-rc3-dbg+- +ACM-9 Hardware name: QEMU Standard PC (i440FX +- PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: dump+AF8-stack+-0x86/0xca print+AF8-address+AF8-description+-0x71/0x239 ? cma+AF8-check+AF8-port+-0x28/0x400 +AFs-rdma+AF8-cm+AF0 kasan+AF8-report.cold.3+-0x1b/0x3e ? cma+AF8-check+AF8-port+-0x28/0x400 +AFs-rdma+AF8-cm+AF0 +AF8AXw-asan+AF8-load8+-0x54/0x90 cma+AF8-check+AF8-port+-0x28/0x400 +AFs-rdma+AF8-cm+AF0 rdma+AF8-bind+AF8-addr+-0xc13/0xe80 +AFs-rdma+AF8-cm+AF0 ? cma+AF8-ndev+AF8-work+AF8-handler+-0xf0/0xf0 +AFs-rdma+AF8-cm+AF0 ? lockdep+AF8-hardirqs+AF8-on+-0x185/0x260 ? +AF8-raw+AF8-spin+AF8-unlock+AF8-irqrestore+-0x57/0x70 ? trace+AF8-hardirqs+AF8-on+-0x24/0x130 ? preempt+AF8-count+AF8-sub+-0x18/0xd0 ? +AF8-raw+AF8-spin+AF8-unlock+AF8-irqrestore+-0x42/0x70 nvmet+AF8-rdma+AF8-add+AF8-port+-0x143/0x1a0 +AFs-nvmet+AF8-rdma+AF0 ? nvmet+AF8-rdma+AF8-remove+AF8-port+-0x40/0x40 +AFs-nvmet+AF8-rdma+AF0 nvmet+AF8-enable+AF8-port+-0x85/0x180 +AFs-nvmet+AF0 nvmet+AF8-port+AF8-subsys+AF8-allow+AF8-link+-0x1bc/0x1e0 +AFs-nvmet+AF0 ? do+AF8-raw+AF8-spin+AF8-unlock+-0xa8/0x140 configfs+AF8-symlink+-0x2b6/0x650 ? configfs+AF8-get+AF8-link+-0x3e0/0x3e0 ? inode+AF8-permission+-0x69/0x200 vfs+AF8-symlink+-0x163/0x230 do+AF8-symlinkat+-0xeb/0x160 ? +AF8AXw-ia32+AF8-sys+AF8-unlink+-0x40/0x40 ? do+AF8-syscall+AF8-64+-0x19/0x210 ? entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe +AF8AXw-x64+AF8-sys+AF8-symlinkat+-0x43/0x50 do+AF8-syscall+AF8-64+-0x71/0x210 entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe RIP: 0033:0x7f35b984d9e7 Code: 73 01 c3 48 8b 0d a9 84 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 0a 01 00 00 0f 05 +ADw-48+AD4 3d 01 f0 ff ff 73 01 c3 48 8b 0d 79 84 0d 00 f7 d8 64 89 01 48 RSP: 002b:00007ffcc3a88af8 EFLAGS: 00000246 ORIG+AF8-RAX: 000000000000010a RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f35b984d9e7 RDX: 000055c1855f82b0 RSI: 00000000ffffff9c RDI: 00007ffcc3a8a7a6 RBP: 00000000ffffff9c R08: 000055c1855f8010 R09: 0000000000000000 R10: fffffffffffff000 R11: 0000000000000246 R12: 000055c1855f82b0 R13: 00007ffcc3a8a7a6 R14: 0000000000000000 R15: 0000000000000000 Allocated by task 10270: save+AF8-stack+-0x43/0xd0 +AF8AXw-kasan+AF8-kmalloc.constprop.9+-0xc7/0xd0 kasan+AF8-kmalloc+-0x9/0x10 kmem+AF8-cache+AF8-alloc+AF8-trace+-0x143/0x350 cma+AF8-alloc+AF8-port+-0x3d/0xf0 +AFs-rdma+AF8-cm+AF0 rdma+AF8-bind+AF8-addr+-0xdf9/0xe80 +AFs-rdma+AF8-cm+AF0 nvmet+AF8-clear+AF8-ctrl+-0x43/0x70 +AFs-nvmet+AF0 rxe+AF8-opcode+-0x15f5/0xfffffffffffef380 +AFs-rdma+AF8-rxe+AF0 rxe+AF8-wr+AF8-opcode+AF8-info+-0xa4c/0xfffffffffffeb360 +AFs-rdma+AF8-rxe+AF0 configfs+AF8-symlink+-0x2b6/0x650 vfs+AF8-symlink+-0x163/0x230 do+AF8-symlinkat+-0xeb/0x160 +AF8AXw-x64+AF8-sys+AF8-symlinkat+-0x43/0x50 do+AF8-syscall+AF8-64+-0x71/0x210 entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe Freed by task 10340: save+AF8-stack+-0x43/0xd0 +AF8AXw-kasan+AF8-slab+AF8-free+-0x139/0x190 kasan+AF8-slab+AF8-free+-0xe/0x10 kfree+-0x103/0x320 rdma+AF8-destroy+AF8-id+-0x42c/0x460 +AFs-rdma+AF8-cm+AF0 nvmet+AF8-ctrl+AF8-fatal+AF8-error+-0x31/0x80 +AFs-nvmet+AF0 rxe+AF8-opcode+-0x175d/0xfffffffffffef380 +AFs-rdma+AF8-rxe+AF0 rxe+AF8-wr+AF8-opcode+AF8-info+-0x644/0xfffffffffffeb360 +AFs-rdma+AF8-rxe+AF0 configfs+AF8-unlink+-0x216/0x350 vfs+AF8-unlink+-0x171/0x260 do+AF8-unlinkat+-0x347/0x490 +AF8AXw-x64+AF8-sys+AF8-unlinkat+-0x60/0x90 do+AF8-syscall+AF8-64+-0x71/0x210 entry+AF8-SYSCALL+AF8-64+AF8-after+AF8-hwframe+-0x49/0xbe The buggy address belongs to the object at ffff8880ba96f810 which belongs to the cache kmalloc-32 of size 32 The buggy address is located 8 bytes inside of 32-byte region +AFs-ffff8880ba96f810, ffff8880ba96f830) The buggy address belongs to the page: page:ffffea0002ea5bc0 count:1 mapcount:0 mapping:ffff88811b003800 index:0x0 flags: 0x1fff000000000200(slab) raw: 1fff000000000200 ffffea0004319200 0000000900000009 ffff88811b003800 raw: 0000000000000000 0000000000550055 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff8880ba96f700: fb fb fc fc fb fb fb fb fc fc fb fb fb fb fc fc ffff8880ba96f780: fb fb fb fb fc fc fb fb fb fb fc fc fb fb fb fb +AD4-ffff8880ba96f800: fc fc fb fb fb fb fc fc fb fb fb fb fc fc fb fb +AF4 ffff8880ba96f880: fb fb fc fc fb fb fb fb fc fc fb fb fb fb fc fc ffff8880ba96f900: fb fb fb fb fc fc fb fb fb fb fc fc fb fb fb fb +AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9AD0APQA9 Disabling lock debugging due to kernel taint nvmet+AF8-rdma: enabling port 1 (192.168.122.49:7777)
