On Sun, May 13, 2018 at 8:21 AM, Eric Biggers <ebiggers3@xxxxxxxxx> wrote: > On Thu, Apr 05, 2018 at 08:15:24PM -0700, Eric Biggers wrote: >> On Mon, Jan 29, 2018 at 01:29:48PM +0800, Tianyu Lan wrote: >> > >> > >> > On 1/27/2018 7:27 AM, Eric Biggers wrote: >> > > On Sat, Dec 16, 2017 at 04:37:02PM +0800, Lan, Tianyu wrote: >> > > > The root cause is that kvm_irqfd_assign() and kvm_irqfd_deassign() can't >> > > > be run in parallel. Some data structure(e.g, irqfd->consumer) will be >> > > > crashed because irqfd may be freed in deassign path before they are used >> > > > in assign path. The other data maybe used in deassign path before >> > > > initialization. Syzbot test hit such case. Add mutx between >> > > > kvm_irqfd_assign() and kvm_irqfd_deassign() can fix such issue. Will >> > > > send patch to fix it. >> > > > >> > > > On 12/16/2017 12:53 PM, Tianyu Lan wrote: >> > > > > I reproduced the issue. Will have a look. >> > > > > >> > > > > -- Best regards Tianyu Lan 2017-12-15 18:14 GMT+08:00 syzbot >> > > > > <bot+7ad82526b98c64c90062275162dacfa48c20dbd3@xxxxxxxxxxxxxxxxxxxxxxxxx>: >> > > > > > syzkaller has found reproducer for the following crash on >> > > > > > 82bcf1def3b5f1251177ad47c44f7e17af039b4b >> > > > > > git://git.cmpxchg.org/linux-mmots.git/master >> > > > > > compiler: gcc (GCC) 7.1.1 20170620 >> > > > > > .config is attached >> > > > > > Raw console output is attached. >> > > > > > C reproducer is attached >> > > > > > syzkaller reproducer is attached. Seehttps://goo.gl/kgGztJ >> > > > > > for information about syzkaller reproducers >> > > > > > >> > > > > > >> > > > > > ================================================================== >> > > > > > BUG: KASAN: use-after-free in __list_add include/linux/list.h:64 [inline] >> > > > > > BUG: KASAN: use-after-free in list_add include/linux/list.h:79 [inline] >> > > > > > BUG: KASAN: use-after-free in irq_bypass_register_consumer+0x4b4/0x500 >> > > > > > virt/lib/irqbypass.c:217 >> > > > > > Write of size 8 at addr ffff8801cdf51180 by task syzkaller436086/15031 >> > > > > > >> > > > > > CPU: 1 PID: 15031 Comm: syzkaller436086 Not tainted 4.15.0-rc2-mm1+ #39 >> > > > > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS >> > > > > > Google 01/01/2011 >> > > > > > Call Trace: >> > > > > > __dump_stack lib/dump_stack.c:17 [inline] >> > > > > > dump_stack+0x194/0x257 lib/dump_stack.c:53 >> > > > > > print_address_description+0x73/0x250 mm/kasan/report.c:252 >> > > > > > kasan_report_error mm/kasan/report.c:351 [inline] >> > > > > > kasan_report+0x25b/0x340 mm/kasan/report.c:409 >> > > > > > __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:435 >> > > > > > __list_add include/linux/list.h:64 [inline] >> > > > > > list_add include/linux/list.h:79 [inline] >> > > > > > irq_bypass_register_consumer+0x4b4/0x500 virt/lib/irqbypass.c:217 >> > > > > > kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:417 [inline] >> > > > > > kvm_irqfd+0x137f/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572 >> > > > > > kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2992 >> > > > > > vfs_ioctl fs/ioctl.c:46 [inline] >> > > > > > do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:686 >> > > > > > SYSC_ioctl fs/ioctl.c:701 [inline] >> > > > > > SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692 >> > > > > > entry_SYSCALL_64_fastpath+0x1f/0x96 >> > > > > > RIP: 0033:0x44d379 >> > > > > > RSP: 002b:00007fc5ff9a9d08 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 >> > > > > > RAX: ffffffffffffffda RBX: 00007fc5ff9aa700 RCX: 000000000044d379 >> > > > > > RDX: 0000000020080fe0 RSI: 000000004020ae76 RDI: 0000000000000005 >> > > > > > RBP: 00000000007ff900 R08: 00007fc5ff9aa700 R09: 00007fc5ff9aa700 >> > > > > > R10: 00007fc5ff9aa700 R11: 0000000000000246 R12: 0000000000000000 >> > > > > > R13: 00000000007ff8ff R14: 00007fc5ff9aa9c0 R15: 0000000000000000 >> > > > > > >> > > > > > Allocated by task 15031: >> > > > > > save_stack+0x43/0xd0 mm/kasan/kasan.c:447 >> > > > > > set_track mm/kasan/kasan.c:459 [inline] >> > > > > > kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:551 >> > > > > > kmem_cache_alloc_trace+0x136/0x750 mm/slab.c:3614 >> > > > > > kmalloc include/linux/slab.h:516 [inline] >> > > > > > kzalloc include/linux/slab.h:705 [inline] >> > > > > > kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:296 [inline] >> > > > > > kvm_irqfd+0x16c/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572 >> > > > > > kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2992 >> > > > > > vfs_ioctl fs/ioctl.c:46 [inline] >> > > > > > do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:686 >> > > > > > SYSC_ioctl fs/ioctl.c:701 [inline] >> > > > > > SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692 >> > > > > > entry_SYSCALL_64_fastpath+0x1f/0x96 >> > > > > > >> > > > > > Freed by task 1402: >> > > > > > save_stack+0x43/0xd0 mm/kasan/kasan.c:447 >> > > > > > set_track mm/kasan/kasan.c:459 [inline] >> > > > > > kasan_slab_free+0x71/0xc0 mm/kasan/kasan.c:524 >> > > > > > __cache_free mm/slab.c:3492 [inline] >> > > > > > kfree+0xca/0x250 mm/slab.c:3807 >> > > > > > irqfd_shutdown+0x13c/0x1a0 arch/x86/kvm/../../../virt/kvm/eventfd.c:148 >> > > > > > process_one_work+0xbfd/0x1bc0 kernel/workqueue.c:2113 >> > > > > > worker_thread+0x223/0x1990 kernel/workqueue.c:2247 >> > > > > > kthread+0x37a/0x440 kernel/kthread.c:238 >> > > > > > ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:524 >> > > > > > >> > > > > > The buggy address belongs to the object at ffff8801cdf51000 >> > > > > > which belongs to the cache kmalloc-512 of size 512 >> > > > > > The buggy address is located 384 bytes inside of >> > > > > > 512-byte region [ffff8801cdf51000, ffff8801cdf51200) >> > > > > > The buggy address belongs to the page: >> > > > > > page:00000000d08a0d63 count:1 mapcount:0 mapping:00000000d54c7be6 index:0x0 >> > > > > > flags: 0x2fffc0000000100(slab) >> > > > > > raw: 02fffc0000000100 ffff8801cdf51000 0000000000000000 0000000100000006 >> > > > > > raw: ffffea00073a7660 ffffea000737f3a0 ffff8801dac00940 0000000000000000 >> > > > > > page dumped because: kasan: bad access detected >> > > > > > >> > > > > > Memory state around the buggy address: >> > > > > > ffff8801cdf51080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> > > > > > ffff8801cdf51100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> > > > > > > ffff8801cdf51180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> > > > > > ^ >> > > > > > ffff8801cdf51200: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc >> > > > > > ffff8801cdf51280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> > > > > > ================================================================== >> > > > > > >> > > Tianyu, are you still planning to send a patch for this? This is still >> > > happening. >> > > >> > > - Eric >> > Yes, I sent out fix patch and wait for review. >> > https://patchwork.kernel.org/patch/10128925/ >> > >> >> This is still happening. Can you please resend your patch? >> > > Still happening; it's pretty easily reproducible, using the program below. > (Sometimes it crashes in different ways too. It also spams the log, ignore > that.) Tianyu's patch at https://patchwork.kernel.org/patch/10128925/ fixes it, > though the locking is difficult to understand so I'm not sure it's correct yet. > Probably the comment that says "do not drop the file until the irqfd is fully > initialized, otherwise we might race against the EPOLLHUP" would be obsoleted > and should be removed? Can the KVM maintainers please review the patch too? > > #include <fcntl.h> > #include <linux/kvm.h> > #include <pthread.h> > #include <sys/eventfd.h> > #include <sys/ioctl.h> > > static int dev, vm, event; > > static void *irqfd_thrproc(void *_flags) > { > struct kvm_irqfd irqfd = { .fd = event, .flags = (long)_flags }; > > for (;;) > ioctl(vm, KVM_IRQFD, &irqfd); > } > > int main() > { > pthread_t t; > > event = eventfd(0, 0); > dev = open("/dev/kvm", 0); > vm = ioctl(dev, KVM_CREATE_VM, 0); > ioctl(vm, KVM_CREATE_IRQCHIP, 0); > pthread_create(&t, NULL, irqfd_thrproc, (void *)(long)0); > irqfd_thrproc((void *)(long)KVM_IRQFD_FLAG_DEASSIGN); > } Tentative duplicates of this bug: BUG: unable to handle kernel NULL pointer dereference in irq_bypass_register_consumer KASAN: use-after-free Read in irq_bypass_unregister_consumer general protection fault in eventfd_ctx_remove_wait_queue KASAN: use-after-free Read in irq_bypass_register_consumer https://syzkaller.appspot.com/bug?id=45591ae3053c59fb50169401fb61cb596735f9d1
