On Wed, Jul 29, 2009 at 11:06 AM, Stephen Donnelly<sfdonnelly@xxxxxxxxx> wrote: > On Tue, Jul 28, 2009 at 8:54 PM, Avi Kivity<avi@xxxxxxxxxx> wrote: >> On 07/28/2009 12:32 AM, Stephen Donnelly wrote: >> You need a variant of qemu_ram_alloc() that accepts an fd and offset and >> mmaps that. I had a go at this, creating qemu_ram_mmap() using qemu_ram_alloc() as a template, but I'm still seeing the same BUG. >> A less intrusive, but uglier, alternative is to call >> qemu_ram_alloc() and them mmap(MAP_FIXED) on top of that. > > I did try this, but ended up with a BUG on the host in > /var/lib/dkms/kvm/84/build/x86/kvm_main.c:1266 gfn_to_pfn() on the > line "BUG_ON(!kvm_is_mmio_pfn(pfn));" when the guest accesses the bar. It looks to me from the call trace like the guest is writing to the memory, gfn_to_pfn() from mmu_guess_page_from_pte_write() gets confused because of the mapping. Inside gfn_to_pfn: addr = gfn_to_hva(kvm, gfn); correctly returns the host virtual address of the external memory mapping. npages = get_user_pages_fast(addr, 1, 1, page); returns -EFAULT, presumably because (vma->vm_flags & (VM_IO | VM_PFNMAP)). It takes then unlikely branch, and checks the vma, but I don't understand what it is doing here: pfn = ((addr - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff; In my case addr == vma->vm_start, and vma->vm_pgoff == 0, so pfn ==0. BUG_ON(!kvm_is_mmio_pfn(pfn)) then triggers. Instrumenting inside gfn_to_pfn I see: gfn_to_pfn: gfn f2010 gpte f2010000 hva 7f3eac2b0000 pfn 0 npages -14 gfn_to_pfn: vma ffff88022142af18 start 7f3eac2b0000 pgoff 0 Any suggestions what should be happening here? [ 1826.807846] ------------[ cut here ]------------ [ 1826.807907] kernel BUG at /build/buildd/linux-2.6.28/arch/x86/kvm/../../../virt/kvm/kvm_main.c:1001! [ 1826.807985] invalid opcode: 0000 [#1] SMP [ 1826.808102] last sysfs file: /sys/module/nf_nat/initstate [ 1826.808159] Dumping ftrace buffer: [ 1826.808213] (ftrace buffer empty) [ 1826.808266] CPU 3 [ 1826.808347] Modules linked in: tun softcard_driver(P) ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ip v4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables kvm_intel kvm input_polldev video output bridge stp lp parport iTCO_wdt iTCO_vendor_support psmouse pcspkr serio_raw joydev i5000_edac edac_core shpchp e1000 us bhid usb_storage e1000e floppy raid10 raid456 async_xor async_memcpy async_tx xor raid1 raid0 multipath linear fbcon til eblit font bitblit softcursor [ 1826.810269] Pid: 9353, comm: qemu-system-x86 Tainted: P 2.6.28-13-server #45-Ubuntu [ 1826.810344] RIP: 0010:[<ffffffffa01da853>] [<ffffffffa01da853>] gfn_to_pfn+0x153/0x160 [kvm] [ 1826.810463] RSP: 0018:ffff88022d857958 EFLAGS: 00010246 [ 1826.810518] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88022d4d32a0 [ 1826.810577] RDX: 0000000000000000 RSI: 0000000000000282 RDI: 0000000000000000 [ 1826.810636] RBP: ffff88022d857978 R08: 0000000000000001 R09: ffff88022d857958 [ 1826.810694] R10: 0000000000000003 R11: 0000000000000001 R12: 00000000000f2010 [ 1826.810753] R13: ffff880212cb0000 R14: ffff880212cb0000 R15: ffff880212cb0000 [ 1826.810812] FS: 00007f5253bfd950(0000) GS:ffff88022f1fa380(0000) knlGS:0000000000000000 [ 1826.810887] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1826.810943] CR2: 00000000b7eb2044 CR3: 0000000212cac000 CR4: 00000000000026a0 [ 1826.811002] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1826.811061] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1826.811120] Process qemu-system-x86 (pid: 9353, threadinfo ffff88022d856000, task ffff88022e0cd980) [ 1826.811196] Stack: [ 1826.811246] ffff88022d857968 0000000000000004 0000000000000004 0000000000000000 [ 1826.811401] ffff88022d8579b8 ffffffffa01e7ccf ffff88022d8579b8 00000000f2010073 [ 1826.811634] 0000000000000004 ffff880212cb15b0 000000001f402b00 ffff880212cb0000 [ 1826.811913] Call Trace: [ 1826.811964] [<ffffffffa01e7ccf>] mmu_guess_page_from_pte_write+0xaf/0x190 [kvm] [ 1826.812076] [<ffffffffa01e820f>] kvm_mmu_pte_write+0x3f/0x4f0 [kvm] [ 1826.812172] [<ffffffffa01da9f1>] ? mark_page_dirty+0x11/0x90 [kvm] [ 1826.812268] [<ffffffffa01dabe8>] ? kvm_write_guest+0x48/0x90 [kvm] [ 1826.812364] [<ffffffffa01de427>] emulator_write_phys+0x47/0x70 [kvm] [ 1826.812460] [<ffffffffa01e0e26>] emulator_write_emulated_onepage+0x66/0x120 [kvm] [ 1826.812571] [<ffffffffa01e0f50>] emulator_write_emulated+0x70/0x90 [kvm] [ 1826.812668] [<ffffffffa01eb36f>] x86_emulate_insn+0x4ef/0x32e0 [kvm] [ 1826.812764] [<ffffffffa01e950e>] ? do_insn_fetch+0x8e/0x100 [kvm] [ 1826.812860] [<ffffffffa01e9454>] ? seg_override_base+0x24/0x50 [kvm] [ 1826.812955] [<ffffffffa01eacb0>] ? x86_decode_insn+0x7a0/0x970 [kvm] [ 1826.813051] [<ffffffffa01e221f>] emulate_instruction+0x15f/0x2f0 [kvm] [ 1826.813148] [<ffffffffa01e7bd5>] kvm_mmu_page_fault+0x65/0xb0 [kvm] [ 1826.813243] [<ffffffffa020ac5f>] handle_exception+0x2ef/0x360 [kvm_intel] [ 1826.813338] [<ffffffffa01eb0a3>] ? x86_emulate_insn+0x223/0x32e0 [kvm] [ 1826.813434] [<ffffffffa0209c25>] kvm_handle_exit+0xb5/0x1d0 [kvm_intel] [ 1826.813526] [<ffffffff80699643>] ? __down_read+0xc3/0xce [ 1826.813618] [<ffffffffa01dd958>] vcpu_enter_guest+0x1f8/0x400 [kvm] [ 1826.813714] [<ffffffffa01dfc29>] __vcpu_run+0x69/0x2d0 [kvm] [ 1826.813751] [<ffffffffa01e38ea>] kvm_arch_vcpu_ioctl_run+0x8a/0x1f0 [kvm] [ 1826.813751] [<ffffffffa01d8582>] kvm_vcpu_ioctl+0x2e2/0x5a0 [kvm] [ 1826.813751] [<ffffffff802f6091>] vfs_ioctl+0x31/0xa0 [ 1826.813751] [<ffffffff802f6445>] do_vfs_ioctl+0x75/0x230 [ 1826.813751] [<ffffffff802e8216>] ? generic_file_llseek+0x56/0x70 [ 1826.813751] [<ffffffff802f6699>] sys_ioctl+0x99/0xa0 [ 1826.813751] [<ffffffff802e70d2>] ? sys_lseek+0x52/0x90 [ 1826.813751] [<ffffffff8021253a>] system_call_fastpath+0x16/0x1b [ 1826.813751] Code: 00 00 65 48 8b 04 25 00 00 00 00 48 8b b8 38 02 00 00 48 83 c7 60 e8 dd 23 09 e0 48 89 df e8 45 fe ff ff 85 c0 0f 85 08 ff ff ff <0f> 0b eb fe 66 0f 1f 84 00 00 00 00 00 55 65 8b 14 25 24 00 00 [ 1826.813751] RIP [<ffffffffa01da853>] gfn_to_pfn+0x153/0x160 [kvm] [ 1826.813751] RSP <ffff88022d857958> [ 1826.816899] ---[ end trace 2437a1197b66fb45 ]--- Stephen. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
