On Tue, Jul 28, 2009 at 8:54 PM, Avi Kivity<avi@xxxxxxxxxx> wrote: > On 07/28/2009 12:32 AM, Stephen Donnelly wrote: >>>> >>>> What I don't understand is how to turn the host address returned from >>>> mmap into a ram_addr_t to pass to pci_register_bar. >>> >>> Memory must be allocated using the qemu RAM functions. >> >> That seems to be the problem. The memory cannot be allocated by >> qemu_ram_alloc, because it is coming from the mmap call. The memory is >> already allocated outside the qemu process. mmap can indicate where in >> the qemu process address space the local mapping should be, but >> mapping it 'on top' of memory allocated with qemu_ram_alloc doesn't >> seem to work (I get a BUG in gfn_to_pfn). > > You need a variant of qemu_ram_alloc() that accepts an fd and offset and > mmaps that. Okay, it sounds like a function to do this is not currently available. That confirms my understanding at least. I will take a look but I don't think I understand the memory management well enough to write this myself. > A less intrusive, but uglier, alternative is to call > qemu_ram_alloc() and them mmap(MAP_FIXED) on top of that. I did try this, but ended up with a BUG on the host in /var/lib/dkms/kvm/84/build/x86/kvm_main.c:1266 gfn_to_pfn() on the line "BUG_ON(!kvm_is_mmio_pfn(pfn));" when the guest accesses the bar. [1847926.363458] ------------[ cut here ]------------ [1847926.363464] kernel BUG at /var/lib/dkms/kvm/84/build/x86/kvm_main.c:1266! [1847926.363466] invalid opcode: 0000 [#1] SMP [1847926.363470] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.5/0000:02:00.0/net/eth0/statistics/collisions [1847926.363473] Dumping ftrace buffer: [1847926.363476] (ftrace buffer empty) [1847926.363478] Modules linked in: softcard_driver(P) nls_iso8859_1 vfat fat usb_storage tun nls_utf8 nls_cp437 cifs nfs lockd nfs_acl sunrpc binfmt_misc ppdev bnep ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm video output input_polldev dm_crypt sbp2 lp parport snd_usb_audio snd_pcm_oss snd_hda_intel snd_mixer_oss snd_pcm snd_seq_dummy snd_usb_lib snd_seq_oss snd_seq_midi snd_seq_midi_event uvcvideo compat_ioctl32 snd_rawmidi snd_seq iTCO_wdt videodev snd_timer snd_seq_device iTCO_vendor_support ftdi_sio usbhid v4l1_compat snd_hwdep intel_agp nvidia(P) usbserial snd soundcore snd_page_alloc agpgart pcspkr ohci1394 ieee1394 atl1 mii floppy fbcon tileblit font bitblit softcursor [last unloaded: softcard_driver] [1847926.363539] [1847926.363542] Pid: 31516, comm: qemu-system-x86 Tainted: P (2.6.28-13-generic #44-Ubuntu) P5K [1847926.363544] EIP: 0060:[<f7f5961f>] EFLAGS: 00010246 CPU: 1 [1847926.363556] EIP is at gfn_to_pfn+0xff/0x110 [kvm] [1847926.363558] EAX: 00000000 EBX: 00000000 ECX: f40d30c8 EDX: 00000000 [1847926.363560] ESI: d0baa000 EDI: 00000001 EBP: f2cddbbc ESP: f2cddbac [1847926.363562] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [1847926.363564] Process qemu-system-x86 (pid: 31516, ti=f2cdc000 task=f163d7f0 task.ti=f2cdc000) [1847926.363566] Stack: [1847926.363567] f2cddbb0 f2cddbc8 00000000 000f2010 f2cddc7c f7f65f00 00000004 f2cddbd4 [1847926.363573] f7f5829f 00000004 f2cddbf4 f7f582ec 00000df4 00000004 d0baa000 f185a370 [1847926.363579] df402c00 0001f719 f2cddc4c f7f66858 f2cddc40 00000004 0001f95f 00000000 [1847926.363585] Call Trace: [1847926.363588] [<f7f65f00>] ? kvm_mmu_pte_write+0x160/0x9a0 [kvm] [1847926.363598] [<f7f5829f>] ? kvm_read_guest_page+0x2f/0x40 [kvm] [1847926.363607] [<f7f582ec>] ? kvm_read_guest+0x3c/0x70 [kvm] [1847926.363616] [<f7f66858>] ? paging32_walk_addr+0x118/0x2d0 [kvm] [1847926.363625] [<f7f59360>] ? mark_page_dirty+0x10/0x70 [kvm] [1847926.363634] [<f7f59412>] ? kvm_write_guest_page+0x52/0x60 [kvm] [1847926.363643] [<f7f5becf>] ? emulator_write_phys+0x4f/0x70 [kvm] [1847926.363652] [<f7f5dcc8>] ? emulator_write_emulated_onepage+0x58/0x130 [kvm] [1847926.363661] [<f7f5ddf9>] ? emulator_write_emulated+0x59/0x70 [kvm] [1847926.363674] [<f7f69d84>] ? x86_emulate_insn+0x414/0x2650 [kvm] [1847926.363684] [<c011f714>] ? handle_vm86_fault+0x4c4/0x740 [1847926.363690] [<c011f714>] ? handle_vm86_fault+0x4c4/0x740 [1847926.363699] [<f7f681e6>] ? do_insn_fetch+0x76/0xd0 [kvm] [1847926.363712] [<c011f716>] ? handle_vm86_fault+0x4c6/0x740 [1847926.363715] [<c011f716>] ? handle_vm86_fault+0x4c6/0x740 [1847926.363719] [<f7f6909a>] ? x86_decode_insn+0x54a/0xe20 [kvm] [1847926.363732] [<f7f5ecfc>] ? emulate_instruction+0x12c/0x2a0 [kvm] [1847926.363741] [<f7f65988>] ? kvm_mmu_page_fault+0x58/0xa0 [kvm] [1847926.363750] [<f7e8797a>] ? handle_exception+0x35a/0x400 [kvm_intel] [1847926.363755] [<f7e83e97>] ? handle_interrupt_window+0x27/0xc0 [kvm_intel] [1847926.363760] [<c011f714>] ? handle_vm86_fault+0x4c4/0x740 [1847926.363763] [<f7e864e9>] ? kvm_handle_exit+0xd9/0x270 [kvm_intel] [1847926.363768] [<f7e87c87>] ? vmx_vcpu_run+0x137/0xa4a [kvm_intel] [1847926.363772] [<f7f6d767>] ? kvm_apic_has_interrupt+0x37/0xb0 [kvm] [1847926.363781] [<f7f6c0b7>] ? kvm_cpu_has_interrupt+0x27/0x40 [kvm] [1847926.363790] [<f7f61306>] ? kvm_arch_vcpu_ioctl_run+0x626/0xb20 [kvm] [1847926.363799] [<c015da68>] ? futex_wait+0x358/0x440 [1847926.363804] [<f7f576e5>] ? kvm_vcpu_ioctl+0x395/0x490 [kvm] [1847926.363812] [<c04fec68>] ? _spin_lock+0x8/0x10 [1847926.363815] [<c015d508>] ? futex_wake+0xc8/0xf0 [1847926.363819] [<f7f57350>] ? kvm_vcpu_ioctl+0x0/0x490 [kvm] [1847926.363827] [<c01ca1d8>] ? vfs_ioctl+0x28/0x90 [1847926.363831] [<c01ca6be>] ? do_vfs_ioctl+0x5e/0x200 [1847926.363834] [<c01ca8c3>] ? sys_ioctl+0x63/0x70 [1847926.363836] [<c0103f6b>] ? sysenter_do_call+0x12/0x2f [1847926.363840] Code: 29 d3 c1 eb 0c 03 58 44 64 a1 00 e0 7a c0 8b 80 cc 01 00 00 83 c0 34 e8 b0 9b 1f c8 89 d8 e8 89 fc ff ff 85 c0 0f 85 50 ff ff ff <0f> 0b eb fe 8d b6 00 00 00 00 8d bc 27 00 00 00 00 55 89 e5 e8 [1847926.363873] EIP: [<f7f5961f>] gfn_to_pfn+0xff/0x110 [kvm] SS:ESP 0068:f2cddbac [1847926.363885] ---[ end trace 314ce851a956cf3c ]--- pseudo code in my pci init function is: { offset = qemu_ram_alloc(64*1024); ptr = qemu_get_ram_ptr(offset); fd = open(charfile, O_RDWR); mmap(ptr, 64*1024, PROT_READ | PROT_WRITE, MAP_SHARED|MAP_FIXED, fd, 0)) pci_register_bar((PCIDevice *)d, 0, 1024*64, PCI_ADDRESS_SPACE_MEM, mmio_map); } mmio_map() { cpu_register_physical_memory(addr + 0, 64*1024, offset); } Regards, Stephen. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html