i try your proposal. but there are no any output in qemu-rbd.log. selinux has been disabled(then reboot my OS).
i do not know apparmor. my OS has no rpm of apparmor.
my os is centps-6.2. 2.6.32-220.el6.x86_64.
1.would you give me the options of libvirt configure. i rebuild it.
2.if there a other way to confirm whether libvirt support rbd.other way to find root cause.
thanks.
----rbd disk----
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writethrough' io='native'/>
<auth username='admin'>
<secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/>
! ; </auth>
<source protocol='rbd' name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=20:debug_auth=20:log_file=/tmp/qemu-rbd.log'>
<host name='192.168.10.4' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
<driver name='qemu' type='raw' cache='writethrough' io='native'/>
<auth username='admin'>
<secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/>
! ; </auth>
<source protocol='rbd' name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=20:debug_auth=20:log_file=/tmp/qemu-rbd.log'>
<host name='192.168.10.4' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
--------------------
At 2012-10-12 15:06:45,"Josh Durgin" <josh.durgin@xxxxxxxxxxx> wrote: >Sorry, I forgot libvirt/qemu eats stderr. The extra logging wasn't >captured. > >Could you try: > >name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_file=/tmp/qemu-rbd.log' > >If /tmp/qemu-rbd.log isn't full of debug messages, make sure selinux or >apparmor aren't preventing >qemu from writing to it. > >Josh > >On 2012-10-11 22:31, yue wrote: >> hi, i add debug options related to rbd in xpSP3.xml. but error seems >> same with privious. >> ----libvirtd.log------------------ >> erfaces >> 2012-10-12 13:25:28.267+0000: 9023: error : virNodeNumOfDevices:13984 >> : this function is not supported by the connection driver: >> virNodeNumOfDevices >> 2012-10-12 13:25:29.264+0000: 9025: error : >> virConnectNumOfInterfaces:11014 : this function is not supported by >> the connection driver: virConnectNumOfInterfaces >> 2012-10-12 13:25:29.266+0000: 9023: error : >> virConnectNumOfDefinedInterfaces:11098 : this function is not >> supported by the connection driver: virConnectNumOfDefinedInterfaces >> 2012-10-12 13:25:29.267+0000: 9022: error : virNodeNumOfDevices:13984 >> : this function is not supported by the connection driver: >> virNodeNumOfDevices >> 2012-10-12 13:25:30.264+0000: 9025: error : >> virConnectNumOfInterfaces:11014 : this f unction is not supported by >> the connection driver: virConnectNumOfInterfaces >> 2012-10-12 13:25:30.267+0000: 9023: error : >> virConnectNumOfDefinedInterfaces:11098 : this function is not >> supported by the connection driver: virConnectNumOfDefinedInterfaces >> 2012-10-12 13:25:30.269+0000: 9022: error : virNodeNumOfDevices:13984 >> : this function is not supported by the connection driver: >> virNodeNumOfDevices >> 2012-10-12 13:25:30.523+0000: 9021: warning : qemuDomainObjTaint:1364 >> : Domain id=1 name='xpSP3' uuid=b45bd66a-6700-4905-c5f4-4c799413d7b7 >> is tainted: high-privileges >> 2012-10-12 13:25:30.943+0000: 9021: error : qemuMonitorOpenUnix:293 : >> failed to connect to monitor socket: No such process >> 2012-10-12 13:25:30.943+0000: 9021: info : qemuConnectMonitor:1247 : >> Failed to connect monitor for xpSP3 >> 2012-10-12 13:25:30.943+0000: 9021: error : >> qemuProcessWaitForMonitor:1634 : internal error process exited while >> connecting to monitor: char device redirected to /dev/pts/4 >> qemu-kvm: -drive >> >> file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: >> could not open disk image >> >> rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: >> No such file or directory >> >> ---------xpSP3.log---is attached.--- >> 2012-10-12 13:11:33.133+0000: starting up >> LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root >> LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S >> -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 >> -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults >> -chardev socket,id=charmonitor,path=/var/lib/libvirt/ >> qemu/xpSP3.monitor,server,nowait -mon >> chardev=charmonitor,id=monitor,mode=readline -rtc >> base=localtime,driftfix=slew -device >> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive >> >> file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none >> -device >> >> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 >> -drive >> >> file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native >> -device >> >> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0 >> -netdev tap,fd=19,id=hostnet0 -device >> >> e1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 >> -chardev pty,id=charserial0 -device >> isa-serial,chardev=charserial0,id=serial0 -device >> usb-tablet,id=input0 >> -vnc 0.0.0.0:0 -vga std -device >> intel-hda,id=sound0,bus=pci.0,addr=0x4 >> -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 >> Domain id=1 is tainted: high-privileges >> 2012-10-12 13:11:33.144+0000: 7815: debug : virFileClose:72 : Closed >> fd 18 >> 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed >> fd 24 >> 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.146+0000: 7818: debug : virCommandHook:2058 : Run >> hook 0x7faa10a62030 0x7faa160d4370 >> 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2589 : >> Obtaining domain lock >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virDomainLockProcessStart:164 : plugin=0x7faa080020a0 >> dom=0x7faa0804af40 paused=1 fd=0x7faa160d3998 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virDomainLockManagerNew:123 : plugin=0x7faa080020a0 >> dom=0x7faa0804af40 >> withResources=1 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virLockManagerPluginGetDriver:280 : plugin=0x7faa080020a0 >> 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerNew:304 : >> driver=0x3712e9cb20 type=0 nparams=4 params=0x7faa160d3830 flags=0 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virLockManagerLogParams:104 : key=uuid type=uuid >> value=b45bd66a-6700-4905-c5f4-4c799413d7b7 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virLockManagerLogParams:100 : key=name type=string value=xpSP3 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virLockManagerLogParams:88 : key=id type=uint value=1 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virLockManagerLogParams:88 : key=pid type=uint value=7818 >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virDomainLockManagerNew:135 : Adding leases >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virDomainLockManagerNew:140 : Adding disks >> 2012-10-12 13:11:33.146+0000: 7818: debug : >> virDomainLockManagerAddDisk:86 : Add disk >> /var/lib/libvirt/images/xpSP3.img >> 2012-10-12 13:11:33.146+0000: 7818: debug : virLockMan >> agerAddResource:333 : lock=0x7faa08060820 type=0 >> name=/var/lib/libvirt/images/xpSP3.img nparams=0 params=(nil) flags=0 >> 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAcquire:350 >> : lock=0x7faa08060820 state='(null)' flags=3 fd=0x7faa160d3998 >> 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerFree:387 : >> lock=0x7faa08060820 >> 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2614 : >> Moving process to cgroup >> 2012-10-12 13:11:33.146+0000: 7818: debug : virCgroupNew:617 : New >> group /libvirt/qemu/xpSP3 >> 2012-10-12 13:11:33.151+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/mapping 0:cpu at /cgroup/cpu in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/mapping 2:cpuset at /cgroup/cpuset in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/ma pping 3:memory at /cgroup/memory in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/mapping 4:devices at /cgroup/devices in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/mapping 5:freezer at /cgroup/freezer in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : >> Detected mount/mapping 6:blkio at /cgroup/blkio in >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:535 : >> Make group /libvirt/qemu/xpSP3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/cpu/libvirt/qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/cpuacct/libvirt/qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/cpuset/libvirt/qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/memory/libvirt/ qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/devices/libvirt/qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/freezer/libvirt/qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : >> Make controller /cgroup/blkio/libvirt/qemu/xpSP3/ >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/cpu/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/cpuacct/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/cpuset/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/memory/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/devices/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/freezer/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 >> : Set value '/cgroup/blkio/libvirt/qemu/xpSP3/tasks' to '7818' >> 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed >> fd 3 >> 2012-10-12 13:11:33.156+0000: 7818: debug : >> qemuProcessInitCpuAffinity:1848 : Setting CPU affinity >> 2 012-10-12 13:11:33.156+0000: 7818: debug : >> qemuProcessInitCpuAffinity:1883 : Set CPU affinity with specified >> cpuset >> 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessHook:2626 : >> Setting up security labelling >> 2012-10-12 13:11:33.157+0000: 7818: debug : virGetUserIDByName:2539 : >> User record for user '0' does not exist >> 2012-10-12 13:11:33.157+0000: 7818: debug : virGetGroupIDByName:2621 >> : Group record for group '0' does not exist >> 2012-10-12 13:11:33.157+0000: 7818: debug : >> virSecurityDACSetProcessLabel:861 : Dropping privileges of DEF to 0:0 >> 2012-10-12 13:11:33.157+0000: 7818: debug : qemuProcessHook:2633 : >> Hook complete ret=0 >> 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2060 : >> Done hook 0 >> 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2074 : >> Notifying parent for handshake start on 21 >> 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2098 : >> Waiting on parent for handshake complete on 22 >> 2012-10-12 13:11:33.157+0 000: 7818: debug : virFileClose:72 : Closed >> fd 21 >> 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed >> fd 22 >> 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2119 : >> Hook is done 0 >> char device redirected to /dev/pts/4 >> qemu-kvm: -drive >> >> file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: >> could not open disk image >> >> rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: >> No such file or directory >> 2012-10-12 13:11:33.557+0000: shutting down >> >> ---- >> >> At 2012-10-12 00:04:19,"Josh Durgi >> n" <josh.durgin@xxxxxxxxxxx> wrote: >>>On 10/11/2012 12:00 AM, yue wrote: >>>> how should i do? i configurate guest-os.xml ,add rbd disk manually. >>>> then virsh start xpSP3 >>> >>>That's the right way to do it. >>> >>>> -------rbd disk------------ >>>> <disk type='network' device='disk'> >>>> <driver name='qemu' type='raw' cache='writethrough' io='native'/> >>>> <auth username='cloud'> >>>> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> >>>> </auth> >>>> <source protocol='rbd' name='cloud/testrbd'> >>>> <host name='192.168.10.4' port='6789'/> >>>> </source> >>>> <target dev='vdb' bus='virtio'/> >>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' >>>> function='0x0'/> >>>> </disk> >>>> ---error log--- >>>> >>>> [root@KVM-ip11 git]# virsh start xpSP3 >>>> error: Failed to start domain xpSP3 >>>> error: internal error process exited while connecting to monitor: >>>> char device redirected to /dev/pts/4 >>>> qemu-kvm: -drive >>>> file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: >>>> could not open disk image >>>> rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: >>>> No such file or directory >>>> ---xpSP3.log----- >>>> 2012-10-11 14:57:58.118+0000: starting up >>>> LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root >>>> LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S >>>> -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 >>>> -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults >>>> -chardev >>>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait >>>> -mon chardev=charmonitor,id=monitor,mode=readline -rtc >>>> base=localtime,driftfix=slew -device >>>> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive >>>> file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none >>>> -device >>>> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 >>>> -drive >>>> file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789, >> if=none,id=drive-virtio-disk1,format=raw -device >> >> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 >> -netdev tap,fd=19,id=hostnet0 -device e1000,netdev! >>>> =hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev >>>> pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 >>>> -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device >>>> intel-hda,id=sound0,bus=pci.0,addr=0x4 -device >>>> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device >>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 >>>> Domain id=3 is tainted: high-privileges >>>> char device redirected to /dev/pts/4 >>>> qemu-kvm: -drive >>>> file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw: >>>> could not open disk image >>>> rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: >>>> No such file or directory >>>> 2012-10-11 14:57:58.548+0000: shutting down >>> >>>It's unclear where the actual problem is - could you try again with >>>name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' >>>and post the instance log (i.e. xpSP3.log)? >>> >>>Josh >>> >>>> >>>> >>>> >>>> >>>> >>>> At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@xxxxxxxxxxx> >>>> wrote: >>>>> On 10/10/2012 10:52 PM, yue wrote: >>>>>> hi, now i rebuild libvirtd, version 0.10.2. and define ceph >>>>>> secret >>>>>> ,define ceph storage-pool. all fine up to now. >>>>>> i operate guest-os through virt-manager-GUI . i add a >>>>>> rbd-file(qemu-img create before) from ceph-rbd-pool, click >>>>>> 'finish',it >>>>>> pop error "NoneType object has no attribute startswith", someone >>>>>> say it >>>>>> is python error. >>>>> >>>>> python-virtinst (and thus virt-manager) don't support the rbd >>>>> storage >>>>> pool. They don't know how to configure rbd disks yet either. >>>>> >>>>> Josh >>> >>>
-- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list