Hi Daniel; Thanks for the quick response and for your help, this is working now. If you are interested in the gory details they are detailed below, at least I'll document them here for the sake of others on the list. See "Gory Details - start" While I have your attention I have another question if you don't mind. I have done all this because I wanted to see if I could use virsh to live migrate a VM between physical hosts. Actually I wanted to use libvirt but virsh is my easiest way to talk to libvirt. My two physical xen servers are connecting to the same iSCSI SAN LUN formatted and mounted with ocfs2. I have a PV vm residing on the LUN (centos 5.1 - 2.6.8.18-xen). If I use standard xen config files and don't "define" my guest using virsh (xml) I can use the XM commands to migrate without any trouble. If I try to "define" the guest and then use XM to migrate it everything gets confused. So I thought I would "define" the guest and then try to use virsh to migrate. The same confusion takes place. I just did this and got some weird behaviour. I started the VM "centpv" up on host2 and then asked it to migrate to host1 using virsh from the (now working) client. The migration worked as expected. I was able to disconnect my VNC connection to the VM on host2 and connect it to the machine on host1. This where it gets weird. Host2 (Source) virsh # list --all Id Name State ---------------------------------- 0 Domain-0 running - centpv shut off - xi-ad-01 shut off - xi3mps01h shut off Host 1 (destination) virsh # list --all Id Name State ---------------------------------- 0 Domain-0 running 1 centpv blocked As you can see centpv is now listed on both machines. The state on host2 is shutoff which is technically correct. The VM is actually running on host 1. Host 1 has it as running. If you look at virt-manager you get an even more confusing story. Host1 says that centpv is "shutoff" until you highlight the machine then it says its running, and it quickly switches to saying it's shutoff. Host 2 claims the machine is still running. Killing virt-manager and re spawning it helps somewhat in that now host1 knows the machine is running. Host 2 thinks its shutdown. (Reflecting what is listed by virsh). The next thing I did was to "virsh undefine centpv" from host2. That also worked. Now all is right with the world. How can I find out more about how the "define" command works? (read the source code?) If the machine is "defined" doesn’t this place a hard entry in the local xenstore on the Xen machine on which it was defined?. I have to go read up on xenstore as I'm not that familiar with it yet. Do we need to migrate and then undefine the machine once the machine lands at the destination? Finally could these issues be related to the way virsh handles the libvirt "migrate" or is this a problem with libvirt? My next plan is to work up a python script that will talk directly to libvirt and execute a migration. This will take me some time as I don't know python... but let's see what happens. In the interest of time (and my sanity) does anyone have any basic python script that talk directly to libvirt that they would be willing to share? Thanks in advance. Gory Details - start. Your suggestion to use Virsh in the first instance was key. >From the Ubuntu workstation when I tried to connect to vxen-01 using virsh I immediately got an error relating to the missing CA.crt (which we both suspected was an issue). After copying the root cert to the correct location I was given a different error message: libvir: QEMU error : internal error cannot create bridge 'virbr0' : File exists >From which I googled my way to the following post: http://bbs.archlinux.org/viewtopic.php?id=5147 So I checked iptables and despite the fact that I don't remember configuring it, there were rules present. I think libvirtd does some stuff to iptables. After shutting down iptables and kicking the libvirtD I stopped getting that error. Now both virsh and virt-manager are working using TLS. Strangely the URI I had to use for virt-manager and virsh were different. Virt-manager URI = xen://vxen-01.domain.com Virsh URI = xen://vxen-01.domain.com/ Virsh needed the trailing "/". Without it I get console messages (on the server) that say "libvir: error : invalid argument in could not parse connection URI". I have typed it exactly, that grammar problem is not a typo, that's what it says. Best Regards -----Original Message----- From: libvir-list-bounces@xxxxxxxxxx [mailto:libvir-list-bounces@xxxxxxxxxx] On Behalf Of Daniel P. Berrange Sent: 24 July 2008 12:19 To: Fedora/Linux Management Tools Cc: libvir-list@xxxxxxxxxx Subject: [libvirt] Re: [et-mgmt-tools] Virt-Manager, libvirt & TLS On Thu, Jul 24, 2008 at 11:13:28AM +0100, Geoff Wiener wrote: > Hi! > > > > This is my first post to either of these list, I have been lurking, > (sorry to cross post but I don't know if this is a virt-manager or > libvirt question). So first off thank you to everyone for all your > efforts. I think libvirt and virt-manager are excellent! I've built > a pair of server s in the lab with a Xen stack and have been attempting > to get virt-manager 0.5.4 to communicate with, first libvirt 0.4.2 and > then libvirt 0.4.4 using TLS across the network in a "client / server" > configuration unsuccessfully. All the machines are on the same subnet > (192.168.4.x/24). I can make Virt-Manager communicate with Libvirt > over TCP without authentication so now that I know the installation > works I want to further secure it using TLS. > > /usr/local/etc/libvirt/libvirtd.conf > > > > Listen_tcp = 1 > > auth_unix_ro = "none" > > auth_unix_rw="none" > > auth_tcp="none" That's all fine. > I followed the configuration notes at: http://libvirt.org/remote.html with a couple of exceptions: > > 1. I already have a linux based CA that I use with OpenVPN so I used that CA root certificate and just generated client and server cert / key pairs for my client and server (I tested with just one server) That's fine - any CA will do the job. > 2. I reverted back to the default libvirtd.conf to setup for TLS and > noticed that the default paths for the certificate locations were not in > line with the documentation on the web page but there were commented sections > as follows that matched the documentation, so I uncommented them: > > key_file = "/etc/pki/libvirt/private/serverkey.pem" > cert_file = "/etc/pki/libvirt/servercert.pem" > ca_file = "/etc/pki/CA/cacert.pem" No need to uncomment any of these - its fine to use the the default settings built-in to libvirt > > #crl_file = "/etc/pki/CA/crl.pem" > Note: I did not uncomment the CRL_FILE path as I do not want to use a CRL at this time Ok, no problem there. > 3. On the server I execute "libvirtd -listen -verbose" (libvirtd output) attached > > 4. virt-manager 0.5.4 (as root) , File, Open Connection > Hypervisor: Xen > > Connection: Remote SSL/TLS with x509 certificate > > Hostname: vxen-01.aenigmacorp.com (I have a host entry for this machine) > > > > The virt-manager console reports "unable to open a connection to the libvirt > management daemon". Verify that the "libvirtd" daemon has been started. Then, > in details there is a lot of info (see virt-manager output) I'd recommend getting it working using virsh as a client first - this gives clearer diagnostics. Once virsh is working, then virt-manager should just work too, although it has an extra step required for VNC access. > That about sums it up. I have not read any instructions that ask me to copy > the CA root certificate to the client, is that required? And if so where would > I put it. Yes, the CA certificate needs to be on all machines - in the same location as for the server - /etc/pki/CA/cacert.pem. The client server needs to be in the loication /etc/pki/libvirt/clientcert.pem There are some additional docs on the virt-manager wiki about the VNC setup steps too http://virt-manager.org/page/RemoteTLS Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- Libvir-list mailing list Libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list -- Libvir-list mailing list Libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list