Just for posterity, my ultimate solution was to patch nova on each compute host to always return True in _check_shared_storage_test_file (nova/virt/libvirt/driver.py) This did make migration work with "nova live-migration", with one caveat. Since Nova is assuming that /var/lib/nova/instances is on shared storage (and since I hard coded the check to say "yes, it really is"), it thinks the /var/lib/nova/instances/<domain> folder will exist at both source and destination, and makes no attempt to create it on the destination. So before I run live-migration, I pop over to the source host, and rsync that folder to the destination. A little dirty, but it allows me to move running VMs around just fine in cases of maintenance on a host, which is exactly what I need. Thanks for everyone's feedback. - Travis On Tue, Mar 12, 2013 at 6:33 PM, Travis Rhoden <trhoden@xxxxxxxxx> wrote: > > On Tue, Mar 12, 2013 at 5:06 PM, Josh Durgin <josh.durgin@xxxxxxxxxxx> > wrote: >> >> On 03/12/2013 01:48 PM, Travis Rhoden wrote: >>> >>> Hi Josh, >>> >>> Thanks for the info. So if I want to do live migration with VMs that >>> were >>> launched with boot-from-volume, I'll need to use virsh to do the >>> migration, >>> rather than Nova. Okay, that should be doable. As an aside, I will >>> probably want to look at the OpenStack DB and figure out how to tell it >>> that the VM has moved to a different host. I'd rather there not be a >>> disconnect between Nova and libvirt about where the VM lives. =) >> >> >> It's probably not too hard to edit nova to skip the checks when the >> instance is volume-backed, but if you don't want to do that, libvirt >> should be fine, and a bit more flexible. > > > After messing with it for a few hours, I'm thinking about doing just that. > The nova edits should be easy. Looks like it tests for shared storage by > writing a file on the migration destination, and trying to read it at the > source. I should be able to just comment out the check entirely, or make > the check always pass. > > The virsh migrate strategy has been surprisingly difficult. Since I'm > migrating a Nova VM, I had to the following pre-requisites (so far). > > Define the /var/lib/nova/instance/<domain> dir on the destination > Define/migrate the nova libvirt-nwfilter for the specific VM > > Then, when I try to do the actual migration, I always get (at the source): > > error: internal error Process exited while reading console log output: > chardev: opening backend "file" failed: Permission denied > > So QEMU is bailing, saying it can't read the console.log file. When I go > look at that file, it is created, but with owner root:root and perms 0600. > However, libvirtd makes it libvirt-qemu:kvm, 0600 before KVM tries to > actually start the VM. I've always found this dynamic file ownership bit in > KVM/libvirt/qemu very confusing. Anyways, I tried a few different things, > debug logging, etc. Even tried disabling apparmor. Still get permission > denied each time. > > The commands I"m running manually should be identical to what OpenStack is > doing, so I can't figure out why their migrate is working and mine wouldn't. > Oh well, will edit Nova and give that shot. > > - Travis _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |