|
I'm trying to setup live migration in Openstack using Ceph RBD backed volumes. From what I understand I also need to put the libvirt folder /var/lib/nova/instances on shared storage for it to work as Nova tests for this as part of the migration process. I decided to look at using CephFS for this purpose.
I've created a CephFS file system and mounted it on my two compute nodes as /var/lib/nova/instances but I'm getting some strange results. It seems like after one of the hosts accesses a file then it becomes locked and the other host can't access it. For example, I complete the following steps: 1\ Create new Openstack instance using the boot from image (create new volume) option. New instance is created with RBD backed volume as expected. On the relevant compute host I see the instance folder created under /var/lib/nova/instances/{instance id} with two files inside libvirt.xml and console.log. If I cat the libvirt.xml file it works as expected. 2\ Live migrate the instance to other host. Appears to work as expected although instance status in Horizon stays as migrating forever. I can see instance has moved to second host by running virsh list on both hosts. 4\ Now, if I attempt to cat the libvirt.xml file on the new host I get "Operation not permitted". I'm assuming this isn't what's expected? To test this further I tried the following basic tests: On Host 2: root@devops-kvm02:/var/lib/nova/instances# echo hello > test root@devops-kvm02:/var/lib/nova/instances# cat test hello root@devops-kvm02:/var/lib/nova/instances# Then from Host 1: root@devops-kvm01:/var/lib/nova/instances# cat test cat: test: Operation not permitted root@devops-kvm01:/var/lib/nova/instances# Then back on Host 2: root@devops-kvm02:/var/lib/nova/instances# cat test cat: test: Operation not permitted root@devops-kvm02:/var/lib/nova/instances# Should this even work? My understanding is CephFS allows concurrent access but I'm not sure if there is some file locking going on that I need to understand. Thanks, Neville |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
