I am trying to integrate Gluster (3.4.0-8) with a RDO deployment (Havana pre-release) of OpenStack with Cinder. I have a simple 3 node depolyment (admin host, network host, hypervisor host). The cinder-volume is running on the admin host and has mounted the glusterfs volume successfully. Excuse me if I am confused on some of these points but the documentation on Gluster and RHOS doesn't seem to cover the exact specifics of all these components interactions. So I can get Cinder and Gluster integrated enough to be able to create volumes, delete volumes etc. My understanding is that from the cinder-volume node it will take that volume and export it to the instance via iSCSI using the tgtd daemons. The thing is that when I attach the volume I get a local libvirt error below[1]. It seems that libvirt is trying to attach a locally mounted volume file. This is not what I expected and I was wondering if anyone has done this integration that might share some insights. [1] 2013-09-26 20:03:43.320 8718 ERROR nova.openstack.common.rpc.amqp [req-acb8d4fa-ab55-45e6-934f-5e723a958acd 92741ec5808647f787a7ea8317d1ca43 064d1904dec8428da2e30e3726b9b943] Exception during message handling 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp **args) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 89, in wrapped 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp payload) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp pass 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info()) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3575, in attach_volume 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp context, instance, mountpoint) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3570, in attach_volume 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp mountpoint, instance) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3612, in _attach_volume 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp connector) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3602, in _attach_volume 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp mountpoint) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1072, in attach_volume 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp disk_dev) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1059, in attach_volume 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp virt_dom.attachDeviceFlags(conf.to_xml(), flags) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 187, in doit 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp result = proxy_call(self._autowrap, f, *args, **kwargs) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 147, in proxy_call 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp rv = execute(f,*args,**kwargs) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 76, in tworker 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp rv = meth(*args,**kwargs) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/site-packages/libvirt.py", line 419, in attachDeviceFlags 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp libvirtError: cannot open file '/var/lib/nova/mnt/0fb9dadec76b71926e1cdb9ca60e1dfe/volume-da84a50f-c0f3-4f55-91e4-6cd3bb5ddc6a': No such file or directory -- --- Derek T. Yarnell University of Maryland Institute for Advanced Computer Studies