Hi Josh, I am trying to use ceph with openstack (grizzly), I have a multi host setup. I followed the instruction http://ceph.com/docs/master/rbd/rbd-openstack/. Glance is working without a problem. With cinder I can create and delete volumes without a problem. But I cannot boot from volumes. I doesn't matter if use horizon or the cli, the vm goes to the error state. >From the nova-compute.log I get this. 2013-05-30 16:08:45.224 ERROR nova.compute.manager [req-5679ddfe-79e3-4adb-b220-915f4a38b532 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc] [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Instance failed block device setup ..... 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] ConnectionError: [Errno 101] ENETUNREACH What tries nova to reach? How could I debug that further? Full Log included. -martin Log: ceph --version ceph version 0.61 (237f3f1e8d8c3b85666529860285dcdffdeda4c5) root@compute1:~# dpkg -l|grep -e ceph-common -e cinder ii ceph-common 0.61-1precise common utilities to mount and interact with a ceph storage cluster ii python-cinderclient 1:1.0.3-0ubuntu1~cloud0 python bindings to the OpenStack Volume API nova-compute.log 2013-05-30 16:08:45.224 ERROR nova.compute.manager [req-5679ddfe-79e3-4adb-b220-915f4a38b532 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc] [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Instance failed block device setup 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Traceback (most recent call last): 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1071, in _prep_block_device 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] return self._setup_block_device_mapping(context, instance, bdms) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 721, in _setup_block_device_mapping 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] volume = self.volume_api.get(context, bdm['volume_id']) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/nova/volume/cinder.py", line 193, in get 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] self._reraise_translated_volume_exception(volume_id) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/nova/volume/cinder.py", line 190, in get 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] item = cinderclient(context).volumes.get(volume_id) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/cinderclient/v1/volumes.py", line 180, in get 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] return self._get("/volumes/%s" % volume_id, "volume") 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 141, in _get 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] resp, body = self.api.client.get(url) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 185, in get 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] return self._cs_request(url, 'GET', **kwargs) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 153, in _cs_request 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] **kwargs) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 123, in request 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] **kwargs) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] return session.request(method=method, url=url, **kwargs) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 279, in request 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 374, in send 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] r = adapter.send(request, **kwargs) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 206, in send 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] raise ConnectionError(sockerr) 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] ConnectionError: [Errno 101] ENETUNREACH 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] 2013-05-30 16:08:45.329 AUDIT nova.compute.manager [req-5679ddfe-79e3-4adb-b220-915f4a38b532 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc] [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Terminating instance _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com