Sorry. I should have mentioned, this is using the bobtail version of ceph. On Mon, May 13, 2013 at 1:13 PM, Mandell Degerness <mandell@xxxxxxxxxxxxxxx> wrote: > I know that there was another report of the bad behavior when deleting > an RBD that is currently mounted on a host. My problem is related, > but slightly different. > > We are using openstack and Grizzly Cinder to create a bootable ceph > volume. The instance was booted and all was well. The server on > which the instance had been booted was unplugged. The user deleted > the instance - which amounts to a database update on the Nova side. > They then tried to delete the volume, which failed with the following > error: > > Traceback (most recent call last): > 21262 File "/usr/lib64/python2.7/site-packages/cinder/volume/driver.py", > line 90, in _try_execute > 21263 self._execute(*command, **kwargs) > 21264 File "/usr/lib64/python2.7/site-packages/cinder/utils.py", > line 190, in execute > 21265 cmd=' '.join(cmd)) > 21266 ProcessExecutionError: Unexpected error while running command. > 21267 Command: rbd rm --pool rbd volume-66e11621-1c38-4e2d-9d90-cc511013c290 > 21268 Exit code: 16 > 21269 Stdout: '\rRemoving image: 1% complete...\rRemoving image: 2% > complete...\rRemoving image: 3% complete...\rRemoving image: 4% > complete...\rRemoving image: 5% complete ...\rRemoving image: 6% > complete...\rRemoving image: 7% complete...\rRemoving image: 8% > complete...\rRemoving image: 9% complete...\rRemoving image: 10% > complete...\r Removing image: 11% complete...\rRemoving image: > 12% complete...\rRemoving image: 13% complete...\rRemoving image: 14% > complete...\rRemoving image: 15% complete...\rR emoving image: > 16% complete...\rRemoving image: 17% complete...\rRemoving image: 18% > complete...\rRemoving image: 19% complete...\rRemoving image: 20% > complete...\rRe moving image: 21% complete...\rRemoving image: > 22% complete...\rRemoving image: 23% complete...\rRemoving image: 24% > complete...\rRemoving image: 25% complete...\rRem oving image: > 26% complete...\rRemoving image: 27% complete...\rRemoving image: 28% > complete...\rRemoving image: 29% complete...\rRemoving image: 30% > complete...\rRemo ving image: 31% complete...\rRemoving image: > 32% complete...\rRemoving image: 33% complete...\rRemoving image: 34% > complete...\rRemoving image: 35% complete...\rRemov ing image: > 36% complete...\rRemoving image: 37% complete...\rRemoving image: 38% > complete...\rRemoving image: 39% complete...\rRemoving image: 40% > complete...\rRemovi ng image: 41% complete...\rRemoving image: > 42% complete...\rRemoving image: 43% complete...\rRemoving image: 44% > complete...\rRemoving image: 45% complete...\rRemovin g image: > 46% complete...\rRemoving image: 47% complete...\rRemoving image: 48% > complete...\rRemoving image: 49% complete...\rRemoving image: 50% > complete...\rRemoving image: 51% complete...\rRemoving image: > 52% complete...\rRemoving image: 53% complete...\rRemoving image: 54% > complete...\rRemoving image: 55% complete...\rRemoving image: > 56% complete...\rRemoving image: 57% complete...\rRemoving image: 58% > complete...\rRemoving image: 59% complete...\rRemoving image: 60% > complete...\rRemoving i mage: 61% complete...\rRemoving image: > 62% complete...\rRemoving image: 63% complete...\rRemoving image: 64% > complete...\rRemoving image: 65% complete...\rRemoving im age: > 66% complete...\rRemoving image: 67% complete...\rRemoving image: 68% > complete...\rRemoving image: 69% complete...\rRemoving image: 70% > complete...\rRemoving ima ge: 71% complete...\rRemoving image: > 72% complete...\rRemoving image: 73% complete...\rRemoving image: 74% > complete...\rRemoving image: 75% complete...\rRemoving imag e: > 76% complete...\rRemoving image: 77% complete...\rRemoving image: 78% > complete...\rRemoving image: 79% complete...\rRemoving image: 80% > complete...\rRemoving image : 81% complete...\rRemoving image: > 82% complete...\rRemoving image: 83% complete...\rRemoving image: 84% > complete...\rRemoving image: 85% complete...\rRemoving image: > 86% complete...\rRemoving image: 87% complete...\rRemoving image: 88% > complete...\rRemoving image: 89% complete...\rRemoving image: 90% > complete...\rRemoving image: 91% complete...\rRemoving image: > 92% complete...\rRemoving image: 93% complete...\rRemoving image: 94% > complete...\rRemoving image: 95% complete...\rRemoving image: 9 > 6% complete...\rRemoving image: 97% complete...\rRemoving image: 98% > complete...\rRemoving image: 99% complete...\rRemoving image: 99% > complete...failed.\n' > 21270 Stderr: 'rbd: error: image still has watchers\nThis means the > image is still open or the client using it crashed. Try again after > closing/unmapping it or waiting 30s for the crashed client to > timeout.\n2013-05-09 21:51:27.522986 7f8aca884780 -1 librbd: error > removing header: (16) Device or resource busy\n' > > It appears to me that Ceph still believes the volume is mounted > somewhere. Is there a way to tell Ceph to delete the RBD, in spite of > it's belief that it is mounted? _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |