On Thu, Nov 22, 2012 at 2:05 AM, Josh Durgin <josh.durgin@xxxxxxxxxxx> wrote: > On 11/21/2012 04:50 AM, Andrey Korolyov wrote: >> >> Hi, >> >> Somehow I have managed to produce unkillable snapshot, which does not >> allow to remove itself or parent image: >> >> $ rbd snap purge dev-rack0/vm2 >> Removing all snapshots: 100% complete...done. > > > I see one bug with 'snap purge' ignoring the return code when removing > snaps. I just fixed this in the next branch. It's probably getting the > same error as 'rbd snap rm' below. > > Could you post the output of: > > rbd snap purge dev-rack0/vm2 --debug-ms 1 --debug-rbd 20 > > >> $ rbd rm dev-rack0/vm2 >> 2012-11-21 16:31:24.184626 7f7e0d172780 -1 librbd: image has snapshots >> - not removing >> Removing image: 0% complete...failed. >> rbd: image has snapshots - these must be deleted with 'rbd snap purge' >> before the image can be removed. >> $ rbd snap ls dev-rack0/vm2 >> SNAPID NAME SIZE >> 188 vm2.snap-yxf 16384 MB >> $ rbd info dev-rack0/vm2 >> rbd image 'vm2': >> size 16384 MB in 4096 objects >> order 22 (4096 KB objects) >> block_name_prefix: rbd_data.1fa164c960874 >> format: 2 >> features: layering >> $ rbd snap rm --snap vm2.snap-yxf dev-rack0/vm2 >> rbd: failed to remove snapshot: (2) No such file or directory >> $ rbd snap create --snap vm2.snap-yxf dev-rack0/vm2 >> rbd: failed to create snapshot: (17) File exists >> $ rbd snap rollback --snap vm2.snap-yxf dev-rack0/vm2 >> Rolling back to snapshot: 100% complete...done. >> $ rbd snap protect --snap vm2.snap-yxf dev-rack0/vm2 >> $ rbd snap unprotect --snap vm2.snap-yxf dev-rack0/vm2 >> >> >> Meanwhile, ``rbd ls -l dev-rack0'' segfaulting with an attached log. >> Is there any reliable way to kill problematic snap? > > > From this log it looks like vm2 used to be a clone, and the snapshot > vm2.snap-yxf was taken before it was flattened. Later, the parent of > vm2.snap-yxf was deleted. Is this correct? I have attached log you asked, hope it will be useful. Here is a two possible flows: snapshot created before and during flatten: Completely linear flow: $ rbd cp install/debian7 dev-rack0/testimg Image copy: 100% complete...done. $ rbd snap create --snap test1 dev-rack0/testimg $ rbd snap clone --snap test1 dev-rack0/testimg dev-rack0/testimg2 rbd: error parsing command 'clone' $ rbd snap protect --snap test1 dev-rack0/testimg $ rbd clone --snap test1 dev-rack0/testimg dev-rack0/testimg2 $ rbd snap create --snap test2 dev-rack0/testimg2 $ rbd flatten dev-rack0/testimg2 Image flatten: 100% complete...done. $ rbd snap unprotect --snap test1 dev-rack0/testimg 2012-11-22 15:11:03.446892 7ff9fb7c1780 -1 librbd: snap_unprotect: can't unprotect; at least 1 child(ren) in pool dev-rack0 rbd: unprotecting snap failed: (16) Device or resource busy $ rbd snap purge dev-rack0/testimg2 Removing all snapshots: 100% complete...done. $ rbd snap ls dev-rack0/testimg2 $ rbd snap unprotect --snap test1 dev-rack0/testimg snapshot created over image with ``flatten'' in progress: $ rbd snap create --snap test3 dev-rack0/testimg $ rbd snap protect --snap test3 dev-rack0/testimg $ rbd clone --snap test3 dev-rack0/testimg dev-rack0/testimg3 rbd $ rbd flatten dev-rack0/testimg3 [here was executed rbd snap create --snap test43 dev-rack0/testimg3] Image flatten: 100% complete...done. $ rbd snap unprotect --snap test3 dev-rack0/testimg $ rbd snap ls dev-rack0/testimg3 SNAPID NAME SIZE 323 test43 640 MB $ rbd snap purge dev-rack0/testimg3 Removing all snapshots: 100% complete...done. $ rbd snap ls dev-rack0/testimg3 SNAPID NAME SIZE 323 test43 640 MB $ rbd snap rm --snap test43 dev-rack0/testimg3 rbd: failed to remove snapshot: (2) No such file or directory Hooray, problem found! Now I`ll avoid this by putting flatten state as exclusive one over the image. ceph version 0.54 (commit:60b84b095b1009a305d4d6a5b16f88571cbd3150) > > It was a bug in 0.53 that protected snapshots could be deleted. > > Josh
Attachment:
snap.txt.gz
Description: GNU Zip compressed data
