[moderator note: .pngs were stripped to avoid overwhelming the mail server and list recipients with 1M of data] -------- Forwarded Message -------- Date: Wed, 30 Mar 2016 11:25:09 +0200 Message-ID: <20160330112509.Horde.0Oad-ZfkzdZnouXf-VkEmw1@xxxxxxxxxxxxxxxxxxxxxxxxxx> From: villeneu@xxxxxxxxxxxxxxxxxxxx To: Franky Van Liedekerke <liedekef@xxxxxxxxxx> Cc: libvirt-users@xxxxxxxxxx Subject: Re: VM crash and lock manager References: <20160329192740.Horde.BZXrmDPzMBshASPvOs_x4Q6@xxxxxxxxxxxxxxxxxxxxxxxxxx> <20160329231311.1ae867a1@xxxxxxxxxx> In-Reply-To: <20160329231311.1ae867a1@xxxxxxxxxx> --- Franky Van Liedekerke <liedekef@xxxxxxxxxx> a écrit : > On Tue, 29 Mar 2016 19:27:40 +0200 > villeneu@xxxxxxxxxxxxxxxxxxxx wrote: > >> Hello >> >> I changed my hypervisors ( 8) from centos 6.0 to fedora 23 and I >> added locks to prevent starting VM on multiple nodes, and it's work >> well. >> >> Since I changed the OS some old VMs ( centos 5.x and 6.x ) with very >> old kernel seem very instable and crash very often ( after one or two >> days ). Before they never have these problems. >> >> I saw, perhaps the reason is the dev is too slow ??? see screenshot >> joined >> >> A major problem with these crashs is that the VMs couldn't be >> destroyed by the virsh command, the qemu process is notified as >> defunct by the ps command. >> >> with virsh destroy VM >> I often get >> >> Failed to terminate process xxx with SIGTERM: Device or resource busy >> and the VM is still in the list >> >> If I try to remove all the file associated with the VM in /var/run or >> /var/lib....channel .. It doesn't give results The >> VM is still running as defunct and after a libvirt restart , libvirt >> and virsh command are pending. >> >> With sanlock it's a problem because the is never release and it's >> impossible to restart the VM. I can't remove the VM in the sanlock >> list. I tried kill commands, rm ... but without a real success. >> >> I tried many command with sanlock perhaps I misunderstood some >> commands but I never succeed in removing lock on the VMs. >> >> sanlock client rem_lockspace -r >> __LIBVIRT__DISKS__:b90b9c61e2d6413077205907ffb3281a:/var/lib/libvirt/images/POOL_ADMIN/sanlock/b90b9c61e2d6413077205907ffb3281a:0:4 >> -p 9383 >> >> So I'm trying to use lock_manager to replace the sanlock but >> lock_manager doesn't have tools to retrieve the lock / VM name in >> case of failure or crash. >> >> My questions are, is there a way to release or to force to release a >> lock with sanlock even if the qemu process is defunc and a way to >> restart libvirt in normal usage ? >> >> Perhaps by the API it`s possible to forece the release the lock ? >> >> Is there tools to retreive or calculate SHA256 lock for a VM with the >> lock_manager lock. I saw it's the lock is a SHA256 hash of the path >> and the VM name but I nerver get the good hash when I try by the hand. >> >> Is there tools to retreive or calculate SHA256 the lock for a VM with >> the lock_manager lock. I saw that the lock is a SHA256 hash of the >> path and the VM name but I never get the good hash when I try by the >> hand. >> >> Thanks. >> Sorry for my poor english ... I 'am french >> Michel > > It seems the lock held by sanlock isn't being released, so the vm can't > be shut down properly. Maybe some selinux issue? > I've been using virtlockd without troubles for more than a year already > (without selinux). For the locks, see if the lslocks command helps you. > > Franky > > _______________________________________________ > libvirt-users mailing list > libvirt-users@redhat.comhttps://www.redhat.com/mailman/listinfo/libvirt-users I do not use selinux too, I preffered sanlock at the beginning because sanlock comes with tools to remove locks. But when VMs crash abnormaly or freeze ( see the sreenshot I missed to attach it yesterday ) there is no way to unlease or remove the locks, they are presents even if I kill the process qemu and I can't restart the VM because the lock is still here. So I try now lock-manager, but do you know exactly how the lock is calculate . In case of problem I want to be able to retrieve the lock coupled with the VM. I have seen that it is a hash SHA256 of the path and the name, but if a try to do by the hand a sha256 of the complete path of the VM I don't have the correct value ( the same value as lock_manager ). Any Ideas ? Thanks crash1.png crash2.png
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ libvirt-users mailing list libvirt-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvirt-users
