Add a page which documents how to configure lock managers, focusing on use of sanlock with the QEMU/KVM driver * docs/locking.html.in: Docs about lock managers * docs/sitemap.html.in: Add lock manager config to the deployment section --- docs/locking.html.in | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++ docs/sitemap.html.in | 4 + 2 files changed, 216 insertions(+), 0 deletions(-) create mode 100644 docs/locking.html.in diff --git a/docs/locking.html.in b/docs/locking.html.in new file mode 100644 index 0000000..068857d --- /dev/null +++ b/docs/locking.html.in @@ -0,0 +1,212 @@ +<?xml version="1.0"?> +<html> + <body> + <h1>Virtual machine disk locking</h1> + + <ul id="toc"></ul> + + <p> + This page describes how to ensure a single disk cannot be + used by more than one running VM at a time, across any + host in a network. This is critical to avoid data corruption + of guest files systems that are not cluster aware. + </p> + + <h2><a name="plugins">Lock manager plugins</a></h2> + + <p> + libvirt includes a pluggable framework for lock managers, + which hypervisor drivers can use to ensure safety for + guest domain disks, and potentially other resources. + At this time there are only two plugin implementations, + a "no op" implementation which does absolutely nothing, + and a <a href="https://fedorahosted.org/sanlock/";>sanlock</a> implementation which uses + the Disk Paxos algorithm to ensure safety. + </p> + + <h2><a name="sanlock">Sanlock daemon setup</a></h2> + + <p> + On many operating systems, the <strong>sanlock</strong> plugin + is distributed in a sub-package which needs to be installed + separately from the main libvirt RPM. On a Fedora/RHEL host + this can be done with the <code>yum</code> command + </p> + + <pre> + $ su - root + # yum install libvirt-lock-sanlock + </pre> + + <p> + The next step is to start the sanlock daemon. For maximum + safety sanlock prefers to have a connection to a watchdog + daemon. This will cause the entire host to be rebooted in + the event that sanlock crashes / terminates abnormally. + To start the watchdog daemon on a Fedora/RHEL host + the following commands can be run: + </p> + + <pre> + $ su - root + # chkconfig wdmd on + # service wdmd start + </pre> + + <p> + Once the watchdog is running, sanlock can be started + as follows + </p> + + <pre> + # chkconfig sanlock on + # service sanlock start + </pre> + + <p> + <em>Note:</em> if you wish to avoid the use of the + watchdog, add the following line to <code>/etc/sysconfig/sanlock</code> + before starting it + </p> + + <pre> + SANLOCKOPTS="-w 0" + </pre> + + <p> + The sanlock daemon must be started on every single host + that will be running virtual machines. So repeat these + steps as neccessary. + </p> + + <h2><a name="sanlockplugin">libvirt sanlock plugin configuration</a></h2> + + <p> + Once the sanlock daemon is running, the next step is to + configure the libvirt sanlock plugin. There is a separate + configuration file for each libvirt driver that is using + sanlock. For QEMU, we will edit <code>/etc/libvirt/qemu-sanlock.conf</code> + There is one mandatory parameter that needs to be set, + the <code>host_id</code>. This is a integer between + 1 and 2000, which must be set to a <strong>unique</strong> + value on each host running virtual machines + </p> + + <pre> + $ su - root + # augtool -s set /files/etc/libvirt/qemu-sanlock.conf/host_id 1 + </pre> + + <p> + Repeat this on every host, changing <strong>1</strong> to a + unique value for the host. + </p> + + <h2><a name="sanlockstorage">libvirt sanlock storage configuration</a></h2> + + <p> + The sanlock plugin needs to create leases in a directory + that is on a filesystem shared between all hosts running + virtual machines. Obvious choices for this include NFS, + or GFS2. The libvirt sanlock plugin expects its lease + directory be at <code>/var/lib/libvirt/sanlock</code> + so update the host's <code>/etc/fstab</code> to mount + a suitable shared/cluster filesystem at that location + </p> + + <pre> + $ su - root + # echo "some.nfs.server:/export/sanlock /var/lib/libvirt/sanlock nfs hard,nointr 0 0" >> /etc/fstab + # mount /var/lib/libvirt/sanlock + </pre> + + <p> + In terms of storage requirements, if the filesystem + uses 512 byte sectors, you need to allow for <code>1MB</code> + of storage for each guest disk. So if you have a network + with 20 virtualization hosts, each running 50 virtual + machines and an average of 2 disks per guest, you will + need <code>20*50*2 == 2000 MB</code> of storage for + sanlock. + </p> + + + <p> + On one of the hosts on the network is it wise to setup + a cron job which runs the <code>virt-sanlock-cleanup</code> + script periodically. This scripts deletes any lease + files which are not currently in use by running virtual + machines, freeing up disk space on the shared filesystem. + Unless VM disks are very frequently created + deleted + it should be sufficient to run the cleanup once a week. + </p> + + <h2><a name="qemuconfig">QEMU/KVM driver configuration</a></h2> + + <p> + The QEMU/KVM driver is fully integrated with the lock + manager framework as of release <span>0.9.3</span>. + The out of the box configuration, however, currently + uses the <strong>nop</strong> lock manager plugin. + To get protection for disks, it is thus neccessary + to reconfigure QEMU to activate the <strong>sanlock</strong> + driver. This is achieved by editing the QEMU driver + configuration file (<code>/etc/libvirt/qemu.conf</code>) + and changing the <code>lock_manager</code> configuration + tunable. + </p> + + <pre> + $ su - root + # augtool -s set /files/etc/libvirt/qemu.conf/lock_manager sanlock + # service libvirtd restart + </pre> + + <p> + If all went well, libvirtd will have talked to sanlock + and created the basic lockspace. This can be checked + by looking for existance of the following file + </p> + + <pre> + # ls /var/lib/libvirt/sanlock/ + __LIBVIRT__DISKS__ + </pre> + + <p> + Every time you start a guest, additional lease files will appear + in this directory, one for each virtual disk. The lease + files are named based on the MD5 checksum of the fully qualified + path of the virtual disk backing file. So if the guest is given + a disk backed by <code>/var/lib/libvirt/images/demo.img</code> + expect to see a lease <code>/var/lib/libvirt/sanlock/bfa0240911bc17753e0b473688822159</code> + </p> + + <p> + It should be obvious that for locking to work correctly, every + host running virtual machines should have storage configured + in the same way. The easiest way todo this is to use the libvirt + storage pool capability to configure any NFS volumes, iSCSI targets, + or SCSI HBAs used for guest storage. Simply replicate the same + storage pool XML across every host. It is important that any + storage pools exposing block devices are configured to create + volume paths under <code>/dev/disks/by-path</code> to ensure + stable paths across hosts. An example iSCSI configuration + which ensures this is: + </p> + + <pre> +<pool type='iscsi'> + <name>myiscsipool</name> + <source> + <host name='192.168.254.8'/> + <device path='your-iscsi-target-iqn'/> + </source> + <target> + <path>/dev/disk/by-path</path> + </target> +</pool> + </pre> + + </body> +</html> diff --git a/docs/sitemap.html.in b/docs/sitemap.html.in index db2963e..897ee94 100644 --- a/docs/sitemap.html.in +++ b/docs/sitemap.html.in @@ -73,6 +73,10 @@ <span>Firewall and network filter configuration</span> </li> <li> + <a href="locking.html">Disk locking</a> + <span>Ensuring exclusive guest access to disks</span> + </li> + <li> <a href="hooks.html">Hooks</a> <span>Hooks for system specific management</span> </li> -- 1.7.4.4 -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list