[PATCHv5 1/6] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



We still default to bhyveloader(1) if no explicit bootloader
configuration is supplied in the domain.

If the /domain/bootloader looks like grub-bhyve and the user doesn't
supply /domain/bootloader_args, we make an intelligent guess and try
chainloading the first partition on the disk (or a CD if one exists,
under the assumption that for a VM a CD is likely an install source).

Caveat: Assumes the HDD boots from the msdos1 partition. I think this is
a pretty reasonable assumption for a VM. (DrvBhyve with Bhyveload
already assumes that the first disk should be booted.)

I've tested both HDD and CD boot and they seem to work.

Sponsored by:  EMC / Isilon storage division

Signed-off-by: Conrad Meyer <conrad.meyer@xxxxxxxxxx>
---
 docs/drvbhyve.html.in     |  94 ++++++++++++++++++++-
 docs/formatdomain.html.in |   4 +-
 src/bhyve/bhyve_command.c | 202 +++++++++++++++++++++++++++++++++++++++++-----
 src/bhyve/bhyve_command.h |   5 +-
 src/bhyve/bhyve_domain.c  |   5 ++
 src/bhyve/bhyve_domain.h  |   1 +
 src/bhyve/bhyve_driver.c  |   2 +-
 src/bhyve/bhyve_process.c |  13 ++-
 8 files changed, 298 insertions(+), 28 deletions(-)

diff --git a/docs/drvbhyve.html.in b/docs/drvbhyve.html.in
index 39afdf5..ee1317e 100644
--- a/docs/drvbhyve.html.in
+++ b/docs/drvbhyve.html.in
@@ -37,8 +37,7 @@ bhyve+ssh://root@xxxxxxxxxxx/system (remote access, SSH tunnelled)
 <h3>Example config</h3>
 <p>
 The bhyve driver in libvirt is in its early stage and under active development. So it supports
-only limited number of features bhyve provides. All the supported features could be found
-in this sample domain XML.
+only limited number of features bhyve provides.
 </p>
 
 <p>
@@ -48,10 +47,21 @@ disk device were supported per-domain. However,
 up to 31 PCI devices.
 </p>
 
+<p>
+Note: the Bhyve driver in libvirt will boot whichever device is first. If you
+want to install from CD, put the CD device first. If not, put the root HDD
+first.
+</p>
+
+<p>
+Note: Only the SATA bus is supported. Only <code>cdrom</code>- and
+<code>disk</code>-type disks are supported.
+</p>
+
 <pre>
 &lt;domain type='bhyve'&gt;
-  &lt;name&gt;bhyve&lt;/name&gt;
-  &lt;uuid&gt;df3be7e7-a104-11e3-aeb0-50e5492bd3dc&lt;/uuid&gt;
+    &lt;name&gt;bhyve&lt;/name&gt;
+    &lt;uuid&gt;df3be7e7-a104-11e3-aeb0-50e5492bd3dc&lt;/uuid&gt;
     &lt;memory&gt;219136&lt;/memory&gt;
     &lt;currentMemory&gt;219136&lt;/currentMemory&gt;
     &lt;vcpu&gt;1&lt;/vcpu&gt;
@@ -76,6 +86,7 @@ up to 31 PCI devices.
         &lt;driver name='file' type='raw'/&gt;
         &lt;source file='/path/to/cdrom.iso'/&gt;
         &lt;target dev='hdc' bus='sata'/&gt;
+        &lt;readonly/&gt;
       &lt;/disk&gt;
       &lt;interface type='bridge'&gt;
         &lt;model type='virtio'/&gt;
@@ -85,6 +96,53 @@ up to 31 PCI devices.
 &lt;/domain&gt;
 </pre>
 
+<p>(The &lt;disk&gt; sections may be swapped in order to install from
+<em>cdrom.iso</em>.)</p>
+
+<h3>Example config (Linux guest)</h3>
+
+<p>
+Note the addition of &lt;bootloader&gt;.
+</p>
+
+<pre>
+&lt;domain type='bhyve'&gt;
+    &lt;name&gt;linux_guest&lt;/name&gt;
+    &lt;uuid&gt;df3be7e7-a104-11e3-aeb0-50e5492bd3dc&lt;/uuid&gt;
+    &lt;memory&gt;131072&lt;/memory&gt;
+    &lt;currentMemory&gt;131072&lt;/currentMemory&gt;
+    &lt;vcpu&gt;1&lt;/vcpu&gt;
+    &lt;bootloader&gt;/usr/local/sbin/grub-bhyve&lt;/bootloader&gt;
+    &lt;os&gt;
+       &lt;type&gt;hvm&lt;/type&gt;
+    &lt;/os&gt;
+    &lt;features&gt;
+      &lt;apic/&gt;
+      &lt;acpi/&gt;
+    &lt;/features&gt;
+    &lt;clock offset='utc'/&gt;
+    &lt;on_poweroff&gt;destroy&lt;/on_poweroff&gt;
+    &lt;on_reboot&gt;restart&lt;/on_reboot&gt;
+    &lt;on_crash&gt;destroy&lt;/on_crash&gt;
+    &lt;devices&gt;
+      &lt;disk type='file' device='disk'&gt;
+        &lt;driver name='file' type='raw'/&gt;
+        &lt;source file='/path/to/guest_hdd.img'/&gt;
+        &lt;target dev='hda' bus='sata'/&gt;
+      &lt;/disk&gt;
+      &lt;disk type='file' device='cdrom'&gt;
+        &lt;driver name='file' type='raw'/&gt;
+        &lt;source file='/path/to/cdrom.iso'/&gt;
+        &lt;target dev='hdc' bus='sata'/&gt;
+        &lt;readonly/&gt;
+      &lt;/disk&gt;
+      &lt;interface type='bridge'&gt;
+        &lt;model type='virtio'/&gt;
+        &lt;source bridge="virbr0"/&gt;
+      &lt;/interface&gt;
+    &lt;/devices&gt;
+&lt;/domain&gt;
+</pre>
 
 <h2><a name="usage">Guest usage / management</a></h2>
 
@@ -119,6 +177,14 @@ to let a guest boot or start a guest using:</p>
 
 <pre>start --console domname</pre>
 
+<p><b>NB:</b> An interactive bootloader will keep the domain from starting (and
+thus <code>virsh console</code> or <code>start --console</code>) until the
+console is opened by a client. To select a boot option and allow the domain to
+finish starting, one must use an alternative terminal client to connect to the
+null modem device. One example is:</p>
+
+<pre>cu -l /dev/nmdm0B</pre>
+
 <h3><a name="xmltonative">Converting from domain XML to Bhyve args</a></h3>
 
 <p>
@@ -157,5 +223,25 @@ An example of domain XML device entry for that will look like:</p>
 <p>Please refer to the <a href="storage.html">Storage documentation</a> for more details on storage
 management.</p>
 
+<h3><a name="grubbhyve">Using grub2-bhyve or Alternative Bootloaders</a></h3>
+
+<p>It's possible to boot non-FreeBSD guests by specifying an explicit
+bootloader, e.g. <code>grub-bhyve(1)</code>. Arguments to the bootloader may be
+specified as well. If the bootloader is <code>grub-bhyve</code> and arguments
+are omitted, libvirt will try and boot the first disk in the domain (either
+<code>cdrom</code>- or <code>disk</code>-type devices). If the disk type is
+<code>disk</code>, it will attempt to boot from the first partition in the disk
+image.</p>
+
+<pre>
+  ...
+    &lt;bootloader&gt;/usr/local/sbin/grub-bhyve&lt;/bootloader&gt;
+    &lt;bootloader_args&gt;...&lt;/bootloader_args&gt;
+  ...
+</pre>
+
+<p>Caveat: <code>bootloader_args</code> does not support any quoting.
+Filenames, etc, must not have spaces or they will be tokenized incorrectly.</p>
+
   </body>
 </html>
diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 0099ce7..b7b6c46 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -217,7 +217,9 @@
       a BIOS, and instead the host is responsible to kicking off the
       operating system boot. This may use a pseudo-bootloader in the
       host to provide an interface to choose a kernel for the guest.
-      An example is <code>pygrub</code> with Xen.
+      An example is <code>pygrub</code> with Xen. The Bhyve hypervisor
+      also uses a host bootloader, either <code>bhyveload</code> or
+      <code>grub-bhyve</code>.
     </p>
 
 <pre>
diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c
index bea4a59..01f1795 100644
--- a/src/bhyve/bhyve_command.c
+++ b/src/bhyve/bhyve_command.c
@@ -26,6 +26,8 @@
 #include <net/if_tap.h>
 
 #include "bhyve_command.h"
+#include "bhyve_domain.h"
+#include "datatypes.h"
 #include "viralloc.h"
 #include "virfile.h"
 #include "virstring.h"
@@ -294,51 +296,215 @@ virBhyveProcessBuildDestroyCmd(bhyveConnPtr driver ATTRIBUTE_UNUSED,
     return cmd;
 }
 
-virCommandPtr
-virBhyveProcessBuildLoadCmd(virConnectPtr conn,
-                            virDomainDefPtr def)
+static void
+virAppendBootloaderArgs(virCommandPtr cmd, virDomainDefPtr def)
+{
+    char **blargs;
+
+    /* XXX: Handle quoted? */
+    blargs = virStringSplit(def->os.bootloaderArgs, " ", 0);
+    virCommandAddArgSet(cmd, (const char * const *)blargs);
+    virStringFreeList(blargs);
+}
+
+static virCommandPtr
+virBhyveProcessBuildBhyveloadCmd(virDomainDefPtr def, virDomainDiskDefPtr disk)
 {
     virCommandPtr cmd;
-    virDomainDiskDefPtr disk;
 
-    if (def->ndisks < 1) {
-        virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
-                       _("domain should have at least one disk defined"));
+    cmd = virCommandNew(BHYVELOAD);
+
+    if (def->os.bootloaderArgs == NULL) {
+        VIR_DEBUG("bhyveload with default arguments");
+
+        /* Memory (MB) */
+        virCommandAddArg(cmd, "-m");
+        virCommandAddArgFormat(cmd, "%llu",
+                               VIR_DIV_UP(def->mem.max_balloon, 1024));
+
+        /* Image path */
+        virCommandAddArg(cmd, "-d");
+        virCommandAddArg(cmd, virDomainDiskGetSource(disk));
+
+        /* VM name */
+        virCommandAddArg(cmd, def->name);
+    } else {
+        VIR_DEBUG("bhyveload with arguments");
+        virAppendBootloaderArgs(cmd, def);
+    }
+
+    return cmd;
+}
+
+static virCommandPtr
+virBhyveProcessBuildCustomLoaderCmd(virDomainDefPtr def)
+{
+    virCommandPtr cmd;
+
+    if (def->os.bootloaderArgs == NULL) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Custom loader requires explicit %s configuration"),
+                       "bootloader_args");
         return NULL;
     }
 
-    disk = def->disks[0];
+    VIR_DEBUG("custom loader '%s' with arguments", def->os.bootloader);
+
+    cmd = virCommandNew(def->os.bootloader);
+    virAppendBootloaderArgs(cmd, def);
+    return cmd;
+}
+
+static bool
+virBhyveUsableDisk(virConnectPtr conn, virDomainDiskDefPtr disk)
+{
 
     if (virStorageTranslateDiskSourcePool(conn, disk) < 0)
-        return NULL;
+        return false;
 
     if ((disk->device != VIR_DOMAIN_DISK_DEVICE_DISK) &&
         (disk->device != VIR_DOMAIN_DISK_DEVICE_CDROM)) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
                        _("unsupported disk device"));
-        return NULL;
+        return false;
     }
 
     if ((virDomainDiskGetType(disk) != VIR_STORAGE_TYPE_FILE) &&
         (virDomainDiskGetType(disk) != VIR_STORAGE_TYPE_VOLUME)) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
                        _("unsupported disk type"));
-        return NULL;
+        return false;
     }
 
-    cmd = virCommandNew(BHYVELOAD);
+    return true;
+}
 
-    /* Memory */
-    virCommandAddArg(cmd, "-m");
+static virCommandPtr
+virBhyveProcessBuildGrubbhyveCmd(virDomainDefPtr def,
+                                 bhyveDomainObjPrivatePtr priv,
+                                 virConnectPtr conn)
+{
+    virDomainDiskDefPtr disk, cd;
+    virCommandPtr cmd;
+    size_t i;
+    FILE *f;
+    int rc;
+
+    if (def->os.bootloaderArgs != NULL)
+        return virBhyveProcessBuildCustomLoaderCmd(def);
+
+    /* Search disk list for CD or HDD device. */
+    cd = disk = NULL;
+    for (i = 0; i < def->ndisks; i++) {
+        if (!virBhyveUsableDisk(conn, def->disks[i]))
+            continue;
+
+        if (cd == NULL &&
+            def->disks[i]->device == VIR_DOMAIN_DISK_DEVICE_CDROM) {
+            cd = def->disks[i];
+            VIR_INFO("Picking %s as boot CD", virDomainDiskGetSource(cd));
+        }
+
+        if (disk == NULL &&
+            def->disks[i]->device == VIR_DOMAIN_DISK_DEVICE_DISK) {
+            disk = def->disks[i];
+            VIR_INFO("Picking %s as HDD", virDomainDiskGetSource(disk));
+        }
+    }
+
+    cmd = virCommandNew(def->os.bootloader);
+
+    VIR_DEBUG("grub-bhyve with default arguments");
+
+    if (priv != NULL) {
+        rc = virAsprintf(&priv->grub_devicesmap_file,
+                         "%s/grub_bhyve-%s-device.map", BHYVE_STATE_DIR,
+                         def->name);
+        if (rc < 0)
+            goto error;
+
+        f = fopen(priv->grub_devicesmap_file, "wb");
+        if (f == NULL) {
+            virReportSystemError(errno, _("Failed to open '%s'"),
+                                 priv->grub_devicesmap_file);
+            goto error;
+        }
+
+        /* Grub device.map (just for boot) */
+        if (disk != NULL)
+            fprintf(f, "(hd0) %s\n", virDomainDiskGetSource(disk));
+
+        if (cd != NULL)
+            fprintf(f, "(cd) %s\n", virDomainDiskGetSource(cd));
+
+        if (VIR_FCLOSE(f) < 0) {
+            virReportSystemError(errno, "%s", _("failed to close file"));
+            goto error;
+        }
+    }
+
+    if (cd != NULL) {
+        VIR_WARN("Trying to boot cd with grub-bhyve. If this is "
+                 "not what you wanted, specify <bootloader_args>");
+
+        virCommandAddArg(cmd, "--root");
+        virCommandAddArg(cmd, "cd");
+    } else {
+        VIR_WARN("Trying to boot hd0,msdos1 with grub-bhyve. If this is "
+                 "not what you wanted, specify <bootloader_args>");
+
+        virCommandAddArg(cmd, "--root");
+        virCommandAddArg(cmd, "hd0,msdos1");
+    }
+
+    virCommandAddArg(cmd, "--device-map");
+    if (priv != NULL)
+        virCommandAddArg(cmd, priv->grub_devicesmap_file);
+    else
+        virCommandAddArg(cmd, "<device.map>");
+
+    /* Memory in MB */
+    virCommandAddArg(cmd, "--memory");
     virCommandAddArgFormat(cmd, "%llu",
                            VIR_DIV_UP(def->mem.max_balloon, 1024));
 
-    /* Image path */
-    virCommandAddArg(cmd, "-d");
-    virCommandAddArg(cmd, virDomainDiskGetSource(disk));
-
     /* VM name */
     virCommandAddArg(cmd, def->name);
 
     return cmd;
+
+ error:
+    virCommandFree(cmd);
+    return NULL;
+}
+
+virCommandPtr
+virBhyveProcessBuildLoadCmd(virConnectPtr conn, virDomainDefPtr def,
+                            bhyveDomainObjPrivatePtr priv)
+{
+    virDomainDiskDefPtr disk;
+
+    if (def->ndisks < 1) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+                       _("domain should have at least one disk defined"));
+        return NULL;
+    }
+
+    if (def->os.bootloader == NULL) {
+        disk = def->disks[0];
+
+        if (!virBhyveUsableDisk(conn, disk))
+            return NULL;
+
+        if (def->ndisks > 1) {
+            VIR_WARN("drvbhyve attempting to boot from device %s of multiple-"
+                     "device configuration", virDomainDiskGetSource(disk));
+        }
+
+        return virBhyveProcessBuildBhyveloadCmd(def, disk);
+    } else if (strstr(def->os.bootloader, "grub-bhyve") != NULL) {
+        return virBhyveProcessBuildGrubbhyveCmd(def, priv, conn);
+    } else {
+        return virBhyveProcessBuildCustomLoaderCmd(def);
+    }
 }
diff --git a/src/bhyve/bhyve_command.h b/src/bhyve/bhyve_command.h
index 5b323bf..1da7f69 100644
--- a/src/bhyve/bhyve_command.h
+++ b/src/bhyve/bhyve_command.h
@@ -22,6 +22,7 @@
 #ifndef __BHYVE_COMMAND_H__
 # define __BHYVE_COMMAND_H__
 
+# include "bhyve_domain.h"
 # include "bhyve_utils.h"
 
 # include "domain_conf.h"
@@ -38,7 +39,7 @@ virBhyveProcessBuildDestroyCmd(bhyveConnPtr driver,
                                virDomainDefPtr def);
 
 virCommandPtr
-virBhyveProcessBuildLoadCmd(virConnectPtr conn,
-                            virDomainDefPtr def);
+virBhyveProcessBuildLoadCmd(virConnectPtr conn, virDomainDefPtr def,
+                            bhyveDomainObjPrivatePtr priv);
 
 #endif /* __BHYVE_COMMAND_H__ */
diff --git a/src/bhyve/bhyve_domain.c b/src/bhyve/bhyve_domain.c
index ecb1758..77d34a7 100644
--- a/src/bhyve/bhyve_domain.c
+++ b/src/bhyve/bhyve_domain.c
@@ -49,6 +49,11 @@ bhyveDomainObjPrivateFree(void *data)
 
     virDomainPCIAddressSetFree(priv->pciaddrs);
 
+    if (priv->grub_devicesmap_file != NULL) {
+        ignore_value(unlink(priv->grub_devicesmap_file));
+        VIR_FREE(priv->grub_devicesmap_file);
+    }
+
     VIR_FREE(priv);
 }
 
diff --git a/src/bhyve/bhyve_domain.h b/src/bhyve/bhyve_domain.h
index b8ef22a..6ecd395 100644
--- a/src/bhyve/bhyve_domain.h
+++ b/src/bhyve/bhyve_domain.h
@@ -31,6 +31,7 @@ typedef bhyveDomainObjPrivate *bhyveDomainObjPrivatePtr;
 struct _bhyveDomainObjPrivate {
     virDomainPCIAddressSetPtr pciaddrs;
     bool persistentAddrs;
+    char *grub_devicesmap_file;
 };
 
 extern virDomainXMLPrivateDataCallbacks virBhyveDriverPrivateDataCallbacks;
diff --git a/src/bhyve/bhyve_driver.c b/src/bhyve/bhyve_driver.c
index eb0d455..5f0ffc7 100644
--- a/src/bhyve/bhyve_driver.c
+++ b/src/bhyve/bhyve_driver.c
@@ -689,7 +689,7 @@ bhyveConnectDomainXMLToNative(virConnectPtr conn,
     if (bhyveDomainAssignAddresses(def, NULL) < 0)
         goto cleanup;
 
-    if (!(loadcmd = virBhyveProcessBuildLoadCmd(conn, def)))
+    if (!(loadcmd = virBhyveProcessBuildLoadCmd(conn, def, NULL)))
         goto cleanup;
 
     if (!(cmd = virBhyveProcessBuildBhyveCmd(conn, def, true)))
diff --git a/src/bhyve/bhyve_process.c b/src/bhyve/bhyve_process.c
index 0bbe388..8a59227 100644
--- a/src/bhyve/bhyve_process.c
+++ b/src/bhyve/bhyve_process.c
@@ -102,7 +102,8 @@ virBhyveProcessStart(virConnectPtr conn,
     virCommandPtr cmd = NULL;
     virCommandPtr load_cmd = NULL;
     bhyveConnPtr privconn = conn->privateData;
-    int ret = -1;
+    bhyveDomainObjPrivatePtr priv = vm->privateData;
+    int ret = -1, rc;
 
     if (virAsprintf(&logfile, "%s/%s.log",
                     BHYVE_LOG_DIR, vm->def->name) < 0)
@@ -151,7 +152,7 @@ virBhyveProcessStart(virConnectPtr conn,
     /* Now bhyve command is constructed, meaning the
      * domain is ready to be started, so we can build
      * and execute bhyveload command */
-    if (!(load_cmd = virBhyveProcessBuildLoadCmd(conn, vm->def)))
+    if (!(load_cmd = virBhyveProcessBuildLoadCmd(conn, vm->def, priv)))
         goto cleanup;
     virCommandSetOutputFD(load_cmd, &logfd);
     virCommandSetErrorFD(load_cmd, &logfd);
@@ -193,6 +194,14 @@ virBhyveProcessStart(virConnectPtr conn,
     ret = 0;
 
  cleanup:
+    if (priv->grub_devicesmap_file != NULL) {
+        rc = unlink(priv->grub_devicesmap_file);
+        if (rc < 0)
+            virReportSystemError(errno, _("cannot unlink file '%s'"),
+                                 priv->grub_devicesmap_file);
+        VIR_FREE(priv->grub_devicesmap_file);
+    }
+
     if (ret < 0) {
         int exitstatus; /* Needed to avoid logging non-zero status */
         virCommandPtr destroy_cmd;
-- 
1.9.3

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]