Re: [PATCH 4/7] libvirt: support an "embed" URI path selector for opening drivers

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 12/2/19 10:03 AM, Daniel P. Berrangé wrote:
> The driver URI scheme:
> 
>   "$drivername:///embed?root=/some/path"
> 
> enables a new way to use the drivers by embedding them directly in the
> calling process. To use this the process must have a thread running the
> libvirt event loop. This URI will then cause libvirt to dynamically load
> the driver module and call its global initialization function. This
> syntax is applicable to any driver, but only those will have been
> modified to support a custom root directory and embed URI path will
> successfully open.
> 
> The application can now make normal libvirt API calls which are all
> serviced in-process with no RPC layer involved.
> 
> It is required to specify an explicit root directory, and locks will be
> acquired on this directory to avoid conflicting with another app that
> might accidentally pick the same directory.
> 
> Use of '/' is not explicitly forbidden, but note that the file layout
> used underneath the embedded driver root does not match the file
> layout used by system/session mode drivers. So this cannot be used as
> a backdoor to interact with, or fake, the system/session mode drivers.
> 
> Libvirt will create arbitrary files underneath this root directory. The
> root directory can be kept untouched across connection open attempts if
> the application needs persistence. The application is responsible for
> purging everything underneath this root directory when finally no longer
> required.
> 
> Even when a virt driver is used in embedded mode, it is still possible
> for it to in turn use functionality that calls out to other secondary
> drivers in libvirtd. For example an embedded instance of QEMU can open
> the network, secret or storage drivers in the system libvirtd.
> 
> That said, the application would typically want to at least open an
> embedded secret driver ("secret:///embed?root=/some/path"). Note that
> multiple different embedded drivers can use the same root prefix and
> co-operate just as they would inside a normal libvirtd daemon.
> 
> A key thing to note is that for this to work, the application that links
> to libvirt *MUST* be built with -Wl,--export-dynamic to ensure that
> symbols from libvirt.so are exported & thus available to the dynamically
> loaded driver module. If libvirt.so itself was dynamically loaded then
> RTLD_GLOBAL must be passed to dlopen().
> 
> Signed-off-by: Daniel P. Berrangé <berrange@xxxxxxxxxx>
> ---
>  src/driver-state.h |  1 +
>  src/driver.h       |  2 ++
>  src/libvirt.c      | 72 ++++++++++++++++++++++++++++++++++++++++++++--
>  3 files changed, 73 insertions(+), 2 deletions(-)
> 
> diff --git a/src/driver-state.h b/src/driver-state.h
> index 1e2f6ed247..6b3f501e05 100644
> --- a/src/driver-state.h
> +++ b/src/driver-state.h
> @@ -50,6 +50,7 @@ typedef virStateDriver *virStateDriverPtr;
>  
>  struct _virStateDriver {
>      const char *name;
> +    bool initialized;
>      virDrvStateInitialize stateInitialize;
>      virDrvStateCleanup stateCleanup;
>      virDrvStateReload stateReload;
> diff --git a/src/driver.h b/src/driver.h
> index ca82ac974b..6278aa05b3 100644
> --- a/src/driver.h
> +++ b/src/driver.h
> @@ -82,6 +82,8 @@ struct _virConnectDriver {
>      bool localOnly;
>      /* Whether driver needs a server in the URI */
>      bool remoteOnly;
> +    /* Whether driver can be used in embedded mode */
> +    bool embeddable;
>      /*
>       * NULL terminated list of supported URI schemes.
>       *  - Single element { NULL } list indicates no supported schemes
> diff --git a/src/libvirt.c b/src/libvirt.c
> index bd2952d036..17b6506faa 100644
> --- a/src/libvirt.c
> +++ b/src/libvirt.c
> @@ -52,6 +52,7 @@
>  # include "rpc/virnettlscontext.h"
>  #endif
>  #include "vircommand.h"
> +#include "virevent.h"
>  #include "virfile.h"
>  #include "virrandom.h"
>  #include "viruri.h"
> @@ -84,6 +85,7 @@
>  #ifdef WITH_BHYVE
>  # include "bhyve/bhyve_driver.h"
>  #endif
> +#include "access/viraccessmanager.h"
>  
>  #define VIR_FROM_THIS VIR_FROM_NONE
>  
> @@ -676,10 +678,12 @@ virStateInitialize(bool privileged,
>          return -1;
>  
>      for (i = 0; i < virStateDriverTabCount; i++) {
> -        if (virStateDriverTab[i]->stateInitialize) {
> +        if (virStateDriverTab[i]->stateInitialize &&
> +            !virStateDriverTab[i]->initialized) {
>              virDrvStateInitResult ret;
>              VIR_DEBUG("Running global init for %s state driver",
>                        virStateDriverTab[i]->name);
> +            virStateDriverTab[i]->initialized = true;
>              ret = virStateDriverTab[i]->stateInitialize(privileged,
>                                                          root,
>                                                          callback,
> @@ -872,6 +876,7 @@ virConnectOpenInternal(const char *name,
>      virConnectPtr ret;
>      g_autoptr(virConf) conf = NULL;
>      char *uristr = NULL;
> +    bool embed = false;
>  
>      ret = virGetConnect();
>      if (ret == NULL)
> @@ -962,6 +967,52 @@ virConnectOpenInternal(const char *name,
>                                             ret->uri) < 0) {
>              goto failed;
>          }
> +
> +        if (STREQ(ret->uri->path, "/embed")) {
> +            const char *root = NULL;
> +            g_autofree char *regMethod = NULL;
> +            VIR_DEBUG("URI path requests %s driver embedded mode",
> +                      ret->uri->scheme);
> +            if (strspn(ret->uri->scheme, "abcdefghijklmnopqrstuvwxyz")  !=
> +                strlen(ret->uri->scheme)) {
> +                virReportError(VIR_ERR_NO_CONNECT,
> +                               _("URI scheme '%s' for embedded driver is not valid"),
> +                               ret->uri->scheme);
> +                goto failed;
> +            }
> +
> +            for (i = 0; i < ret->uri->paramsCount; i++) {
> +                virURIParamPtr var = &ret->uri->params[i];
> +                if (STREQ(var->name, "root"))
> +                    root = var->value;
> +            }
> +
> +            if (!root) {
> +                virReportError(VIR_ERR_INVALID_ARG, "%s",
> +                               _("root parameter required for embedded driver"));
> +                goto failed;
> +            }
> +
> +            if (virEventRequireImpl() < 0)
> +                goto failed;
> +
> +            regMethod = g_strdup_printf("%sRegister", ret->uri->scheme);
> +
> +            if (virDriverLoadModule(ret->uri->scheme, regMethod, false) < 0)
> +                goto failed;
> +
> +            if (virAccessManagerGetDefault() == NULL) {
> +                virAccessManagerPtr acl = virAccessManagerNew("none");
> +                if (!acl)
> +                    goto failed;
> +                virAccessManagerSetDefault(acl);
> +            }
> +
> +            if (virStateInitialize(geteuid() == 0, true, root, NULL, NULL) < 0)
> +                goto failed;
> +
> +            embed = true;
> +        }

It would be nice if this logic was moved to a separate function

I've hit a couple issues in testing, not sure if/where the fixes will
live, so I'll just mention them here. Also the reviewed patches are
pushable IMO

I tried this code:

from gi.repository import LibvirtGLib

import libvirt



LibvirtGLib.init(None)

LibvirtGLib.event_register()



conn1 = libvirt.open("qemu:///embed?root=/tmp/foo")

conn2 = libvirt.open("qemu:///embed?root=/tmp/bar")

print(conn1.listAllDomains())

print(conn2.listAllDomains())


With /tmp/foo populated with a VM: both connections see the same values.
So this should be rejected. Even trying to close conn1 fully and open a
new embed root will only see the old root. So maybe this needs knowledge
in the driver lookup.

The second issue: testing with virt-manager, everything locks up with
OpenGraphicsFD:

#0  0x00007ffff7a7f07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at
/lib64/libpthread.so.0
#1  0x00007fffe94be113 in virCondWaitUntil (c=c@entry=0x7fffc8071f98,
m=m@entry=0x7fffc8071ed0, whenms=whenms@entry=1576602286073) at
/home/crobinso/src/libvirt/src/util/virthread.c:159
#2  0x00007fffe44fc549 in qemuDomainObjBeginJobInternal
(driver=driver@entry=0x7fffc8004f30, obj=0x7fffc8071ec0,
job=job@entry=QEMU_JOB_MODIFY,
agentJob=agentJob@entry=QEMU_AGENT_JOB_NONE,
asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_NONE, nowait=nowait@entry=false)
at /home/crobinso/src/libvirt/src/qemu/qemu_domain.c:9357
#3  0x00007fffe4500aa1 in qemuDomainObjBeginJob
(driver=driver@entry=0x7fffc8004f30, obj=<optimized out>,
job=job@entry=QEMU_JOB_MODIFY) at
/home/crobinso/src/libvirt/src/qemu/qemu_domain.c:9521
#4  0x00007fffe4582572 in qemuDomainOpenGraphicsFD (dom=<optimized out>,
idx=<optimized out>, flags=0) at
/home/crobinso/src/libvirt/src/qemu/qemu_driver.c:18990
#5  0x00007fffe968699c in virDomainOpenGraphicsFD (dom=0x7fffd0005830,
idx=0, flags=0) at /home/crobinso/src/libvirt/src/libvirt-domain.c:10664
#6  0x00007fffe98cc6b1 in libvirt_virDomainOpenGraphicsFD () at
/usr/lib64/python3.7/site-packages/libvirtmod.cpython-37m-x86_64-linux-gnu.so


I didn't dig into it any more than that. Otherwise in some mild testing
I'm surprised how much things seemed to 'just work' :)

- Cole

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux