On 12/29/2011 04:03 AM, D. Herrendoerfer wrote: > Sorry to come back on this. [Caveat - I didn't write the threading model used in libvirt, so my answers are my interpretation of the code as I have come to understand it, but might not be 100% accurate] > > How are the different threads of the libvirt daemon separated ? > I understand that there are 2 separate threads running that are > forked. There are also 4 more that appear to be threads. > Is that correct ? I'm not quite sure what you were asking, or which processes vs. threads you are viewing. Remember, in Linux, threads and processes share the same namespace of thread ids (that is, a process id in Linux is the thread-id of its primary thread), and the difference between a single-threaded app and a multi-threaded app is whether the process owns more than one thread id. Calling fork() creates a new process (and thus a new thread), and the process of starting libvirtd ends up calling fork() twice to get the daemonization right, so that may explain the "2 separate threads running that are forked". Once the daemon is up and running, it becomes multi-threaded, by spawning helper threads all within the same process, which would explain the "4 more that appear to be threads". Basically, libvirtd is set up in an event loop, where the main thread exists solely to monitor shutdown requests, and all other threads do round-robin servicing of RPC events that come in over the sockets. > > Now, the virEvents main polling loop is running in a forked thread, > it has no access to worker threads that where created through > virsh for example > Is that correct ? Threads aren't forked, only processes are (but forking a new process creates a new thread, mainly the primary thread of the new process). Also, processes do not share memory, but threads within a process do, so a polling loop in a main thread _does_ have access to worker threads in the same process. It appears that you have some confusion in the terminology, which is in turn making it hard for me to know if I'm answering your questions. This question added in virsh, which adds another wrinkle into the picture - virsh is often capable of doing its job as a single-threaded application, which connects to the libvirtd socket to request libvirtd to do something with the qemu hypervisor. However, virsh is also designed for multi-threaded tasks - in particular, the ability to start a long-running task such as a migration, and cancel it midway, requires that virsh have a couple of threads, the primary thread that processes user commands delegates to a helper thread that interacts with libvirtd as well as listening for Ctrl-C events from the user or for migration completion events from libvirtd. This means that both libvirtd _and_ virsh are using an event loop - libvirtd to listen for incoming activity on the sockets as well as completion of those activities by the hypervisor, and virsh to listen for incoming activity from libvirtd completion. So in true code reuse fashion, we reuse the event loop in both processes, and you thus have to be careful which process you are asking about when discussing threads that are handling various events. > > Last, if a VM needs something to watch over the state of its > network devices, a separate thread needs to be created to do > that because the VMs libvirt daemon thread can't make use > of the virEvent functions without creating a new poll-loop thread ? The event loop driver supports both file-based events, such as recognizing when traffic arrives on the public libvirtd socket which needs handling by one of the RPC worker threads, as well as time-based events. But it is not limited to the public libvirtd socket; the qemu hypervisor driver can register additional fd's to be watched, such as the monitor socket tied to the qemu running each domain. As long as you can come up with a scenario where an fd can track a change, then you can register that fd with the existing event loop. And don't forget that you can use pipes as a way to set up fds (after all, our SIGINT handler for Ctrl-C in virsh is nothing more than a pipe-to-self so that we have an fd to wake up the event loop). > > Best regards, > > D.Herrendoerfer <herrend at de dot ibm dot com > <d.herrendoerfer at > herrendoerfer dot name> > > On Dec 22, 2011, at 6:04 PM, Dirk Herrendoerfer wrote: > >> Hi all, >> >> I'm trying to get libvirt to re-associate lost connections when a vepa >> connection >> is lost due to a switch error, or lldpad restart. >> >> My take was to use the virtEvent infrastructure to poll for messages >> on a netlink >> socket and then restart the association if the message indicates that >> a link came >> back up. >> >> I ran into a problem, that if I start the polling netlink event from >> the daemon thread >> I would get the file events an the netlink messages, but I cannot >> configure the >> event handler because the rpc client threads and the daemon do not >> share the >> same address space. So it sounds like you are asking how to make a client, such as virsh, aware that an event was recognized in libvirtd. At this point, it boils down to an RPC event - whenever something in libvirtd fires that a client might usefully want to learn about, we can then map that to the RPC event mechanisms. Look at the virConnectDomainEventRegisterAny API, which is a means for setting up a callback function in the client that will be called when the event happens in libvirtd, by means of an RPC message from libvirtd to the client containing the details appropriate for the callback. Also, docs/internals/rpc.html describes the details behind RPC event calls; normally, things are driven by the client (client issues a call, server issues a response), but in the case of events, it is an asynchronous call where the server issues the event, and the client reacts to the event by calling any registered callback functions. I'm not sure if I'm on the right track for answering what you are really trying to do, but hopefully it helped, or someone else can chime in to correct anything I got wrong. >> >> Is there a way to get file events in the VMs setup code so I can >> register a callback >> at VM initialization time to receive netlink messages and restart >> association if needed ? >> >> Best regards, >> >> D.Herrendoerfer <herrend at de dot ibm dot com > <d.herrendoerfer at >> herrendoerfer dot name> >> >> -- >> libvir-list mailing list >> libvir-list@xxxxxxxxxx >> https://www.redhat.com/mailman/listinfo/libvir-list > > -- > libvir-list mailing list > libvir-list@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/libvir-list > -- Eric Blake eblake@xxxxxxxxxx +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Attachment:
signature.asc
Description: OpenPGP digital signature
-- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list