On Wed, Jul 02, 2014 at 11:56:23AM -0400, Francesco Romani wrote: > > > This is made only worse by the fact that VDSM is a python 2.7 application, > > > and notoriously > > > python 2.x behaves very badly with threads. We are already working to > > > improve our code, > > > but I'd like to bring the discussion here and see if and when the querying > > > API can be improved. > > > > > > We currently use these APIs for our sempling: > > > virDomainBlockInfo > > > virDomainGetInfo > > > virDomainGetCPUStats > > > virDomainBlockStats > > > virDomainBlockStatsFlags > > > virDomainInterfaceStats > > > virDomainGetVcpusFlags > > > virDomainGetMetadata > > > > > > What we'd like to have is > > > > > > * asynchronous APIs for querying domain stats > > > (https://bugzilla.redhat.com/show_bug.cgi?id=1113106) > > > This would be just awesome. Either a single callback or a different one > > > per call is fine > > > (let's discuss this!). > > > please note that we are much more concerned about thread reduction then > > > about performance > > > numbers. We had report of thread number becoming a real harm, while > > > performance so far > > > is not yet a concern > > > (https://bugzilla.redhat.com/show_bug.cgi?id=1102147#c54) > > > > I'm not a big fan of this approach. I mean, IIRC python has this Big > > Python Lock, which effectively prevents two threads run concurrently. > > It has the GIL, yes. Only one thread can run python code at any given time. > This however it is not true for extensions modules written in C which if carefully > designed (read: coded to properly release the GIL) can run concurrently. > This is one of the reasons while threading in python it is tolerated for I/O, > evne though never recommended. > > AFAIK/IIRC the code the libvirt module for python allows this, so we should > be good to go. For the sake of completeness I'll point out that there's another theoretical option. The libvirt-gobject binding to libvirt provides async APIs to libvirt APIs. It does this by using threads internally. Since these are C level threads though, if VDSM were to use libvirt-gobject it could get async APIs and the benefits of real threads, while remaining single threaded at the python layer. That all said, I'm not sure whether libvirt-gobject has sufficient API coverage for all the APIs VDSM needs. It primarily just has bindings for the APIs used by GNOME Boxes & libvirt-sandbox so far. Also not sure if it is a widely deployed enough dep for VDSM to mandate. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list