2011/5/6 Daniel P. Berrange <berrange@xxxxxxxxxx>: > On Sun, Apr 24, 2011 at 11:13:47AM +0200, Matthias Bolte wrote: >> Richard W.M. Jones suggested [1] that the code that directly deals with the >> XDR protocol should be generated. The remote_generate_stubs.pl script >> already generates all the headers, just the bodies in the daemon and remote >> driver are manually written. But most of the functions just follow simple >> patterns. So I extended the generator to exploit this patterns and move >> 11 kLOC code from manually written to generated code. > > As expected the generator itself is pretty complex & hard to understand, > but given the complexity of our API there's nothing that can really be > done about this. It probably makes it harder that the commits just document the process of how I moved groups of functions to the generator. I meant to add patch 23/22 to the series that adds more documentation in the guts of the generator to explain its mechanics, but I didn't come around to do that yet. > I see it takes a blacklist approach, so I'm wondering what the behaviour > is if someone adds a new function that it isn't designed to cope with > yet. Is it able to raise immediate errors for stuff it can't cope with, > or does it silently generate bogus code like our python generator often > does ? ÂIdeally the former, but it isn't critical - just something we > need to be aware of for future API patch review Hm, I didn't think of this problem yet. But I would expect that in almost all cases the generator will complain about stuff it can't handle. Currently there are some general-catch-the-rest-else cases in the generator. Those can be made much stricter. I think that it'll be possible to make the generator reliably report stuff it can't handle. I'll work on that. To be really sure, we might add a whitelist aside the backlist and once the generator sees something that's not on one of the lists it complains with an error that tells the programmer to either add the function to whitelist and check that the generated code does the right thing, or add it to the blacklist and add a function body manually. On the other hand, stuff like flags parameter being unsigned in the public API but signed in the XDR protocol need manual special cases in the generator. We could add some general sanity checks to avoid something like this in the future. The generator could complain when a parameter is called flags but is signed. I'll work on that too. > Also, as per my other patch today, I think we shouldn't actually store > the generated bodies in GIT, once this is done. That's true. >> During this I came a cross many small variations and problems in the XDR >> protocol. For example, NWFilterDefineXML has a flags parameter in the public >> API, but it's not transferred in the XDR protocol. Another things is the >> variations in the usage of unsigned VS signed types. This comes in two forms. >> public API VS XDR procotol and in between different functions. For example, >> some functions use int for the flags paramater and some use unsigned int. >> >> This results in quite a lot of special case handling in the generator. >> >> Âcfg.mk               Â|  10 +- >> Âdaemon/Makefile.am         Â|  46 +- >> Âdaemon/qemu_dispatch_args.h     |  Â2 +- >> Âdaemon/qemu_dispatch_bodies.c    |  Â6 + >> Âdaemon/qemu_dispatch_prototypes.h  |  Â2 +- >> Âdaemon/qemu_dispatch_ret.h     Â|  Â2 +- >> Âdaemon/qemu_dispatch_table.h    Â|  Â2 +- >> Âdaemon/remote.c           | 5765 +---------------------------------- >> Âdaemon/remote_dispatch_args.h    |  Â2 +- >> Âdaemon/remote_dispatch_bodies.c   | 5933 +++++++++++++++++++++++++++++++++++ >> Âdaemon/remote_dispatch_prototypes.h |  80 +- >> Âdaemon/remote_dispatch_ret.h    Â|  Â2 +- >> Âdaemon/remote_dispatch_table.h   Â| Â158 +- >> Âdaemon/remote_generate_stubs.pl   | Â195 -- >> Âdaemon/remote_generator.pl     Â| 1198 +++++++ >> Âpo/POTFILES.in           Â|  Â1 + >> Âsrc/Makefile.am           |  13 +- >> Âsrc/remote/qemu_client_bodies.c   |  Â4 + >> Âsrc/remote/qemu_protocol.c     Â|  Â2 +- >> Âsrc/remote/qemu_protocol.h     Â|  Â2 +- >> Âsrc/remote/qemu_protocol.x     Â|  Â2 +- >> Âsrc/remote/remote_client_bodies.c  | 4664 +++++++++++++++++++++++++++ >> Âsrc/remote/remote_driver.c     Â| 4907 +---------------------------- >> Âsrc/remote/remote_protocol.c    Â|  26 +- >> Âsrc/remote/remote_protocol.h    Â|  26 +- >> Âsrc/remote/remote_protocol.x    Â|  34 +- >> Âsrc/remote_protocol-structs     |  26 +- >> Â27 files changed, 12093 insertions(+), 11017 deletions(-) > > It is pretty hard to review this, but I've looked at the end result > remote_client_bodies.c and remote_dispatch_bodies.c files and they > both look sane. ÂAlso, the protocol definition itself hasn't been > changed, so I'm inclined to ACK this and let us deal with any fallout > as followups. > > Daniel Thanks, for reviewing this huge patch series. I'm quite sure I didn't break anything major because the TCK still runs fine. I also checked per function that the generated code matches the original code in functionality. I hope that nothing slipped but I cannot guarantee it. So, are you just inclined to ACK or do you ACK this? :) I'd suggest that I push this series as is to get it in the wild early in the 0.9.2 cycle, so it can get enough testing. I'll make the general-catch-the-rest-else cases in the generator stricter to make the generator more robust against future additions to the XDR protocol. I might also add the additional whitelist when we decide that its necessary. Matthias -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list