2010/11/27 Blue Swirl <blauwirbel@xxxxxxxxx>: > On Sat, Nov 27, 2010 at 8:53 AM, Yoshiaki Tamura > <tamura.yoshiaki@xxxxxxxxxxxxx> wrote: >> 2010/11/27 Stefan Hajnoczi <stefanha@xxxxxxxxx>: >>> On Sat, Nov 27, 2010 at 4:29 AM, Yoshiaki Tamura >>> <tamura.yoshiaki@xxxxxxxxxxxxx> wrote: >>>> 2010/11/27 Blue Swirl <blauwirbel@xxxxxxxxx>: >>>>> On Thu, Nov 25, 2010 at 6:06 AM, Yoshiaki Tamura >>>>> <tamura.yoshiaki@xxxxxxxxxxxxx> wrote: >>>>>> Hi, >>>>>> >>>>>> This patch series is a revised version of Kemari for KVM, which >>>>>> applied comments for the previous post and KVM Forum 2010. The >>>>>> current code is based on qemu.git >>>>>> f711df67d611e4762966a249742a5f7499e19f99. >>>>>> >>>>>> For general information about Kemari, I've made a wiki page at >>>>>> qemu.org. >>>>>> >>>>>> http://wiki.qemu.org/Features/FaultTolerance >>>>>> >>>>>> The changes from v0.1.1 -> v0.2 are: >>>>>> >>>>>> - Introduce a queue in event-tap to make VM sync live. >>>>>> - Change transaction receiver to a state machine for async receiving. >>>>>> - Replace net/block layer functions with event-tap proxy functions. >>>>>> - Remove dirty bitmap optimization for now. >>>>>> - convert DPRINTF() in ft_trans_file to trace functions. >>>>>> - convert fprintf() in ft_trans_file to error_report(). >>>>>> - improved error handling in ft_trans_file. >>>>>> - add a tmp pointer to qemu_del_vm_change_state_handler. >>>>>> >>>>>> The changes from v0.1 -> v0.1.1 are: >>>>>> >>>>>> - events are tapped in net/block layer instead of device emulation layer. >>>>>> - Introduce a new option for -incoming to accept FT transaction. >>>>>> - Removed writev() support to QEMUFile and FdMigrationState for now. I would >>>>>> post this work in a different series. >>>>>> - Modified virtio-blk save/load handler to send inuse variable to >>>>>> correctly replay. >>>>>> - Removed configure --enable-ft-mode. >>>>>> - Removed unnecessary check for qemu_realloc(). >>>>>> >>>>>> The first 6 patches modify several functions of qemu to prepare >>>>>> introducing Kemari specific components. >>>>>> >>>>>> The next 6 patches are the components of Kemari. They introduce >>>>>> event-tap and the FT transaction protocol file based on buffered file. >>>>>> The design document of FT transaction protocol can be found at, >>>>>> http://wiki.qemu.org/images/b/b1/Kemari_sender_receiver_0.5a.pdf >>>>>> >>>>>> Then the following 4 patches modifies dma-helpers, virtio-blk >>>>>> virtio-net and e1000 to replace net/block layer functions with >>>>>> event-tap proxy functions. Please note that if Kemari is off, >>>>>> event-tap will just passthrough, and there is most no intrusion to >>>>>> exisiting functions including normal live migration. >>>>> >>>>> Would it be possible to make the changes only in the block/net layer, >>>>> so that the devices are not modified at all? That is, the proxy >>>>> function would always replaces the unproxied version. >>>> >>>> I understand the benefit of your suggestion. However it seems a bit >>>> tricky. It's because event-tap uses functions of emulators and net, >>>> but block.c is also linked for utilities like qemu-img that doesn't >>>> need emulators or net. In the previous version, I added function >>>> pointers to get around. >>>> >>>> http://lists.nongnu.org/archive/html/qemu-devel/2010-05/msg02378.html >>>> >>>> I wasn't confident of this approach and discussed it at KVM Forum, and >>>> decided to give a try to replace emulator functions with proxies. >>>> Suggestions are welcomed of course. >>>> >>>>> Somehow I find some similarities to instrumentation patches. Perhaps >>>>> the instrumentation framework could be used (maybe with some changes) >>>>> for Kemari as well? That could be beneficial to both. >>>> >>>> Yes. I had the same idea but I'm not sure how tracing works. I think >>>> Stefan Hajnoczi knows it better. >>>> >>>> Stefan, is it possible to call arbitrary functions from the trace >>>> points? >>> >>> Yes, if you add code to ./tracetool. I'm not sure I see the >>> connection between Kemari and tracing though. >> >> The connection is that it may be possible to remove Kemari >> specific hook point like in ioport.c and exec.c, and let tracing >> notify Kemari instead. > > This all depends on how generic we want the trace points become. > > One possible extension to the event injection or instrumentation could > be fault injection: based on some rule, make the instrumented function > return error. That would be interesting for testing how guest handles > failure cases. > > Maybe it should be also possible to handle event injection in a > generic way. Split the instrumented function to two, before and after > the tracepoint. The tracepoint registers the tail function in addition > to the parameters. This may require a lot of refactoring though. The idea looks cool but it's a bit out of the range I can handle now:-) Let's keep the idea of binding with trace points for now, and focus on how to insert net/block tap points. >>> One question I have about Kemari is whether it adds new constraints to >>> the QEMU codebase? Fault tolerance seems like a cross-cutting concern >>> - everyone writing device emulation or core QEMU code may need to be >>> aware of new constraints. For example, "you are not allowed to >>> release I/O operations to the outside world directly, instead you need >>> to go through Kemari code which makes I/O transactional and >>> communicates with the passive host". You have converted e1000, >>> virtio-net, and virtio-blk. How do we make sure new devices that are >>> merged into qemu.git don't break Kemari? How do we go about >>> supporting the existing hw/* devices? >> >> Whether Kemari adds constraints such as you mentioned, yes. If >> the devices (including existing ones) don't call Kemari code, >> they would certainly break Kemari. Altough using proxies looks >> explicit, to make it unaware from people writing device >> emulation, it's possible to remove proxies and put changes only >> into the block/net layer as Blue suggested. > > I'd prefer that approach if possible. Thanks. Let me see how others think too. Yoshi > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
