On 01/08/2017 18:33, Tamas K Lengyel wrote: >> If you actually pause the whole VM (through QEMU's monitor commands >> "stop" and "cont") everything should be safe. Of course there can be >> bugs and PCI passthrough devices should be problematic, but in general >> the device emulation is quiescent. This however is not the case when >> only the VCPUs are paused. > IMHO for some use-cases it is sufficient to have the guest itself be > limited in the modifications it makes to memory. So for example if > just a vCPU is paused there are areas of memory that you can interact > with without having to worry about it changing underneath the > introspecting application (ie. thread-specific datastructures like the > KPCR, etc..). If the introspecting application needs access to areas > that non-paused vCPUs may touch, or QEMU, or a pass-through device, > then it should be a decision for the introspecting app whether to > pause the VM completely. It may still choose to instead do some > error-detection on reads/writes to detect inconsistent accesses and > perhaps just re-try the operation till it succeeds. This may have less > of an impact on the performance of the VM as no full VM pause had to > be performed. It is all very application specific, so having options > is always a good thing. Fair enough. There is another issue however. If a guest is runnnig in the kernel, it can be easily paused while KVMI processes events and the like. While a guest is outside the kernel, it could be running or paused. If running, the value of a register might change before the VM reenters execution (due to a reset, or due to ugly features such as the VMware magic I/O port 0x5658). So the introspector would probably prefer anyway to do any changes while the guest is in the kernel: one idea I had was a KVMI_PAUSE_VCPU command that replies with a KVMI_VCPU_PAUSED event---then the introspector can send commands that do the required patching and then restart the guest by replying to the event. But if the guest is paused, KVMI_PAUSE_VCPU would never be processed. So how could the introspector distinguish the two cases and avoid the KVMI_PAUSE_VCPU if the guest is paused? (There is another complication: the guest could be running with the APIC emulated in userspace. In that case, a VCPU doing "cli;hlt" spends infinite time in userspace even though it's running, and KVM has no idea why. This is less common, but it's worth mentioning too). Paolo
