Hi Daniel, Thanks for the quick reply. Things seem to be working well now, Main changes from last time: - Fixed issues with the QEMU monitor interface: - Due to the TTY layer, sending "\n" to the qemu monitor translates into "\r\n" when received. This triggers a bug in older versions of QEMU (KVM <= 33) because the same command is executed twice, and still has problems with fixed QEMU because the "(qemu)" prompt is printed twice. Switch all monitor commands to end with "\r" which avoids both issues. - The QEMU monitor sends frequent terminal escape sequences, typically \033[D and \033[K. At times, these interfere with the prompt detection when they get sent between "\n" and "(qemu) ". Fix the issue by filtering out these sequences when they are received. - Migration data is fed to QEMU via stdin rather than with a filename. This allows us to feed it just the data it needs. With this and the monitor fixes, I think save/restore should now work with KVM <= 33, as a nice side effect. - Fixed quoting issues. Since the filename is passed to the shell in single quotes, the filename just needs to have ' replaced with '\'' . Also, the QEMU monitor itself requires that we use escape sequences \r, \n, \", and \\ for the corresponding characters. - New save file format, with header, XML, migration stream. Like Xen, the saved image starts with a 16-byte magic, in this case "LibvirtQemudSave". - The paused/running state of the VM before saving is stored in the save file, and the restored VM is put into the same state. I can remove this and just unconditionally resume the machine as mentioned earlier, but being able to restore paused is very useful especially for testing without disk snapshots. resume test save test /tmp/test restore /tmp/test -> starts resumed suspend test save test /tmp/test restore /tmp/test -> starts suspended (As before, the VM is still always paused as part of the migration itself) - Refuses to restore if the name or UUID is already active. Since it's getting a bit large I've split the patches up and will send them as replies to this message. -jim -- Libvir-list mailing list Libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list