Thomas Gleixner <tglx@xxxxxxxxxxxxx> wrote: > Wei, > > On Tue, Dec 14 2021 at 16:11, Wei W. Wang wrote: >> On Tuesday, December 14, 2021 11:40 PM, Thomas Gleixner wrote: >>> On Tue, Dec 14 2021 at 15:09, Wei W. Wang wrote: >>> > On Tuesday, December 14, 2021 10:50 AM, Thomas Gleixner wrote: >>> >> + * Return: 0 on success, error code otherwise */ int >>> >> +__fpu_update_guest_features(struct fpu_guest *guest_fpu, u64 xcr0, >>> >> +u64 >>> >> +xfd) { >>> > >>> > I think there would be one issue for the "host write on restore" case. >>> > The current QEMU based host restore uses the following sequence: >>> > 1) restore xsave >>> > 2) restore xcr0 >>> > 3) restore XFD MSR >>> >>> This needs to be fixed. Ordering clearly needs to be: >>> >>> XFD, XCR0, XSTATE >> >> Sorry, just to clarify that the ordering in QEMU isn't made by us >> for this specific XFD enabling. >> It has been there for long time for the general restoring of all the >> XCRs and MSRs. >> (if you are interested..FYI: >> https://github.com/qemu/qemu/blob/master/target/i386/kvm/kvm.c#L4168). >> - kvm_put_xsave() >> - kvm_put_xcrs() >> - kvm_put_msrs() >> >> We need to check with the QEMU migration maintainer (Dave and Juan CC-ed) >> if changing that ordering would be OK. >> (In general, I think there are no hard rules documented for this ordering) > > There haven't been ordering requirements so far, but with dynamic > feature enablement there are. > > I really want to avoid going to the point to deduce it from the > xstate:xfeatures bitmap, which is just backwards and Qemu has all the > required information already. Hi First of all, I claim ZERO knowledge about low level x86_64. Once told that, this don't matter for qemu migration, code is at target/i386/kvm/kvm.c:kvm_arch_put_registers() ret = kvm_put_xsave(x86_cpu); if (ret < 0) { return ret; } ret = kvm_put_xcrs(x86_cpu); if (ret < 0) { return ret; } /* must be before kvm_put_msrs */ ret = kvm_inject_mce_oldstyle(x86_cpu); if (ret < 0) { return ret; } ret = kvm_put_msrs(x86_cpu, level); if (ret < 0) { return ret; } If it needs to be done in any other order, it is completely independent of whatever is inside the migration stream. I guess that Paolo will put some light here. Later, Juan.