On 8/10/21 12:23 PM, Andi Kleen wrote: >> But, not everyone is going to agree with me. > > Both the Intel TDX and the AMD SEV side independently came to opposite > conclusions. In general people care a lot about boot time of VM guests. I was also saying that getting to userspace fast is important to me. Almost everyone agrees there. >> This also begs the question of how folks know when this "blip" is over. >> Do we have a counter for offline pages? Is there any way to force page >> acceptance? Or, are we just stuck allocating a bunch of memory to warm >> up the system? >> >> How do folks who care about these new blips avoid them? > > It's not different than any other warmup. At warmup time you always have > lots of blips until the working set stabilizes. For example in > virtualization first touch of a new page is usually an EPT violation > handled to the host. Or in the native case you may need to do IO or free > memory. Everybody who based their critical latency percentiles around a > warming up process would be foolish, the picture would be completely > distorted. > > So the basic operation is adding some overhead, but I don't think > anything is that unusual compared to the state of the art. Except that today, you can totally avoid the allocation latency (not sure about the EPT violation/fill latency) from things like QEMU's -mem-prealloc. > Now perhaps the locking might be a problem if the other operations all > run in parallel, causing unnecessary serialization If that's really a > problem I guess we can optimize later. I don't think there's anything > fundamental about the current locking. These boot blips are not the biggest issue in the world. But, it is fully under the guest's control and I think the guest has some responsibility to provide *some* mitigation for it. 1. Do background acceptance, as opposed to relying 100% on demand-driven acceptance. Guarantees a limited window in which blips can occur. 2. Do acceptance based on user input, like from sysfs. 3. Add a command-line argument to accept everything up front, or at least before userspace runs. 4. Add some statistic for how much unaccepted memory remains. I can think of at least four ways we could mitigate it. A sysfs statistic file would probably take ~30 lines of code to loop over the bitmap. A command-line option would probably be <10 lines of code to just short-circuit the bitmap and accept everything up front. A file to force acceptance would probably be pretty quick too. Nothing there seem too onerous.
