I have put my current series here https://github.com/npiggin/linux/tree/kvm-in-c-new It contains existing Cify series plus about 50 patches, it's getting fairly stable with both L0 and L1 hypervisors. The aim of the series is to speed up the P9 entry/exit code and also simplify things where possible. It does this in several main ways: - Rearrange code to optimise SPR accesses. Mainly, avoid scoreboard stalls. - Test SPR values to avoid mtSPRs where possible. mtSPRs are expensive. - Reduce mftb. mftb is expensive. - Demand fault certain facilities to avoid saving and/or restoring them (at the cost of fault when they are used, but this is mitigated over a number of entries, like the facilities when context switching processes). PM, TM, and EBB so far. - Defer some sequences that are made just in case a guest is interrupted in the middle of a critical section to the case where the guest is scheduled on a different CPU, rather than every time (at the cost of an extra IPI in this case). Namely the tlbsync sequence for radix with GTSE, which is very expensive. - Reduce barriers, atomics, start shedding some of vcore complexity to reduce path length, locking, etc. So far this speeds up the full entry/exit cycle (measured by guest spinning in 'sc 1' to cause exits, with a host hack make it exit rather than SIGILL), by about 2x on P9 and more on a P10. There is some more that can be done (xive optimisation, more complexity reduction, removing another mftb) but there are not many easy gains left here. The big thing which is not yet addressed is a light weight exit that does not switch all context each time. That will take a bit more design to get working really well, so I prefer to do that over a longer period perhaps with the help of some realistic workloads. It's very simple to hack something up to work fast with a few TCE or HPT hcalls for example, but really we may prefer on balance to do something which is slightly slower for those but works for other host interrupts like timers, device irqs, IPIs, partition scope page faults, etc. I will submit this after the first Cify series is accepted into the powerpc/kvm tree. Thanks, Nick
