Alexander Graf wrote: > I figured I go and try to find out what the emulation distribution is in > random use cases. The one I measured here was a: > > $ for i in `seq 1000`; do ls -la > /dev/null; done > > inside the guest. This should give pretty good hints on process spawning > overhead. Below are the results on what is issued most often. > > Number of invocations | Opcode in hex | OP | XOP | asm name | sprn > parameter | sprn name > > 00488520 2101346470 OP: 31 XOP: 83 mfmsr > 00487702 1275068452 OP: 19 XOP: 18 rfid > 00244822 2108900006 OP: 31 XOP: 339 mfspr 275 > SPRN_SPRG3 > 00244799 2107310758 OP: 31 XOP: 339 mfspr 27 > SPR_SRR1 > 00243110 2103116710 OP: 31 XOP: 467 mtspr 27 > SPR_SRR1 > 00242910 2107245478 OP: 31 XOP: 467 mtspr 26 > SPR_SRR0 > 00242854 2105148070 OP: 31 XOP: 339 mfspr 26 > SPR_SRR0 > 00206254 2101412196 OP: 31 XOP: 178 mtmsrd > 00163540 2103509348 OP: 31 XOP: 178 mtmsrd > 00162348 2108769190 OP: 31 XOP: 467 mtspr 273 > SPRN_SPRG1 > 00158986 2100380326 OP: 31 XOP: 339 mfspr 273 > SPRN_SPRG1 > 00142246 2080375332 OP: 31 XOP: 274 tlbiel > 00122541 2107311014 OP: 31 XOP: 467 mtspr 27 > SPR_SRR1 > 00122527 2105148326 OP: 31 XOP: 467 mtspr 26 > SPR_SRR0 > 00089577 2102592166 OP: 31 XOP: 339 mfspr 19 > SPR_DAR > 00089562 2102526630 OP: 31 XOP: 339 mfspr 18 > SPR_DSISR > 00082629 2103443622 OP: 31 XOP: 83 mfmsr > 00080937 2098922406 OP: 31 XOP: 467 mtspr 27 > SPR_SRR1 > 00080937 2096759718 OP: 31 XOP: 467 mtspr 26 > SPR_SRR0 > 00054759 2080393764 OP: 31 XOP: 274 tlbiel > 00042033 2080440676 OP: 31 XOP: 178 mtmsrd > 00042013 2080374950 OP: 31 XOP: 83 mfmsr > 00040733 2099315044 OP: 31 XOP: 178 mtmsrd > 00039939 2081817254 OP: 31 XOP: 339 mfspr 22 > SPR_DECR > 00039401 2088829284 OP: 31 XOP: 178 mtmsrd > 00039386 2088436646 OP: 31 XOP: 467 mtspr 27 > SPR_SRR1 > 00039377 2088763558 OP: 31 XOP: 83 mfmsr > 00039343 2086273958 OP: 31 XOP: 467 mtspr 26 > SPR_SRR0 > > > Obviously we could PV mfmsr. Most of the mfmsr and mtmsrs can also be > easily replaced by stda/lda to a negative address with a magic page. > Rfid is pretty much impossible, mtmsrd is _very_ difficult without more > logic inside the guest. The only way around tlbiel would be a queuing > invalidation mechanism - and I doubt that's possible as the kernel > expects the page to be gone instantly. > > Overall, this looks pretty promising though. Apparently > 60% of the > emulated instructions can be pretty easily patched to non-emulated ones. > So this is definitely the next low hanging performance fruit to get! > After optimizing the above instructions away, I end up with the following results: 00682354 OP: 19 XOP: 18 rfid 00237114 OP: 31 XOP: 178 mtmsrd 00232302 OP: 31 XOP: 339 mfspr SPR_DECR 00231933 OP: 31 XOP: 178 mtmsrd 00231036 OP: 31 XOP: 178 mtmsrd 00217262 OP: 31 XOP: 274 tlbiel 00132854 OP: 31 XOP: 178 mtmsrd 00112533 OP: 31 XOP: 178 mtmsrd 00037159 OP: 31 XOP: 274 tlbiel 00036223 OP: 31 XOP: 178 mtmsrd 00019193 OP: 31 XOP: 566 tlbsync 00019171 OP: 31 XOP: 306 tlbie 00007915 OP: 31 XOP: 402 slbmte 00007738 OP: 31 XOP: 434 slbie 00004772 OP: 31 XOP: 467 mtspr SPR_DECR I'm not sure how to properly deal with that. Rfid is usually called for a good reason, we need to exit the guest for that. All decrementor magic needs to be handled by KVM too. I can certainly nop out tlbsync, I guess I'll just do that. Mtmsrd should be possible to squeeze into the VM, but it's definitely not as easy as patching a single instruction somewhere. This needs serious logic. So for now the only low hanging fruit I can see is tlbsync. Expect a nice patchset soon! Alex -- To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
