On Mon, Dec 18, 2017 at 1:14 PM, Andrew Jones <drjones@xxxxxxxxxx> wrote: > Hi Shih-Wei, > > Thanks for doing this! Porting Christoffer's selftests to kvm-unit-tests > has been on the kvm-unit-tests' TODO list since it was first introduced. > > On Fri, Dec 15, 2017 at 04:15:38PM -0500, Shih-Wei Li wrote: >> The patch provides support for quantifying the cost of micro level >> operations on arm64 hardware. The supported operations include hypercall, >> mmio accesses, EOI virtual interrupt, and IPI send. Measurements are >> currently obtained using timer counters. Further modifications in KVM >> will be required to support timestamping using cycle counters, as KVM >> now disables accesses to the PMU counters from the VM. > > KVM only disables access when userspace tells it to, which it doesn't > do by default. Is there something else missing keeping the PMU counters > from being used? Thanks for the feedback! What I meant by PMU counters here was for "CPU cycle counter" specifically. I'm not aware of a way to enable the PMU cycle counter from QEMU, did I miss something here? > >> >> We iterate each of the tests for millions of times and output their >> average, minimum and maximum cost in timer counts. Instruction barriers > > Can we reduce the number of iterations and still get valid results? The > test takes so long that of all the platforms I tested it on timed out > before it completed, except seattle. The default timeout for kvm-unit- > tests is 90 seconds. I'd rather a unit test execute in much shorter time > than that too, in order to keep people encouraged to run them frequently. > If these tests must run a long time, then I think we should add them to > the nodefault group. I think it's possible to reduce the timeout without losing accuracy. I can look into this further. > >> were used before and after taking timestamps to avoid out-of-order >> execution or pipelining from skewing our measurements. >> >> To improve precision in the measurements, one should consider pinning >> each VCPU to a specific physical CPU (PCPU) and ensure no other task >> could run on that PCPU to skew the results. This can be achieved by >> enabling QMP server in the QEMU command in unittest.cfg for micro test, >> allowing a client program to get the thread_id for each VCPU thread >> from the QMP server. Based on the information, the client program can >> then pin the corresponding VCPUs to dedicated PCPUs and isolate >> interrupts and tasks from those PCPUs. > > To isolate the CPUs one would need to boot the host with the isolcpus > kernel command line option. Pinning the VCPUs is pretty easy though, > so we could provide a script that does that in kvm-unit-tests and then > always use it for this test. The script could also warn if we're > pinning to CPUs that haven't been isolated. > My intention was to support VCPU pinning as an optional feature, so the users that care about extra precision can add qmp option in QEMU config and run the script to pin VCPUs. Otherwise, the test can be conducted in a fashion similar to what's done in vmexit on x86. >> >> The patch has been tested on arm64 hardware including AMD Seattle and >> ThunderX2, which has GICv2 and GICv3 respectively. > > I tried thunderx2, amberwing, mustang, and seattle. Only seattle > completed, the rest timed out. I have only tested the code by invoking test directly using make standalone like the following. I did notice that it took ~90 seconds to finish the test itself. ./"tests/micro-cost" Thanks, Shih-Wei > > Thanks, > drew > >> >> Shih-Wei Li (1): >> arm64: add micro test >> >> arm/Makefile.common | 1 + >> arm/micro-test.c | 289 ++++++++++++++++++++++++++++++++++++++++++++++++++++ >> arm/unittests.cfg | 6 ++ >> 3 files changed, 296 insertions(+) >> create mode 100644 arm/micro-test.c >> >> -- >> 1.9.1 >> >>
