TLDR
====
Multichase in 64 microVMs achieved 6% more total samples (in ~4 hours) after this patchset [1].
Hardware
========
HOST $ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 43 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Vendor ID: AuthenticAMD
Model name: AMD Ryzen Threadripper PRO 3995WX 64-Cores
CPU family: 23
Model: 49
Thread(s) per core: 2
Core(s) per socket: 64
Socket(s): 1
Stepping: 0
Frequency boost: disabled
CPU max MHz: 4308.3979
CPU min MHz: 2200.0000
BogoMIPS: 5390.20
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2
...
Virtualization features:
Virtualization: AMD-V
Caches (sum of all):
L1d: 2 MiB (64 instances)
L1i: 2 MiB (64 instances)
L2: 32 MiB (64 instances)
L3: 256 MiB (16 instances)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-127
Vulnerabilities:
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
Srbds: Not affected
Tsx async abort: Not affected
HOST $ numactl -H
available: 1 nodes (0)
node 0 cpus: 0-127
node 0 size: 257542 MB
node 0 free: 224855 MB
node distances:
node 0
0: 10
HOST $ cat /sys/class/nvme/nvme0/model
INTEL SSDPF21Q800GB
HOST $ cat /sys/class/nvme/nvme0/numa_node
0
Software
========
HOST $ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"
HOST $ uname -a
Linux x86 6.4.0-rc5+ #1 SMP PREEMPT_DYNAMIC Wed Jun 7 22:17:47 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
HOST $ cat /proc/swaps
Filename Type Size Used Priority
/dev/nvme0n1p2 partition 466838356 0 -2
HOST $ cat /sys/kernel/mm/lru_gen/enabled
0x000f
HOST $ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
HOST $ cat /sys/kernel/mm/transparent_hugepage/defrag
always defer defer+madvise madvise [never]
Procedure
=========
HOST $ git clone https://github.com/google/multichase
HOST $ <Build multichase>
HOST $ <Unpack /boot/initrd.img into ./initrd/>
HOST $ cp multichase/multichase ./initrd/bin/
HOST $ sed -i \
"/^maybe_break top$/i multichase -t 2 -m 4g -n 28800; poweroff" \
./initrd/init
HOST $ <Pack ./initrd/ into ./initrd.img>
HOST $ cat run_microvms.sh
memcgs=64
run() {
path=/sys/fs/cgroup/memcg$1
mkdir $path
echo $BASHPID >$path/cgroup.procs
qemu-system-x86_64 -M microvm,accel=kvm -cpu host -smp 2 -m 6g \
-nographic -kernel /boot/vmlinuz -initrd ./initrd.img \
-append "console=ttyS0 loglevel=0"
}
for ((memcg = 0; memcg < $memcgs; memcg++)); do
run $memcg &
done
wait
Results
=======
Before [1] After Change
----------------------------------------------
Total samples 6824 7237 +6%
Notes
=====
[1] "mm: rmap: Don't flush TLB after checking PTE young for page
reference" was included so that the comparison is apples to
Apples.
https://lore.kernel.org/r/20220706112041.3831-1-21cnbao@xxxxxxxxx/
