On Tue, Nov 03, 2020 at 07:00:22AM -0600, YiFei Zhu wrote: > My peers pointed out that in my previous benchmark there are still a > few mitigations left on, and suggested to use "noibrs noibpb nopti > nospectre_v2 nospectre_v1 l1tf=off nospec_store_bypass_disable > no_stf_barrier mds=off tsx=on tsx_async_abort=off mitigations=off". > Results with "Mitigations Off" updated: > > Mitigations On Mitigations Off > With Cache Without Cache With Cache Without Cache > Native 18:17.38 18:13.78 17:43.42 17:47.68 > D. no seccomp 18:15.54 18:17.71 17:34.59 17:37.54 > D. + seccomp 20:42.47 20:45.04 17:35.70 17:37.16 > > Whether seccomp is on or off seems not to make much of a difference > for this benchmark. Bitmap being enabled does seem to decrease the > overall compilation time but it also affects where seccomp is off, so > the speedup is probably from other factors. We are thinking about > using more syscall-intensive workloads, such as httpd. Yeah, this is very interesting. That there is anything measurably _slower_ with the cache is surprising. Though with only 4 runs, I wonder if it's still noisy? What happens at 10 runs -- more importantly what is the standard deviation? > Thugh, this does make me wonder, where does the 3-minute overhead with > seccomp with mitigations come from? Is it data cache misses? If that > is the case, can we somehow preload the seccomp bitmap cache maybe? I > mean, mitigations only cause around half a minute slowdown without > seccomp but seccomp somehow amplify the slowdown with an additional > 2.5 minutes, so something must be off here. I assume this is from Indirect Branch Prediction Barrier (IBPB) and Single Threaded Indirect Branch Prediction (STIBP) (which get enabled for threads under seccomp by default). Try booting with "spectre_v2_user=prctl" https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/spectre.html#spectre-mitigation-control-command-line -- Kees Cook
