Dear Linux folks,
On an two socket AMD EPYC 7601, we noticed a decrease in raid6 avx2x4
speed shown at the beginning of the boot.
5.4.95 5.10.24
----------------------------------------------
raid6: avx2x4 gen() 18429 MB/s 6155 MB/s
raid6: avx2x4 xor() 6644 MB/s 4274 MB/s
raid6: avx2x2 gen() 17894 MB/s 18744 MB/s
raid6: avx2x2 xor() 11642 MB/s 11950 MB/s
raid6: avx2x1 gen() 13992 MB/s 17112 MB/s
raid6: avx2x1 xor() 10855 MB/s 11143 MB/s
We are able to reproduce this with different models: Supermicro
AS-2023US-TR4/H11DSU-iN and Dell PowerEdge R7425 (with different
microcode versions).
Can you reproduce this on your systems?
Bisecting is going to be hard, so the systems are in production and also
take a while to boot. (Maybe kexec would help here.)
Kind regards,
Paul
PS: Some more information:
```
[ 0.000000] Linux version 5.4.97.mx64.368
(root@xxxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 7.5.0 (GCC
)) #1 SMP Wed Feb 10 18:22:50 CET 2021
[…]
[ 0.000000] DMI: Supermicro AS -2023US-TR4/H11DSU-iN, BIOS 1.1 02/07/2018
[…]
[ 0.630603] raid6: avx2x4 gen() 18429 MB/s
[ 0.651607] raid6: avx2x4 xor() 6644 MB/s
[ 0.672605] raid6: avx2x2 gen() 17894 MB/s
[ 0.693603] raid6: avx2x2 xor() 11642 MB/s
[ 0.714605] raid6: avx2x1 gen() 13992 MB/s
[ 0.735604] raid6: avx2x1 xor() 10855 MB/s
[ 0.756607] raid6: sse2x4 gen() 12246 MB/s
[ 0.777605] raid6: sse2x4 xor() 5724 MB/s
[ 0.798605] raid6: sse2x2 gen() 10945 MB/s
[ 0.819603] raid6: sse2x2 xor() 8097 MB/s
[ 0.840606] raid6: sse2x1 gen() 5941 MB/s
[ 0.861606] raid6: sse2x1 xor() 5894 MB/s
[ 0.866565] raid6: using algorithm avx2x4 gen() 18429 MB/s
[ 0.871567] raid6: .... xor() 6644 MB/s, rmw enabled
[ 0.877566] raid6: using avx2x2 recovery algorithm
[…]
```
```
[ 0.000000] Linux version 5.10.24.mx64.375
(root@xxxxxxxxxxxxxxxxxxxxxxxxx) (gcc (GCC) 7.5.0, GNU ld (GNU Binutils)
2.32) #1 SMP Fri Mar 19 12:29:21 CET 2021
[…]
[ 0.000000] DMI: Supermicro AS -2023US-TR4/H11DSU-iN, BIOS 1.1 02/07/2018
[…]
[ 0.655382] raid6: avx2x4 gen() 6155 MB/s
[ 0.676382] raid6: avx2x4 xor() 4274 MB/s
[ 0.697380] raid6: avx2x2 gen() 18744 MB/s
[ 0.718380] raid6: avx2x2 xor() 11950 MB/s
[ 0.739380] raid6: avx2x1 gen() 17112 MB/s
[ 0.760380] raid6: avx2x1 xor() 11143 MB/s
[ 0.781381] raid6: sse2x4 gen() 11062 MB/s
[ 0.802380] raid6: sse2x4 xor() 5180 MB/s
[ 0.823380] raid6: sse2x2 gen() 12467 MB/s
[ 0.844380] raid6: sse2x2 xor() 7672 MB/s
[ 0.865381] raid6: sse2x1 gen() 9733 MB/s
[ 0.886380] raid6: sse2x1 xor() 5717 MB/s
[ 0.890674] raid6: using algorithm avx2x2 gen() 18744 MB/s
[ 0.895673] raid6: .... xor() 11950 MB/s, rmw enabled
[ 0.901673] raid6: using avx2x2 recovery algorithm
```
```
$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 48 bits physical, 48 bits virtual
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 2
Core(s) per socket: 32
Socket(s): 2
NUMA node(s): 8
Vendor ID: AuthenticAMD
CPU family: 23
Model: 1
Model name: AMD EPYC 7601 32-Core Processor
Stepping: 2
Frequency boost: enabled
CPU MHz: 3100.798
CPU max MHz: 2200.0000
CPU min MHz: 1200.0000
BogoMIPS: 4399.53
Virtualization: AMD-V
L1d cache: 2 MiB
L1i cache: 4 MiB
L2 cache: 32 MiB
L3 cache: 128 MiB
NUMA node0 CPU(s): 0-7,64-71
NUMA node1 CPU(s): 8-15,72-79
NUMA node2 CPU(s): 16-23,80-87
NUMA node3 CPU(s): 24-31,88-95
NUMA node4 CPU(s): 32-39,96-103
NUMA node5 CPU(s): 40-47,104-111
NUMA node6 CPU(s): 48-55,112-119
NUMA node7 CPU(s): 56-63,120-127
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass
disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers
and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Full AMD retpoline, IBPB
conditional, STIBP disabled, RSB filling
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic
sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx
mmxex
t fxsr_opt pdpe1gb rdtscp lm
constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm
aperfmperf pni pclmulqd
q monitor ssse3 fma cx16 sse4_1 sse4_2
movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic
cr8_lega
cy abm sse4a misalignsse 3dnowprefetch
osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc
mwaitx c
pb hw_pstate ssbd ibpb vmmcall
fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt
xsavec xgetbv1
xsaves clzero irperf xsaveerptr arat
npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid
decodeassists paus
efilter pfthreshold avic
v_vmsave_vmload vgif overflow_recov succor smca
```
