* tip-bot2 for Tony Luck <tip-bot2@xxxxxxxxxxxxx> wrote:
> The following commit has been merged into the x86/asm branch of tip:
>
> Commit-ID: f444a5ff95dce07cf4353cbb85fc3e785019d430
> Gitweb: https://git.kernel.org/tip/f444a5ff95dce07cf4353cbb85fc3e785019d430
> Author: Tony Luck <tony.luck@xxxxxxxxx>
> AuthorDate: Mon, 16 Dec 2019 13:42:54 -08:00
> Committer: Borislav Petkov <bp@xxxxxxx>
> CommitterDate: Wed, 08 Jan 2020 11:29:25 +01:00
>
> x86/cpufeatures: Add support for fast short REP; MOVSB
>
> >From the Intel Optimization Reference Manual:
>
> 3.7.6.1 Fast Short REP MOVSB
> Beginning with processors based on Ice Lake Client microarchitecture,
> REP MOVSB performance of short operations is enhanced. The enhancement
> applies to string lengths between 1 and 128 bytes long. Support for
> fast-short REP MOVSB is enumerated by the CPUID feature flag: CPUID
> [EAX=7H, ECX=0H).EDX.FAST_SHORT_REP_MOVSB[bit 4] = 1. There is no change
> in the REP STOS performance.
>
> Add an X86_FEATURE_FSRM flag for this.
>
> memmove() avoids REP MOVSB for short (< 32 byte) copies. Check FSRM and
> use REP MOVSB for short copies on systems that support it.
>
> [ bp: Massage and add comment. ]
>
> Signed-off-by: Tony Luck <tony.luck@xxxxxxxxx>
> Signed-off-by: Borislav Petkov <bp@xxxxxxx>
> Link: https://lkml.kernel.org/r/20191216214254.26492-1-tony.luck@xxxxxxxxx
BTW., just for the record, the 32-bit version of memmove() has a similar
cut-off as well, at 680 bytes (!):
/*
* movs instruction have many startup latency
* so we handle small size by general register.
*/
"cmp $680, %0\n\t"
"jb 3f\n\t"
...
/*
* Start to prepare for backward copy.
*/
".p2align 4\n\t"
"2:\n\t"
"cmp $680, %0\n\t"
"jb 5f\n\t"
This logic was introduced in 2010 via:
3b4b682becdf: ("x86, mem: Optimize memmove for small size and unaligned cases")
However because those patches came without actual performance
measurements, I'd be inclined to switch back to the old REP MOVSB version
- which would also automatically improve it should anyone run 32-bit
kernels on the very latest CPUs.
Thanks,
Ingo
|