Hi,
On 12/28/23 10:12, Aoife Moloney wrote:
Wiki -> https://fedoraproject.org/wiki/Changes/Optimized_Binaries_for_the_AMD64_Architecture
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Additional paths will be inserted into the search path used for
executables on systems which have a compatible CPU.
Those additional paths will mirror the AMD64 "microarchitecture
levels" supported by the glibc-hwcaps mechanism: `x86-64-v2`,
`x86-64-v3`, `x86_64-v4`.
Systemd will be modified to insert the additional directories into the
`$PATH` environment variable (affecting all programs on the system)
and the equivalent internal mechanism in `systemd` (affecting what
executables are used by services).
Individual packages can provide optimized libraries via the
(trimming)
Glibc-hwcaps together with the new feature in systemd provide a
generic mechanism. It will be up to individual packages to actually
provide code which makes use of it. Individual package maintainers are
encouraged to benchmark their packages after recompilation, and
provide the optimized variants if useful. (I.e. the code in question
is measureably faster '''and''' the program is ran often enough for
this to make a difference.)
The Change Owners will implement the packaging changes for a few
packages while developing the general mechanism and will submit those
as pull requests. Other maintainers are asked to do the same for their
packages.
Optimized variants of programs and libraries MAY be packaged in a
separate subpackage. The general packaging rules should be applied,
i.e. a separate package or packages SHOULD be created if it is files
are large enough.
Available benchmark results [2,4] are narrow and not very convincing.
We should plan an evaluation of results after one release. If it
turns out that the real gains are too small, we can scrap the effort.
On the other hand, we should also consider other architectures. For
example, microarchitecture levels `z{14,15}` for `s390x` or
`power{9,10}` for `ppc64le`. Other architectures are not included in
this Change Proposal to reduce its scope.
While I realize this is intended as an amd64 specific change, I'm
interested in helping to assure the infrastructure is in place for
similar testing of aarch64 packages, since we have similar problems.
There remains a desire to support arm v8.0 machines (ex: the rpi4) into
the foreseeable future, but significant uplifts exist for some
packages/workloads with newer arch revisions. In most cases those
features are currently being enabled at runtime but have perf hits for
operations that traditionally are just part of the instruction stream.
Outline-atomics, which leverage the v8.2 atomics are one such example,
but also SVE (vectors, used for crypto, regex, etc), and more recently
the v8.8 CPY (memcpy) should all ideally be inline.
So, while some other organizations have moved their platform/OSs to a
v8.2 baseline, it would be nice to have another fedora tool that can be
utilized to remain perf competitive.
== Feedback ==
== Benefit to Fedora ==
The developers who are interested in this kind of optimization work
can perform it within Fedora, without having to build separate
repositories. The users who have the appropriate hardware will gain
performance benefits. Faster code is also more energy-efficient. The
change will be automatic and transparent to users.
Note that other distributions use higher microarchitecture levels. For
example RHEL 9 uses x86-64-v2 as the baseline, RHEL 10 will use
x86-64-v3, and other distros provide optimized variants (OpenSUSE,
Arch Linux). We implement the same change in Fedora in a way that is
scoped more narrowly, but should provide the same performance and
energy benefits.
== Scope ==
* Proposal owners:
** Extend systemd to set the executable search path using the same
criteria as the dynamic linker.
** Implement packaging changes for at least one package with a library
and at least one package with executables and submit this as pull
requests.
** Provide a pull request for the Packaging Guidelines to describe the
changes listed in Description above.
* Other developers:
** Do benchmarking and implement packaging changes for other packages
if beneficial.
* Release engineering: [https://pagure.io/releng/issue/11864 #11864]
* Policies and guidelines: TBD.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives:
== Upgrade/compatibility impact ==
No impact.
== How To Test ==
* Use `/usr/bin/ld.so --help` to check which hwcaps are supported by the system.
* Install one or more packages which provide optimized code.
* Restart the system or re-login to reinitialize `$PATH`.
* Check that appropriate directories are present in `$PATH`.
* Run some benchmarks and check that the optimized code is indeed faster.
== User Experience ==
There should be no impact for users. If the optimized code is
available and installed for their hardware, various tasks may finish
faster and use less energy.
== Dependencies ==
== Contingency Plan ==
* Contingency mechanism: Undo the changes in packages which introduced
them and recompile.
* Contingency deadline: Any time.
* Blocks release? No.
== Documentation ==
== Release Notes ==
Packages which benefit from being compiled for higher AMD64
microarchitecture levels (`x86-64-v2`, `x86-64-v3`, `x86_64-v4`) are
now provided with optimized variants which will be used automatically
on appropriate CPUs. This includes: TBD1, TBD2, TBD3.
--
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
