On 16 March 2017 at 04:50, Nico Kadel-Garcia <nkadel@xxxxxxxxx> wrote:
[..]
> And one more clarification: remove static libraries from glibc distro
> packages does not blocks static linking.
> It will only removes possibility linking against static glibc libraries.
Yes. This is a blocker for some people, who want *completely* static
binaries to have complete binary control of their active package.
glibc changing from a random update and possibly introducing a
regression problem is anathema to some critical software developers,
and the compilation of completely static binaries has been helpful for
cross-platform compatibility, for building chroot cages, and for
building well managed containers of various sorts.
There are few typical scenarios when someone may want to have 100% static binaries:
1) to have binary independent from ABI changes on distribution level in any used shared libraries.
So here is kind contradiction because my past experience that such binaries are used so long (+6 years) that it causes silent issues with conflicts on kernel<->user space and sooner or later initial intention turns into disaster as no one remembers who and how initially such binary was made.
2) some bootstrapping scenarios like for example static linking grub binaries
In such cases binaries are will be regenerated with every small change in non-public ABI/API changes will be followed by immediate recompilation of such binaries so ris here is effective null and such limited number binaries should be accepted and carefully maintained.
Scenario when such special binaries been crafted for initrd already are nullified because today even smallest systems have enough memory to use regular shared libraries. Simple one one needs today to fit such initrds on 3.5' floppy disk with 1.44MB available storage.
3) some people are thinking that static linking make sense from performance or resource consumption perspective.
Here is the issue that such binaries would be typically moved/propagated around to different location executed in multiple instances. What was initial intention of saving memory typically turns into higher memory consumption. As long as such binaries will be detached from original locations installed by regular locations risk that those binaries will be used longer to expose them to risk created by kernel<->user space ABI changes and internal glibc is relatively high.
[..]
> Number of changes in glibc in internal NSS interface where quite big in
> recent years, and if someone will/is using statically linked binary where
> NSS interface is in use such binary will be loading NSS modules (probably)
> compiled against not correct glibc.Result: execution of such binaries will
> fail or even crash with SIGSEV.
> This scenario is covered literally as well in glibc documentation:
> https://sourceware.org/glibc/wiki/FAQ#Even_statically_ linked_programs_need_some_ shared_libraries_which_is_not_ acceptable_for_me.__What_can_ I_do.3F
You seem to pointing out that NSS is a stability problem. You're quite
right. Saying that "NSS is unstable, therefore glibc should be forced
to dynamic libraries only" does not follow. The underlying API for
nsswitch.conf and NSS does not change that quickly, it's the feature
churn for add-ons that are being tied into NSS. For high stability
software, *who cared*? You won't use the most recent NSS changes, and
if you do, they're quite likely to be available the the glibc static
library at the time you compile them. Statically.
You are wrong that this is abut messiness/stability of the NSS interface.
We are talking about internal glibc ABI/API on which none of the system/distribution binaries should rely on such internal interfaces which any project maintainer has freedom to change without even noticing this in changelogs.
Using statically linked binaries creates RISK here if those binaries will be not refreshed.
As long as number of such binaries will be limited and will be under regular distro hood control I have no problems with such binaries.
As long as whole distribution consumer may start using such internal ABI interface we are entering on area where sooner or later initial honest intention will turn against someone who had such intention.
Here is really end of the story if you will really accept meaning, consequences and existence of internal libraries APIs/ABIs. This is why executable binaries have public symbols tables. NSS is not part of the public interface.
Risk of not to be exposed on internal NSS ABI changes can be very easy nullified by using in such rare cases static linking with libc like uClibc which has no NSS interface.
> Example: RH/Fedora is using bash which is linked against own copy of
> readline (provided within bash original source tree). It was few CVEs in
> readline in recent years and some people been exploiting fact that bash
> was/is linked against readline with well known security issue.
> So here is surprise: in case finding new 0-day bug(s) in readline it will be
> necessary to provide not one readline fixed package but two .. readline and
> bash.
>
> Yes. RH/Fedora bash has some very well known security flaw which hanging
> above whole distro reputation like a Damocles sword!!!
> In last 20 years I've been trying to convince few people to remove exactly
> this bash risk but last time when I've been trying this my English was not
> good enough to express this enough clear and strong. Maybe this time ..
Good luck with that. Then please, raise it. I don't think you're going
to get anywhere with it. Bash is one of the components that *must* be
stable. A glibc update that introduces a problem could break the very
update processes used to update bash.
Just try to check *BSDs, Solaris and even few other Linux distros how it is with readline linking before raising this that it is about stability of something.
BTW: this subject has a bit bigger context as bash provides /bin/sh in Fedora.
Problem is that other OSes are a bit more strict about using non-SH syntax when "#!/bin/sh" is used in scrip preamble. Occasionally it causes some confusion when such script written on Linux will be moved to other OS platform and suddenly it stop correctly working.
/bin/sh is one of the core parts of the OS binaries. Dependencies here should be as short as it is really only possible.
All those problems is possible to solve in few months (I've done it one time on scale whole Linux distribution) by changing /bin/sh to real SH like ksh93 or pdksh (today ksh is better candidate).
A lot of work could to be done before final switch but it will produce system base image without risks which are come with bash as /bin/sh provider.
In last decade bash already created enough number CVEs to start thinking about moving away from bash as /bin/sh provider.
Sun/Oracle already done careful ksh security review which bash never had done. Because ksh is very small and his main goal is provide /bin/sh number of any future changes will be very well known and limited to cleanups and bug fixes.
Changing /bin/sh to real SH interpreter IMO should be kind of long term Fedora target. It will be not easy and maybe even painful but minimize /bin/sh dependencies and minimize security risk is IMO worth to start thinking about some preparations to be opened on such change in the future.
Other issue is that bash is not the fastest /bin/sh interpreter :)
Nevertheless I don't want to discuss this subject now. This should be discussed separately.
If someone want to continue please extract above and change subject.
[..]
> Summarising. There are at least three reasons why static libraries should be
> completely removed from distribution:
> 1) Constant ABI changes
> 2) Security risk
> 3) Waste of time/resources on building and providing static libraries
And reasons to keep them.
1) Complete software stability
2) Regession failures generated by buggy library updates
3) The difficulty of forcing restarts of all daemons which use the
updated libraries to incorporate the updates, rather than forcing an
update of the package itself with associated, schedulable daemon
restarts.
Again: minimize /bin/sh dependencies. Minimize number of other dependencies. Here are laying biggest deposits of minimize such risks (waaay bigger than those related to static linking).
Small example.
Long time ago I've been able to gain minimize number of dependencies by injecting LDFLAGS="-Wl,--as-needed" into %configure macros. As on mean time cmake emerged this move will be not so effective as it was decade ago. Today I think that better solution could apply small change in ld default behavior to use by default ---as-need (and make -fno-as-needed optional).
This could be done in one few lines patch (few lines .. because it would be good IMO add printing warning that linking with some libraries was dropped). Result would be reduction whole distribution rpm REQUIRES entries by at least 10-20%.
For example Fedora rawhide still is in the middle openssl migration to 1.1.x. If before this migration all packages would be linked with --as-need it will be necessary to rebuild much less packages.
Printing by ld warnings may allow fix original build code by remove redundant -l<foo> in those frameworks implementations.
Few weeks ago I've done short experiment with injecting LDFLAGS="-Wl,--as-need" into %configure parameters of the gnome-shell.spec.
After producing two binary packages with and without -Wl,--as-need here is the diff of the output of the queries:
$ rpm -qp --qf "[%{REQUIRENAME} %{REQUIREFLAGS:depflags} %{REQUIREVERSION}\n]" gnome-shell-3.2*.fc26.x86_64.rpm
--- gnome-shell.cur 2016-09-18 12:51:51.496412774 +0100 +++ gnome-shell.new 2016-09-19 20:16:59.613362311 +0100 @@ -62,10 +62,6 @@ libgthread-2.0.so.0()(64bit) libgtk-3.so.0()(64bit) libical.so.2()(64bit) -libicalss.so.2()(64bit) -libicalvcal.so.2()(64bit) -libicui18n.so.57()(64bit) -libicuuc.so.57()(64bit) libinput.so.10()(64bit) libjson-glib-1.0.so.0()(64bit) libm.so.6()(64bit) @@ -96,7 +92,6 @@ libsecret-1.so.0()(64bit) libsoup-2.4.so.1()(64bit) libstartup-notification-1.so.0()(64bit) -libstdc++.so.6()(64bit) libsystemd.so.0()(64bit) libsystemd.so.0(LIBSYSTEMD_209)(64bit) libtelepathy-glib.so.0()(64bit) @@ -114,12 +109,10 @@
As you see only this package is not using libstdc++ and in case of any ABI changes in libstdc++ library force rebuild of this package will be not necessary.
In this exact case libical SONAME dependencies have been reduced so ld.so will be not allocating and initializing jump tables between some binaries and ical libraries.
Remove libstdc++ dependency has been done by jet another not widely known trick that gcc can be used to compile C++ code as long as it is relatively simple code. So in this case %configure parameters has been changes to add:
%configure \
+ CXX="%{__cc}" \
+ CXXFLAGS="%{optflags} -fno-rtti -fno-exceptions" \
+ LDFLAGS="-Wl,--as-needed" \
In some extreme cases using -fno-exceptions can reduce size of the generated code by up to 30% and such code will be no longer using libstdc++.
Such compile time modification may be a bit risk in case using it to compile libraries (as produced libraries will be not possible to handle exceptions across C++ object inheritance trees but some projects like KDE have strict policy about not use and not provide exceptions support as to heavy technique (on code size) so they are already in original build framework deliberately injecting -fno-exceptions, however in case regular executable program binaries this trick brings only good things.
Back to linker optimisations.
I know that in some extreme packages cases using linker with --as-need can halven number of SONAME package dependencies.
In other words all those goals which you mention are not strictly related to static linking and it is possible to do a lot (or much more) to minimize such risks without even thinking about provide glibc-static. Static linking with glibc touches at the moment only few packages. Reduction of SONAME dependencies will be related probably to ~3/4 of all Fedora packages so we are talking about ~15k packages.
Do you see how few lines change cay change few magnitude orders more? (if you are really care about regression problems/risks coming from dependencies).
Spreading ld with a bit different behavior printing warnings about overuse some -l<foo> in longer consequences will not only improve Fedora binaries health but as long as source code maintainers of the packages used by Fedora will start use changed ld they will see that they can improve linker settings of own projects.
Such subjects should be consulted with binutils maintainers because they would able to asses risk of doing such change.
As long as I've done many times full recompilation on the scale of whole distribution in the past I know that sometimes is necessary to add small linker fix when library X is linked with other libraries A and B and only A is used and linked executable is using libX and libB ABI but has no -lB in linker options, but majority of those changes have been already merged to most of the packages source trees.
kloczek
--
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
