On Thu, Mar 16, 2017 at 6:58 AM, Tomasz Kłoczko
<kloczko.tomasz@xxxxxxxxx> wrote:
>
> On 16 March 2017 at 04:50, Nico Kadel-Garcia <nkadel@xxxxxxxxx> wrote:
> [..]
>>
>> > And one more clarification: remove static libraries from glibc distro
>> > packages does not blocks static linking.
>> > It will only removes possibility linking against static glibc libraries.
>>
>> Yes. This is a blocker for some people, who want *completely* static
>> binaries to have complete binary control of their active package.
>> glibc changing from a random update and possibly introducing a
>> regression problem is anathema to some critical software developers,
>> and the compilation of completely static binaries has been helpful for
>> cross-platform compatibility, for building chroot cages, and for
>> building well managed containers of various sorts.
>
>
> There are few typical scenarios when someone may want to have 100% static
> binaries:
> 1) to have binary independent from ABI changes on distribution level in any
> used shared libraries.
And regression errors not present in the version of the library they
built and tested the software with. This is a *very big deal*. Been
there, done that, got the T-shirt. Having provenance for your
software, and certainty that library updates will not affect anything
but the expected software is critical. Not protecting critical,
high-availability from software patches or updates to the rest of the
operating system isa very, very big deal for some of us.
> So here is kind contradiction because my past experience that such binaries
> are used so long (+6 years) that it causes silent issues with conflicts on
> kernel<->user space and sooner or later initial intention turns into
> disaster as no one remembers who and how initially such binary was made.
That is also a problem. It's also not really solved by dynamic
libraries. if the ABI for the libraries is changing that much, it's
likely necessary to recompile the base software *anyway*.
> 2) some bootstrapping scenarios like for example static linking grub
> binaries
>
> In such cases binaries are will be regenerated with every small change in
> non-public ABI/API changes will be followed by immediate recompilation of
> such binaries so ris here is effective null and such limited number binaries
> should be accepted and carefully maintained.
And... we're to build them without glibc-static.... how?
> Scenario when such special binaries been crafted for initrd already are
> nullified because today even smallest systems have enough memory to use
> regular shared libraries. Simple one one needs today to fit such initrds on
> 3.5' floppy disk with 1.44MB available storage.
Boot media are one of those cases where binary stability are *most* critical.
> 3) some people are thinking that static linking make sense from performance
> or resource consumption perspective.
>
> Here is the issue that such binaries would be typically moved/propagated
> around to different location executed in multiple instances. What was
> initial intention of saving memory typically turns into higher memory
> consumption. As long as such binaries will be detached from original
> locations installed by regular locations risk that those binaries will be
> used longer to expose them to risk created by kernel<->user space ABI
> changes and internal glibc is relatively high.
Straw man argument, 5 yard penalty.
The relocation of binaries to different places in the same operating
system is not the problem. Moving it from one operating sytem or
container to another, *that* is the problem. And running exactly the
version you expect, even if it's inside a heavily modified container
or distinct operating system of some sort is a powerful goal. I don't
run into it as much as I used to, but I certainly used to have to
compile components statically in various environments for use
elsewhere.
>> You seem to pointing out that NSS is a stability problem. You're quite
>> right. Saying that "NSS is unstable, therefore glibc should be forced
>> to dynamic libraries only" does not follow. The underlying API for
>> nsswitch.conf and NSS does not change that quickly, it's the feature
>> churn for add-ons that are being tied into NSS. For high stability
>> software, *who cared*? You won't use the most recent NSS changes, and
>> if you do, they're quite likely to be available the the glibc static
>> library at the time you compile them. Statically.
>
>
> You are wrong that this is abut messiness/stability of the NSS interface.
> We are talking about internal glibc ABI/API on which none of the
> system/distribution binaries should rely on such internal interfaces which
> any project maintainer has freedom to change without even noticing this in
> changelogs.
If they're dinking with it that much, it's likely to lead to changes
in the ABI as well and force recompilation. That level of change is
not normally permitted within a single Fedora release, and packages
for Fedora are rebuilt for new releases.
Again, straw man argument, and another 5 yeard penalty.
> Using statically linked binaries creates RISK here if those binaries will be
> not refreshed.
Now, *that* is real. However, see above, to wit:
* Packages for Fedora are rebuilt for new releases.
If you install a package from a previous Fedora release, or run it on
your newer Fedora release, well, you have a risk of trouble. Mind you,
I've *done* so when build environments for older software didn't work
for new releases. It's more common with highly proprietary software
whose authors tended to "tune" their performance at the cost of
portability.
> As long as number of such binaries will be limited and will be under regular
> distro hood control I have no problems with such binaries.
> As long as whole distribution consumer may start using such internal ABI
> interface we are entering on area where sooner or later initial honest
> intention will turn against someone who had such intention.
> Here is really end of the story if you will really accept meaning,
> consequences and existence of internal libraries APIs/ABIs. This is why
> executable binaries have public symbols tables. NSS is not part of the
> public interface.
>
> Risk of not to be exposed on internal NSS ABI changes can be very easy
> nullified by using in such rare cases static linking with libc like uClibc
> which has no NSS interface.
And if I walk on only my right foot, I can save wear and tear on my
left shoe. Doesn't seem worth the effort. uClibc is a cool idea, but
unlikely to work well for a broad variety of complex software which
has ever been compiled or tested with it.
> In last decade bash already created enough number CVEs to start thinking
> about moving away from bash as /bin/sh provider.
> Sun/Oracle already done careful ksh security review which bash never had
> done. Because ksh is very small and his main goal is provide /bin/sh number
> of any future changes will be very well known and limited to cleanups and
> bug fixes.
And we can get C++ programmers to going back to C, to avoid the
function overloading problem.
I'm afraid it's not gonna happen.
> Changing /bin/sh to real SH interpreter IMO should be kind of long term
> Fedora target. It will be not easy and maybe even painful but minimize
> /bin/sh dependencies and minimize security risk is IMO worth to start
> thinking about some preparations to be opened on such change in the future.
> Other issue is that bash is not the fastest /bin/sh interpreter :)
It's a lot of work with a profound loss of shell functionality and
features, with a nebulous security benefit. Sorry, but I don't see it
happening anytime in the foreseeable future.
> Again: minimize /bin/sh dependencies. Minimize number of other dependencies.
> Here are laying biggest deposits of minimize such risks (waaay bigger than
> those related to static linking).
> Small example.
> Long time ago I've been able to gain minimize number of dependencies by
> injecting LDFLAGS="-Wl,--as-needed" into %configure macros. As on mean time
> cmake emerged this move will be not so effective as it was decade ago. Today
> I think that better solution could apply small change in ld default behavior
> to use by default ---as-need (and make -fno-as-needed optional).
> This could be done in one few lines patch (few lines .. because it would be
> good IMO add printing warning that linking with some libraries was dropped).
> Result would be reduction whole distribution rpm REQUIRES entries by at
> least 10-20%.
Now, *THAT* is an interesting point. I like it. Want to toss in a
feature request for it?
> For example Fedora rawhide still is in the middle openssl migration to
> 1.1.x. If before this migration all packages would be linked with --as-need
> it will be necessary to rebuild much less packages.
> Printing by ld warnings may allow fix original build code by remove
> redundant -l<foo> in those frameworks implementations.
> Few weeks ago I've done short experiment with injecting
> LDFLAGS="-Wl,--as-need" into %configure parameters of the gnome-shell.spec.
> After producing two binary packages with and without -Wl,--as-need here is
> the diff of the output of the queries:
>
> $ rpm -qp --qf "[%{REQUIRENAME} %{REQUIREFLAGS:depflags}
> %{REQUIREVERSION}\n]" gnome-shell-3.2*.fc26.x86_64.rpm
>
> --- gnome-shell.cur 2016-09-18 12:51:51.496412774 +0100
> +++ gnome-shell.new 2016-09-19 20:16:59.613362311 +0100
> @@ -62,10 +62,6 @@
> libgthread-2.0.so.0()(64bit)
> libgtk-3.so.0()(64bit)
> libical.so.2()(64bit)
> -libicalss.so.2()(64bit)
> -libicalvcal.so.2()(64bit)
> -libicui18n.so.57()(64bit)
> -libicuuc.so.57()(64bit)
Ye gods, yes, this kind of dependency improvement is invaluable.
> libinput.so.10()(64bit)
> libjson-glib-1.0.so.0()(64bit)
> libm.so.6()(64bit)
> @@ -96,7 +92,6 @@
> libsecret-1.so.0()(64bit)
> libsoup-2.4.so.1()(64bit)
> libstartup-notification-1.so.0()(64bit)
> -libstdc++.so.6()(64bit)
That it includes libstdc++ is fascinating.
> Back to linker optimisations.
> I know that in some extreme packages cases using linker with --as-need can
> halven number of SONAME package dependencies.
I *like* it!!!
> In other words all those goals which you mention are not strictly related to
> static linking and it is possible to do a lot (or much more) to minimize
> such risks without even thinking about provide glibc-static. Static linking
Umm. No. You're mistaking "reduction of library dependencies" with
"discaarding static library compilation".
> with glibc touches at the moment only few packages. Reduction of SONAME
> dependencies will be related probably to ~3/4 of all Fedora packages so we
> are talking about ~15k packages.
> Do you see how few lines change cay change few magnitude orders more? (if
> you are really care about regression problems/risks coming from
> dependencies).
As I said, i *like* this.
> Spreading ld with a bit different behavior printing warnings about overuse
> some -l<foo> in longer consequences will not only improve Fedora binaries
> health but as long as source code maintainers of the packages used by Fedora
> will start use changed ld they will see that they can improve linker
> settings of own projects.
> Such subjects should be consulted with binutils maintainers because they
> would able to asses risk of doing such change.
> As long as I've done many times full recompilation on the scale of whole
> distribution in the past I know that sometimes is necessary to add small
> linker fix when library X is linked with other libraries A and B and only A
> is used and linked executable is using libX and libB ABI but has no -lB in
> linker options, but majority of those changes have been already merged to
> most of the packages source trees.
*Cool*. And you've my sympathies, those kinds of subtle undetected
dependencies can be tricky to resolve.
> kloczek
> --
> Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
>
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
>
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
