Hi Jonathan,
Thanks for clarifying that explicit class template instantiations should
work. Trying to produce a minimal reproducer was a very interesting
exercise. As it turns out, the problematic situation which I am
experiencing does not occur with explicit template instantiations per
se, but only when those are combined with inheritance in a certain way.
Let us consider a toy library composed of this header...
----
#pragma once
template <typename T>
class Superclass {
public:
void print(const T& value) const;
};
template <typename T>
class Subclass : public Superclass<T> {
public:
void print_sub(const T& value) const;
};
extern template class Subclass<int>;
----
...and this source file...
----
#include <iostream>
#include "lib.hpp"
template <typename T>
void Superclass<T>::print(const T& value) const {
std::cout << "The value is " << value << std::endl;
}
template <typename T>
void Subclass<T>::print_sub(const T& value) const {
this->print(value);
}
template class Subclass<int>;
----
On a regular build of this library, both Superclass<int>::print() and
Subclass<int>::print_sub() are exposed as weak symbols and can be linked
against:
----
00000000000011da w F .text 0000000000000051 Superclass<int>::print(int const&) const
00000000000011b4 w F .text 0000000000000026 Subclass<int>::print_sub(int const&) const
----
On an LTO build from GCC 8.2, however, Superclass<int>::print() becomes
a local symbol which cannot be linked against:
----
000000000000116c l F .text 0000000000000050 Superclass<int>::print(int const&) const
0000000000001146 g F .text 0000000000000025 Subclass<int>::print_sub(int const&) const
----
If Superclass<int> is also explicitly instantiated, both symbols will be
global in the LTO build too. However, having to explicitly instantiate
the whole class hierarchy like this could quickly get tedious.
I would spontaneously expect an explicit instantiation of Subclass to
feature all the code that one needs in order to use Subclass, including
the Superclass code. Is the behavior above expected ?
Cheers,
Hadrien
Le 11/10/2018 à 15:44, Jonathan Wakely a écrit :
On Thu, 11 Oct 2018 at 14:01, Hadrien Grasland <grasland@xxxxxxxxxxxx> wrote:
Hi everyone,
This is a follow-up to my previous e-mail about linkers, now that I
understand the problem that I'm dealing with well enough to describe it :)
I'm trying to enable LTO in a large, heavily templated C++ codebase,
whose developers made a lot of effort to improve compilation efficiency.
To this end...
* Significant effort is expended in separating class declarations and
definitions in regular OO code
* C++11's "extern template" and explicit template instantiations are
extensively used to reduce template-induced duplicate work.
* Shared libraries are also used, sometimes in a cascading fashion
where an executable depends on library A, which depends on another
library B
When I enable LTO, I notice that many weak function template symbols
turn into local symbols, and are thus not available anymore to clients
of a shared library. This is a somewhat expected side-effect of LTO,
since AFAIK one of its design goals is to discard unused symbols.
My problem is that this weak symbol pruning process is in my case a
little bit too agressive. For example, in the "cascading" scenario
above, I end up in a situation where library A fails to link because it
cannot find the function symbols associated with "extern template class"
declarations in the headers of library B. Note that this only happens
with extern template classes: extern template functions keep working as
expected.
That suggests a bug in library B. The explicit instantiation
declaration (the "extern template" bit) needs to be matched by an
explicit instantiation *definition*. That definition should not be
weak, and so should not be discarded.
It sounds like you've been using "extern template" to suppress
implicit instantiations, and then assuming that some other library
will happen to provide the definition via some other implicit
instantiation. With LTO the other implicit instantiations are being
pruned, and the assumption fails.
If you're doing it properly, using explicit instantiation definitions
to provide the needed symbols, then they should not get dropped by the
LTO linker. If they do, I think that's a bug in GCC and/or the linker.
I can work around this by providing library A with the definitions of
the relevant templates instead of the declarations, falling back to a
classic duplicate instantiation model, but I am curious if there is a
better way. Can I somehow structure or annotate the C++ code so that the
linker knows that the explicit template instantiations of library B are
used by its clients (like library A) and must be kept around as weak or
global symbols of the output shared library?
Does library B really have explicit instantiation definitions? When I
create a shared library using LTO any explicit instantiations are
turned into GLOBAL symbols, not WEAK, and are not removed.
It would help if you can create a minimal example showing the problem
you describe.
