On Mon, 23 Jul 2018, kefu chai wrote: > in xenial, we are using the toolchain test build ppa[0] for GCC-7.3. > but unlike the hevaily pached GCC offered by SCLs in centos/rhel 7, > the GCC on ubuntu xenial does not link against libstdc++ by default. > we need to use -static-libstdc++ and -static-libgcc to avoid the > runtime dependency on libstdc++ and libgcc of ceph when it runs on > xenial without the ppa repo for building it. > > it's a viable solution. but we need to be careful. as all shared > libraries we are linking against should not rely on libstdc++ if we go > with it. namely, the C++ libraries we are using are: librocksdb, > libsnappy, libtcmalloc. > > * rocksdb is included by our dist tarball, and can be optionally built > by ourself. so it's fine. > * snappy's case is a little bit complicated. we use it in two ways: > - we have a snappy compressor plugin. so we can link the snappy > plugin against it statically. > - rocksdb can optionally use snappy as a inline compress algorithm. > but currently rocksdb's building system does not offer a way to link > against libsnappy.a. so at this moment, we can simply disable > rocksdb's snappy support. and probably, we can address it on the > rocksdb side later on. I think we'll need to make this change to rocksdb. I think we have compression off by default, but there are certainly clusters out there that had it on so it needs to be built in. We can always apply the build change in our fork until it goes upstream. > * tcmalloc is a allocator library offered by gperf-tools, which is > also a C++ library. and libtcmalloc.a is available in xenial's > libgoogle-perftools-dev package. but it not is compiled with -fPIC > option. so we will run into following error: > > /usr/bin/ld: /usr/lib/libtcmalloc.a(libtcmalloc_la-tcmalloc.o): > relocation R_X86_64_32 against `.rodata.str1.8' can not be used when > making a shared object; recompile with -fPIC > /usr/lib/libtcmalloc.a: error adding symbols: Bad value > collect2: error: ld returned 1 exit status > src/CMakeFiles/ceph-common.dir/build.make:434: recipe for target > 'lib/libceph-common.so.0' failed > > recompiling libtcmalloc with -fPIC addresses this issue. but to > recompile it, we need to include gperft-tools' source in our dist > tarball. we also compile without tcmalloc, but on xenial, tcmalloc > offers better performance than the ptmalloc does in multi-threaded > environment, see [1]. i think that's why we introduced make tcmalloc > the default, and also offer no-tcmalloc build of ceph. > > so i think, the safest option is to include gperftool as a submodule > and our dist tarball. This sounds reasonable to me. It would eliminate some of the variability we've seen with problematic gperftools versions, too. This is a great summary... thanks, Kefu! sage > > thoughts? > > -- > [0] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test > [1] https://gperftools.github.io/gperftools/tcmalloc.html > > > > -- > Regards > Kefu Chai > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
