On 10-10-2017 15:31, kefu chai wrote:
> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@xxxxxxxxxxx> wrote:
>> On 10-10-2017 13:14, kefu chai wrote:
>>> + ceph-devel
>>>
>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@xxxxxxxxxxx> wrote:
>>>> Hi Kefu,
>>>>
>>>> Sort of wondering what in the current code below is the relation between
>>>> gpertools and tcmalloc?
>>>
>>> could you be more specific?
>>
>> While looking at the code of CMakeLists.txt:286, in the case that no
>> ALLOCATOR is defined.
>> There the current flow of code seems to suggest that gperf-tools is
>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>> is set to tcmalloc. This then breaks rocksdb building.
>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>> like:
>> ```
>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>> gperf-tools
>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>> 'cmake -DALLOCATOR=libc'.
>> Call Stack (most recent call first):
>> src/CMakeLists.txt:825 (build_rocksdb)
>> ```
>>
>>>> Reason for the question is that once the gperf packages is installed,
>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>> But gperf under FreeBSD does not have tcmalloc.
>>>
>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>> you elaborate this a little bit?
>>
>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>> And yes, gperftools brings tcmalloc but not in the version mix that is
>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>> on FreeBSD[1]. Getting other versions will require manual work to get it
>> to work.
>>
>> But the question is:
>> Do I really want my allocator to be tcmalloc when the native
>> libc allocator is jemalloc?
>
> i think it's up to you.
Oke, it is what the code now forces...
And I've tried avoiding tcmalloc uptil now, and would like to do so for
the time being..
>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>
> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
The tracker suggests that it is an error in 2.5.93, but
cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
2.6.2...??
>> The workaround this (for the time being) is to call cmake with
>> -DALLOCATOR=libc as is suggested.
I'll use this as workaround for the time being, since that is least
invasive. And when that works out, try and make a PR for the allocator
selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
available on FreeBSD.
--WjW
>> [1]
>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>
>>>
>>> --
>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>
>>>>
>>>> So I guess I need to fix the code below the to prevent this implicit
>>>> relation
>>>> under FreeBSD.
>>>>
>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>> because
>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>> The introduction of the new rocksdb building makes this clear.
>>>>
>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>> gperf-tools
>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>> -DALLOCATOR=libc'.
>>>> Call Stack (most recent call first):
>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>
>>>>
>>>> --WjW
>>>>
>>>>
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>>> if(ALLOCATOR)
>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>>> find_package(gperftools REQUIRED)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>>> set(HAVE_LIBTCMALLOC ON)
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>>> find_package(JeMalloc REQUIRED)
>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>>> set(HAVE_JEMALLOC 1)
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>>> else(ALLOCATOR)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>>> find_package(gperftools)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>>> GPERFTOOLS_FOUND)
>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>>> find_package(JeMalloc)
>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>>> endif(NOT GPERFTOOLS_FOUND)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>>> if(GPERFTOOLS_FOUND)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>>> set(ALLOCATOR tcmalloc)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>>> elseif(JEMALLOC_FOUND)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>>> set(ALLOCATOR jemalloc)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>> if(NOT FREEBSD)
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>>> FreeBSD already has jemalloc as its default allocator
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>>> set(ALLOCATOR "libc")
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>>> endif(GPERFTOOLS_FOUND)
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>>> endif(ALLOCATOR)
>>>>
>>>
>>>
>>>
>>
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
