On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@xxxxxxxxxxx> wrote:
> On 10-10-2017 15:31, kefu chai wrote:
>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@xxxxxxxxxxx> wrote:
>>> On 10-10-2017 13:14, kefu chai wrote:
>>>> + ceph-devel
>>>>
>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@xxxxxxxxxxx> wrote:
>>>>> Hi Kefu,
>>>>>
>>>>> Sort of wondering what in the current code below is the relation between
>>>>> gpertools and tcmalloc?
>>>>
>>>> could you be more specific?
>>>
>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>> ALLOCATOR is defined.
>>> There the current flow of code seems to suggest that gperf-tools is
>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>> is set to tcmalloc. This then breaks rocksdb building.
>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>> like:
>>> ```
>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>> gperf-tools
>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>> 'cmake -DALLOCATOR=libc'.
>>> Call Stack (most recent call first):
>>> src/CMakeLists.txt:825 (build_rocksdb)
>>> ```
>>>
>>>>> Reason for the question is that once the gperf packages is installed,
>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>
>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>> you elaborate this a little bit?
>>>
>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>> to work.
>>>
>>> But the question is:
>>> Do I really want my allocator to be tcmalloc when the native
>>> libc allocator is jemalloc?
>>
>> i think it's up to you.
>
> Oke, it is what the code now forces...
> And I've tried avoiding tcmalloc uptil now, and would like to do so for
> the time being..
>
>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>
>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>
> The tracker suggests that it is an error in 2.5.93, but
> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
> 2.6.2...??
2.6.2 is not yet released. so if you want to stick with tcmalloc, you
need to build from the latest (or recent enough) source. actually, i
am not sure what you question is..
>
>>> The workaround this (for the time being) is to call cmake with
>>> -DALLOCATOR=libc as is suggested.
>
> I'll use this as workaround for the time being, since that is least
> invasive. And when that works out, try and make a PR for the allocator
> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
> available on FreeBSD.
what is issue you want to address?
>
> --WjW
>
>>> [1]
>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>>
>>>>
>>>> --
>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>>
>>>>>
>>>>> So I guess I need to fix the code below the to prevent this implicit
>>>>> relation
>>>>> under FreeBSD.
>>>>>
>>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>>> because
>>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>>> The introduction of the new rocksdb building makes this clear.
>>>>>
>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>> gperf-tools
>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>>> -DALLOCATOR=libc'.
>>>>> Call Stack (most recent call first):
>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>
>>>>>
>>>>> --WjW
>>>>>
>>>>>
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>>>> if(ALLOCATOR)
>>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>>>> find_package(gperftools REQUIRED)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>>>> set(HAVE_LIBTCMALLOC ON)
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>>>> find_package(JeMalloc REQUIRED)
>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>>>> set(HAVE_JEMALLOC 1)
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>>>> else(ALLOCATOR)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>>>> find_package(gperftools)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>>>> GPERFTOOLS_FOUND)
>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>>>> find_package(JeMalloc)
>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>>>> endif(NOT GPERFTOOLS_FOUND)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>>>> if(GPERFTOOLS_FOUND)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>>>> set(ALLOCATOR tcmalloc)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>>>> elseif(JEMALLOC_FOUND)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>>>> set(ALLOCATOR jemalloc)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>>> if(NOT FREEBSD)
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>>>> FreeBSD already has jemalloc as its default allocator
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>>>> set(ALLOCATOR "libc")
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>>>> endif(GPERFTOOLS_FOUND)
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>>>> endif(ALLOCATOR)
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
--
Regards
Kefu Chai
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
