Thanks for your detailed response. I freshly installed a CentOS 7.6 and run install-deps.sh and do_cmake.sh this time, and it works this time. Maybe the problem was caused by dirty environment. Best, Can Zhang On Fri, Apr 19, 2019 at 6:28 PM Brad Hubbard <bhubbard@xxxxxxxxxx> wrote: > > OK. So this works for me with master commit > bdaac2d619d603f53a16c07f9d7bd47751137c4c on Centos 7.5.1804. > > I cloned the repo and ran './install-deps.sh' and './do_cmake.sh > -DWITH_FIO=ON' then 'make all'. > > # find ./lib -iname '*.so*' | xargs nm -AD 2>&1 | grep > _ZTIN13PriorityCache8PriCacheE > ./lib/libfio_ceph_objectstore.so:00000000018f72d0 V > _ZTIN13PriorityCache8PriCacheE > > # LD_LIBRARY_PATH=./lib ./bin/fio --enghelp=libfio_ceph_objectstore.so > conf : Path to a ceph configuration file > oi_attr_len : Set OI(aka '_') attribute to specified length > snapset_attr_len : Set 'snapset' attribute to specified length > _fastinfo_omap_len : Set '_fastinfo' OMAP attribute to specified length > pglog_simulation : Enables PG Log simulation behavior > pglog_omap_len : Set pglog omap entry to specified length > pglog_dup_omap_len : Set duplicate pglog omap entry to specified length > single_pool_mode : Enables the mode when all jobs run against > the same pool > preallocate_files : Enables/disables file preallocation (touch > and resize) on init > > So my result above matches your result on ubuntu but not on centos. It > looks to me like we used to define in libceph-common but currently > it's defined in libfio_ceph_objectstore.so. For reasons that are > unclear you are seeing the old behaviour. Why this is and why it isn't > working as designed is not clear to me but I suspect if you clone the > repo again and build from scratch (maybe in a different directory if > you wish to keep debugging, see below) you should get a working > result. Could you try that as a test? > > If, on the other hand, you wish to keep debugging your current > environment I'd suggest looking at the output of the following command > as it may shed further light on the issue. > > # LD_DEBUG=all LD_LIBRARY_PATH=./lib ./bin/fio > --enghelp=libfio_ceph_objectstore.so > > 'LD_DEBUG=lib' may suffice but that's difficult to judge without > knowing what the problem is. I still suspect somehow you have > mis-matched libraries and, if that's the case, it's probably not worth > pursuing. If you can give me specific steps so I can reproduce this > from a freshly cloned tree I'd be happy to look further into it. > > Good luck. > > On Thu, Apr 18, 2019 at 7:00 PM Brad Hubbard <bhubbard@xxxxxxxxxx> wrote: > > > > Let me try to reproduce this on centos 7.5 with master and I'll let > > you know how I go. > > > > On Thu, Apr 18, 2019 at 3:59 PM Can Zhang <can@xxxxxxx> wrote: > > > > > > Using the commands you provided, I actually find some differences: > > > > > > On my CentOS VM: > > > ``` > > > # sudo find ./lib* -iname '*.so*' | xargs nm -AD 2>&1 | grep > > > _ZTIN13PriorityCache8PriCacheE > > > ./libceph-common.so:000000000221cc08 V _ZTIN13PriorityCache8PriCacheE > > > ./libceph-common.so.0:000000000221cc08 V _ZTIN13PriorityCache8PriCacheE > > > ./libfio_ceph_objectstore.so: U _ZTIN13PriorityCache8PriCacheE > > > ``` > > > ``` > > > # ldd libfio_ceph_objectstore.so |grep common > > > libceph-common.so.0 => /root/ceph/build/lib/libceph-common.so.0 > > > (0x00007fd13f3e7000) > > > ``` > > > On my Ubuntu VM: > > > ``` > > > $ sudo find ./lib* -iname '*.so*' | xargs nm -AD 2>&1 | grep > > > _ZTIN13PriorityCache8PriCacheE > > > ./libfio_ceph_objectstore.so:00000000019d13e0 V _ZTIN13PriorityCache8PriCacheE > > > ``` > > > ``` > > > $ ldd libfio_ceph_objectstore.so |grep common > > > libceph-common.so.0 => > > > /home/can/work/ceph/build/lib/libceph-common.so.0 (0x00007f024a89e000) > > > ``` > > > > > > Notice the "U" and "V" from nm results. > > > > > > > > > > > > > > > Best, > > > Can Zhang > > > > > > On Thu, Apr 18, 2019 at 9:36 AM Brad Hubbard <bhubbard@xxxxxxxxxx> wrote: > > > > > > > > Does it define _ZTIN13PriorityCache8PriCacheE ? If it does, and all is > > > > as you say, then it should not say that _ZTIN13PriorityCache8PriCacheE > > > > is undefined. Does ldd show that it is finding the libraries you think > > > > it is? Either it is finding a different version of that library > > > > somewhere else or the version you have may not define that symbol. > > > > > > > > On Thu, Apr 18, 2019 at 11:12 AM Can Zhang <can@xxxxxxx> wrote: > > > > > > > > > > It's already in LD_LIBRARY_PATH, under the same directory of > > > > > libfio_ceph_objectstore.so > > > > > > > > > > > > > > > $ ll lib/|grep libceph-common > > > > > lrwxrwxrwx. 1 root root 19 Apr 17 11:15 libceph-common.so -> > > > > > libceph-common.so.0 > > > > > -rwxr-xr-x. 1 root root 211853400 Apr 17 11:15 libceph-common.so.0 > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > Can Zhang > > > > > > > > > > On Thu, Apr 18, 2019 at 7:00 AM Brad Hubbard <bhubbard@xxxxxxxxxx> wrote: > > > > > > > > > > > > On Wed, Apr 17, 2019 at 1:37 PM Can Zhang <can@xxxxxxx> wrote: > > > > > > > > > > > > > > Thanks for your suggestions. > > > > > > > > > > > > > > I tried to build libfio_ceph_objectstore.so, but it fails to load: > > > > > > > > > > > > > > ``` > > > > > > > $ LD_LIBRARY_PATH=./lib ./bin/fio --enghelp=libfio_ceph_objectstore.so > > > > > > > > > > > > > > fio: engine libfio_ceph_objectstore.so not loadable > > > > > > > IO engine libfio_ceph_objectstore.so not found > > > > > > > ``` > > > > > > > > > > > > > > I managed to print the dlopen error, it said: > > > > > > > > > > > > > > ``` > > > > > > > dlopen error: ./lib/libfio_ceph_objectstore.so: undefined symbol: > > > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > > > > > > > > $ c++filt _ZTIN13PriorityCache8PriCacheE > > > > > > typeinfo for PriorityCache::PriCache > > > > > > > > > > > > $ sudo find /lib* /usr/lib* -iname '*.so*' | xargs nm -AD 2>&1 | grep > > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > > /usr/lib64/ceph/libceph-common.so:00000000008edab0 V > > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > > /usr/lib64/ceph/libceph-common.so.0:00000000008edab0 V > > > > > > _ZTIN13PriorityCache8PriCacheE > > > > > > > > > > > > It needs to be able to find libceph-common, put it in your path or preload it. > > > > > > > > > > > > > ``` > > > > > > > > > > > > > > I found a not-so-relevant > > > > > > > issue(https://tracker.ceph.com/issues/38360), the error seems to be > > > > > > > caused by mixed versions. My build environment is CentOS 7.5.1804 with > > > > > > > SCL devtoolset-7, and ceph is latest master branch. Does someone know > > > > > > > about the symbol? > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > Can Zhang > > > > > > > > > > > > > > Best, > > > > > > > Can Zhang > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 16, 2019 at 8:37 PM Igor Fedotov <ifedotov@xxxxxxx> wrote: > > > > > > > > > > > > > > > > Besides already mentioned store_test.cc one can also use ceph > > > > > > > > objectstore fio plugin > > > > > > > > (https://github.com/ceph/ceph/tree/master/src/test/fio) to access > > > > > > > > standalone BlueStore instance from FIO benchmarking tool. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Igor > > > > > > > > > > > > > > > > On 4/16/2019 7:58 AM, Can ZHANG wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I'd like to run a standalone Bluestore instance so as to test and tune > > > > > > > > > its performance. Are there any tools about it, or any suggestions? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > Can Zhang > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > ceph-users mailing list > > > > > > > > > ceph-users@xxxxxxxxxxxxxx > > > > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > _______________________________________________ > > > > > > > ceph-users mailing list > > > > > > > ceph-users@xxxxxxxxxxxxxx > > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Cheers, > > > > > > Brad > > > > > > > > > > > > > > > > -- > > > > Cheers, > > > > Brad > > > > > > > > -- > > Cheers, > > Brad > > > > -- > Cheers, > Brad _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |