Re: ceph 16.2.5: three build issues

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hey Tomasz,

On Sat, Aug 7, 2021 at 10:21 AM kefu chai <tchaikov@xxxxxxxxx> wrote:


And another two are just compile issues:
1)
cd /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/src/kv && /usr/bin/g++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/src/include -I/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src -isystem /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/include -isystem /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/xxHash -isystem /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/rapidjson/include -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -DNDEBUG -fPIC   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -std=c++17 -MD -MT src/kv/CMakeFiles/kv.dir/rocksdb_cache/BinnedLRUCache.cc.o -MF CMakeFiles/kv.dir/rocksdb_cache/BinnedLRUCache.cc.o.d -o CMakeFiles/kv.dir/rocksdb_cache/BinnedLRUCache.cc.o -c /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.cc
In file included from /usr/include/c++/11/x86_64-redhat-linux/bits/c++allocator.h:33,
                 from /usr/include/c++/11/bits/allocator.h:46,
                 from /usr/include/c++/11/string:41,
                 from /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.h:13,
                 from /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.cc:14:
/usr/include/c++/11/ext/new_allocator.h: In instantiation of ‘void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = rocksdb_cache::BinnedLRUCache; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}; _Tp = rocksdb_cache::BinnedLRUCache]’:
/usr/include/c++/11/bits/alloc_traits.h:512:17:   required from ‘static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = rocksdb_cache::BinnedLRUCache; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}; _Tp = rocksdb_cache::BinnedLRUCache; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<rocksdb_cache::BinnedLRUCache>]’
/usr/include/c++/11/bits/shared_ptr_base.h:519:39:   required from ‘std::_Sp_counted_ptr_inplace<_Tp, _Alloc, _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}; _Tp = rocksdb_cache::BinnedLRUCache; _Alloc = std::allocator<rocksdb_cache::BinnedLRUCache>; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’
/usr/include/c++/11/bits/shared_ptr_base.h:650:16:   required from ‘std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = rocksdb_cache::BinnedLRUCache; _Alloc = std::allocator<rocksdb_cache::BinnedLRUCache>; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’
/usr/include/c++/11/bits/shared_ptr_base.h:1342:14:   required from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<rocksdb_cache::BinnedLRUCache>; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}; _Tp = rocksdb_cache::BinnedLRUCache; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’
/usr/include/c++/11/bits/shared_ptr.h:409:59:   required from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<rocksdb_cache::BinnedLRUCache>; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}; _Tp = rocksdb_cache::BinnedLRUCache]’
/usr/include/c++/11/bits/shared_ptr.h:862:14:   required from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = rocksdb_cache::BinnedLRUCache; _Alloc = std::allocator<rocksdb_cache::BinnedLRUCache>; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}]’
/usr/include/c++/11/bits/shared_ptr.h:878:39:   required from ‘std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = rocksdb_cache::BinnedLRUCache; _Args = {ceph::common::CephContext*&, long unsigned int&, int&, bool&, double&}]’
/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.cc:620:42:   required from here
/usr/include/c++/11/ext/new_allocator.h:162:11: error: invalid new-_expression_ of abstract class type ‘rocksdb_cache::BinnedLRUCache’
  162 |         { ::new((void *)__p) _Up(std::forward<_Args>(__args)...); }
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.cc:14:
/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.h:295:7: note:   because the following virtual functions are pure within ‘rocksdb_cache::BinnedLRUCache’:
  295 | class BinnedLRUCache : public ShardedCache {
      |       ^~~~~~~~~~~~~~
In file included from /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/ShardedCache.h:17,
                 from /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.h:16,
                 from /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/kv/rocksdb_cache/BinnedLRUCache.cc:14:
/usr/include/rocksdb/cache.h:343:21: note:     ‘virtual void (* rocksdb::Cache::GetDeleter(rocksdb::Cache::Handle*) const)(const rocksdb::Slice&, void*)’
  343 |   virtual DeleterFn GetDeleter(Handle* handle) const = 0;
      |             
        ^~~~~~~~~~
 
/usr/include/rocksdb/cache.h:368:16: note:     ‘virtual void rocksdb::Cache::ApplyToAllEntries(const std::function<void(const rocksdb::Slice&, void*, long unsigned int, void (*)(const rocksdb::Slice&, void*))>&, const rocksdb::Cache::ApplyToAllEntriesOptions&)’
  368 |   virtual void ApplyToAllEntries(
      |                ^~~~~~~~~~~~~~~~~
make[2]: *** [src/kv/CMakeFiles/kv.dir/build.make:135: src/kv/CMakeFiles/kv.dir/rocksdb_cache/BinnedLRUCache.cc.o] Error 1
make[2]: Target 'src/kv/CMakeFiles/kv.dir/build' not remade because of errors.
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu'
make[1]: *** [CMakeFiles/Makefile2:6679: src/kv/CMakeFiles/kv.dir/all] Error 2

I see you are using distro supplied rocksdb, can you let us know what distro and rocksdb version you are using.
(Meanwhile, if you do not issues with ceph building rocksdb, maybe uninstalling distro package rocksdb and making ceph build it might be worth a try!)

2)
cd /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5 /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/blk /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/src/blk /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/src/blk/CMakeFiles/blk.dir/DependInfo.cmake
Consolidate compiler generated dependencies of target blk
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu'
make  -f src/blk/CMakeFiles/blk.dir/build.make src/blk/CMakeFiles/blk.dir/build
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu'
[ 14%] Building CXX object src/blk/CMakeFiles/blk.dir/BlockDevice.cc.o
cd /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/src/blk && /usr/bin/g++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/src/include -I/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src -I/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/blk/. -isystem /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu/include -isystem /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/xxHash -isystem /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/rapidjson/include -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -DNDEBUG -fPIC   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -std=c++17 -MD -MT src/blk/CMakeFiles/blk.dir/BlockDevice.cc.o -MF CMakeFiles/blk.dir/BlockDevice.cc.o.d -o CMakeFiles/blk.dir/BlockDevice.cc.o -c /home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/blk/BlockDevice.cc
/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/blk/BlockDevice.cc: In static member function ‘static BlockDevice::block_device_t BlockDevice::detect_device_type(const string&)’:
/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/src/blk/BlockDevice.cc:108:26: error: ‘aio’ is not a member of ‘BlockDevice::block_device_t’
  108 |   return block_device_t::aio;
      |                          ^~~
make[2]: *** [src/blk/CMakeFiles/blk.dir/build.make:79: src/blk/CMakeFiles/blk.dir/BlockDevice.cc.o] Error 1
make[2]: Target 'src/blk/CMakeFiles/blk.dir/build' not remade because of errors.
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu'
make[1]: *** [CMakeFiles/Makefile2:6737: src/blk/CMakeFiles/blk.dir/all] Error 2

My cmake settings are:

[tkloczko@barrel x86_64-redhat-linux-gnu]$ cmake -L
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.


CMake Error: The source directory "/home/tkloczko/rpmbuild/BUILD/ceph-16.2.5/x86_64-redhat-linux-gnu" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
-- Cache values
ALLOCATOR:STRING=
BABELTRACE_EXECUTABLE:FILEPATH=BABELTRACE_EXECUTABLE-NOTFOUND
CEPH_GRAFANA_DASHBOARDS_DIR:PATH=/etc/grafana/dashboards/ceph-dashboard
CEPH_MAN_DIR:STRING=share/man
CEPH_SYSTEMD_ENV_DIR:PATH=/etc/sysconfig
CEPH_TEST_TIMEOUT:STRING=3600
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
CMAKE_INSTALL_PREFIX:PATH=/usr
CMAKE_INSTALL_SYSTEMD_SERVICEDIR:PATH=/usr/lib/systemd/system
CTAGS_EXECUTABLE:FILEPATH=/usr/bin/ctags
CTAG_EXCLUDES:BOOL=OFF
DASHBOARD_FRONTEND_LANGS:STRING=
DEBUG_GATHER:BOOL=ON
DIAGNOSTICS_COLOR:STRING=auto
ENABLE_COVERAGE:BOOL=OFF
ENABLE_GIT_VERSION:BOOL=ON
ENABLE_SHARED:BOOL=ON
FUSE_LIBRARIES:FILEPATH=/usr/lib64/libfuse3.so
GMock_GMock_LIBRARY:FILEPATH=/usr/lib64/libgmock.so
GMock_INCLUDE_DIR:PATH=/usr/include
GMock_Main_LIBRARY:FILEPATH=/usr/lib64/libgmock_main.so
GPERF:FILEPATH=/usr/bin/gperf
GTest_DIR:PATH=/usr/lib64/cmake/GTest
HAVE_LEVELDB_FILTER_POLICY:FILEPATH=/usr/include/leveldb/filter_policy.h
ICU_DERB_EXECUTABLE:FILEPATH=/usr/bin/derb
ICU_GENBRK_EXECUTABLE:FILEPATH=/usr/bin/genbrk
ICU_GENCCODE_EXECUTABLE:FILEPATH=/usr/sbin/genccode
ICU_GENCFU_EXECUTABLE:FILEPATH=/usr/bin/gencfu
ICU_GENCMN_EXECUTABLE:FILEPATH=/usr/sbin/gencmn
ICU_GENCNVAL_EXECUTABLE:FILEPATH=/usr/bin/gencnval
ICU_GENDICT_EXECUTABLE:FILEPATH=/usr/bin/gendict
ICU_GENNORM2_EXECUTABLE:FILEPATH=/usr/sbin/gennorm2
ICU_GENRB_EXECUTABLE:FILEPATH=/usr/bin/genrb
ICU_GENSPREP_EXECUTABLE:FILEPATH=/usr/sbin/gensprep
ICU_ICU-CONFIG_EXECUTABLE:FILEPATH=/usr/bin/icu-config
ICU_ICUINFO_EXECUTABLE:FILEPATH=/usr/bin/icuinfo
ICU_ICUPKG_EXECUTABLE:FILEPATH=/usr/sbin/icupkg
ICU_INCLUDE_DIR:PATH=/usr/include
ICU_MAKECONV_EXECUTABLE:FILEPATH=/usr/bin/makeconv
ICU_MAKEFILE_INC:FILEPATH=/usr/lib64/icu/69.1/Makefile.inc
ICU_PKGDATA_EXECUTABLE:FILEPATH=/usr/bin/pkgdata
ICU_PKGDATA_INC:FILEPATH=/usr/lib64/icu/69.1/pkgdata.inc
ICU_UCONV_EXECUTABLE:FILEPATH=/usr/bin/uconv
LIBXFS:FILEPATH=/usr/lib64/libhandle.so
LTTNG_GEN_TP:FILEPATH=/usr/bin/lttng-gen-tp
PG_DEBUG_REFS:BOOL=OFF
PROMTOOL_EXECUTABLE:FILEPATH=PROMTOOL_EXECUTABLE-NOTFOUND
RDMACM_INCLUDE_DIR:PATH=/usr/include
RT_LIBRARY:FILEPATH=/usr/lib64/librt.a
SPAWN_TEST_ADDRESS_SANITIZER:BOOL=ON
SPHINX_BUILD:FILEPATH=/usr/bin/sphinx-build
SQLite3_INCLUDE_DIR:PATH=/usr/include
SQLite3_LIBRARY:FILEPATH=/usr/lib64/libsqlite3.so
Tcmalloc_INCLUDE_DIR:PATH=/usr/include
VERBS_INCLUDE_DIR:PATH=/usr/include
WITH_ASAN:BOOL=OFF
WITH_ASAN_LEAK:BOOL=OFF
WITH_BABELTRACE:BOOL=ON
WITH_BLKIN:BOOL=OFF
WITH_BLUEFS:BOOL=OFF
WITH_BLUESTORE:BOOL=OFF

why did you disable bluestone? Are you only interested in the client side?


WITH_BROTLI:BOOL=OFF
WITH_CCACHE:BOOL=OFF
WITH_CEPHFS:BOOL=ON
WITH_CEPHFS_JAVA:BOOL=OFF
WITH_CEPHFS_SHELL:BOOL=ON
WITH_CEPHFS_TOP:BOOL=ON
WITH_CEPH_DEBUG_MUTEX:BOOL=OFF
WITH_DMCLOCK_TESTS:BOOL=OFF
WITH_DOKAN:BOOL=OFF
WITH_DPDK:BOOL=OFF
WITH_FIO:BOOL=OFF
WITH_FUSE:BOOL=ON
WITH_GRAFANA:BOOL=ON
WITH_GSSAPI:BOOL=OFF
WITH_GTEST_PARALLEL:BOOL=OFF
WITH_JAEGER:BOOL=OFF
WITH_KRBD:BOOL=ON
WITH_KVS:BOOL=ON
WITH_LEVELDB:BOOL=ON
WITH_LIBCEPHFS:BOOL=ON
WITH_LIBCEPHSQLITE:BOOL=ON
WITH_LIBRADOSSTRIPER:BOOL=ON
WITH_LTTNG:BOOL=ON
WITH_LZ4:BOOL=ON
WITH_MANPAGE:BOOL=ON
WITH_MGR:BOOL=ON
WITH_MGR_DASHBOARD_FRONTEND:BOOL=OFF
WITH_OCF:BOOL=ON
WITH_OPENLDAP:BOOL=ON
WITH_OSD_INSTRUMENT_FUNCTIONS:BOOL=OFF
WITH_PROFILER:BOOL=OFF
WITH_PYTHON3:STRING=3.8
WITH_QATZIP:BOOL=OFF
WITH_RADOSGW:BOOL=ON
WITH_RADOSGW_AMQP_ENDPOINT:BOOL=ON
WITH_RADOSGW_BEAST_FRONTEND:BOOL=ON
WITH_RADOSGW_BEAST_OPENSSL:BOOL=ON
WITH_RADOSGW_FCGI_FRONTEND:BOOL=OFF
WITH_RADOSGW_KAFKA_ENDPOINT:BOOL=ON
WITH_RADOSGW_LUA_PACKAGES:BOOL=ON
WITH_RBD:BOOL=ON
WITH_RBD_MIGRATION_FORMAT_QCOW_V1:BOOL=ON
WITH_RBD_RWL:BOOL=OFF
WITH_RBD_SSD_CACHE:BOOL=ON
WITH_RDMA:BOOL=ON
WITH_REENTRANT_STRSIGNAL:BOOL=OFF
WITH_SEASTAR:BOOL=OFF
WITH_SELINUX:BOOL=OFF
WITH_SPDK:BOOL=OFF
WITH_STATIC_LIBSTDCXX:BOOL=OFF
WITH_SYSTEMD:BOOL=ON
WITH_SYSTEM_BOOST:BOOL=ON
WITH_SYSTEM_GTEST:BOOL=ON
WITH_SYSTEM_NPM:BOOL=OFF
WITH_SYSTEM_ROCKSDB:BOOL=ON
WITH_TESTS:BOOL=ON
WITH_THREAD_SAFE_RES_QUERY:BOOL=OFF
WITH_TSAN:BOOL=OFF
WITH_UBSAN:BOOL=OFF
WITH_XFS:BOOL=ON
WITH_ZFS:BOOL=OFF
dmclock_TEST:BOOL=OFF
rabbitmq_INCLUDE_DIR:PATH=/usr/include
rabbitmq_LIBRARY:FILEPATH=/usr/lib64/librabbitmq.so
rdkafka_INCLUDE_DIR:PATH=/usr/include
rdkafka_LIBRARY:FILEPATH=/usr/lib64/librdkafka.so

Please let me know if you need more details.

kloczek

--
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
--
Regards
Kefu Chai
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx

[Index of Archives]     [CEPH Users]     [Ceph Devel]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux