On Wed, Jun 15, 2016 at 8:10 PM, Brad Hubbard <bhubbard@xxxxxxxxxx> wrote: > Sorry to those that recieve this twice but the initial message bounced > from the list because some HTML slipped in (thanks Google!). > > On Tue, Jun 7, 2016 at 1:32 PM, Mark Nelson <mnelson@xxxxxxxxxx> wrote: >> >> After running a bisect I'm semi-confident that this was a result of the rocksdb pull up to latest master in #9466: >> >> https://github.com/ceph/ceph/commit/f485596dd0656b01ba130b4e5040fcd4c02d1a31 > > > I'm hitting something very similar while running unittest_rocksdb_option_static. Embarassingly and frustratingly this turned out to be an affect of some artifact in the rocksdb subdirectory. After running a make clean in that directory the segfault disappeared. Sorry for the noise, Brad > > (gdb) bt > #0 __atomic_add_dispatch (__val=1, __mem=0x7ffff5f774d5 <operator > new(unsigned long)+37>) at /usr/include/c++/4.8.3/ext/atomicity.h:96 > #1 _M_add_ref_copy (this=0x7ffff5f774cd <operator new(unsigned > long)+29>) at /usr/include/c++/4.8.3/bits/shared_ptr_base.h:131 > #2 __shared_count (__r=..., this=0x7fffffffd180) at > /usr/include/c++/4.8.3/bits/shared_ptr_base.h:553 > #3 __shared_ptr (this=0x7fffffffd178) at > /usr/include/c++/4.8.3/bits/shared_ptr_base.h:779 > #4 shared_ptr (this=0x7fffffffd178) at > /usr/include/c++/4.8.3/bits/shared_ptr.h:103 > #5 rocksdb::ColumnFamilyOptions::ColumnFamilyOptions > (this=0x7fffffffd170, options=...) at util/options.cc:201 > #6 0x0000555555804c6a in rocksdb::GetOptionsFromString > (base_options=..., opts_str="compression=kNoCompression", > new_options=new_options@entry=0x7fffffffdad0) at > util/options_helper.cc:1327 > #7 0x0000555555734b19 in RocksDBStore::ParseOptionsFromString > (this=0x55555deefa40, opt_str=..., opt=...) at kv/RocksDBStore.cc:188 > #8 0x000055555572b361 in RocksDBOption_simple_Test::TestBody > (this=<optimized out>) at > test/objectstore/TestRocksdbOptionParse.cc:31 > #9 0x0000555555874b14 in > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void> (object=0x55555deee060, method=&virtual > testing::Test::TestBody(), location=0x55555596c7db "the test body") at > ./src/gtest.cc:2078 > #10 0x000055555586fc16 in > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void> (object=0x55555deee060, method=&virtual > testing::Test::TestBody(), location=0x55555596c7db "the test body") at > ./src/gtest.cc:2114 > #11 0x000055555585688f in testing::Test::Run (this=0x55555deee060) at > ./src/gtest.cc:2151 > #12 0x0000555555857088 in testing::TestInfo::Run (this=0x55555de581e0) > at ./src/gtest.cc:2326 > #13 0x000055555585774c in testing::TestCase::Run (this=0x55555de58600) > at ./src/gtest.cc:2444 > #14 0x000055555585e214 in testing::internal::UnitTestImpl::RunAllTests > (this=0x55555de582a0) at ./src/gtest.cc:4315 > #15 0x0000555555875efa in > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool> (object=0x55555de582a0, > method=(bool > (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * > const)) 0x55555585df7c > <testing::internal::UnitTestImpl::RunAllTests()>, > location=0x55555596d030 "auxiliary test code (environments or > event listeners)") at ./src/gtest.cc:2078 > #16 0x0000555555870a70 in > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool> (object=0x55555de582a0, > method=(bool > (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * > const)) 0x55555585df7c > <testing::internal::UnitTestImpl::RunAllTests()>, > location=0x55555596d030 "auxiliary test code (environments or > event listeners)") at ./src/gtest.cc:2114 > #17 0x000055555585cde4 in testing::UnitTest::Run (this=0x555555c569a0 > <testing::UnitTest::GetInstance()::instance>) at ./src/gtest.cc:3929 > #18 0x00005555556e86c6 in RUN_ALL_TESTS () at > ../src/gmock/gtest/include/gtest/gtest.h:2288 > #19 main (argc=1, argv=0x7fffffffe308) at > test/objectstore/TestRocksdbOptionParse.cc:84 > (gdb) f 5 > #5 rocksdb::ColumnFamilyOptions::ColumnFamilyOptions > (this=0x7fffffffd170, options=...) at util/options.cc:201 > 201 paranoid_file_checks(options.paranoid_file_checks), > > That line is part of the initialiser list for the ColumnFamilyOptions > struct which has changed rescently according to git blame. > > (gdb) up > #6 0x0000555555804c6a in rocksdb::GetOptionsFromString > (base_options=..., opts_str="compression=kNoCompression", > new_options=new_options@entry=0x7fffffffdad0) at > util/options_helper.cc:1327 > 1327 } > (gdb) l > 1322 Status s = StringToMap(opts_str, &opts_map); > 1323 if (!s.ok()) { > 1324 return s; > 1325 } > 1326 return GetColumnFamilyOptionsFromMap(base_options, opts_map, > new_options); > 1327 } > > I believe this is related to base_options and new_options which are > the same object of type rocksdb::Options. > > Valgrind has this to say. > > ==26974== Conditional jump or move depends on uninitialised value(s) > ==26974== at 0x3B09A6: > rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(rocksdb::Options > const&) (shared_ptr_base.h:552) > ==26974== by 0x3B8C69: > rocksdb::GetOptionsFromString(rocksdb::Options const&, std::string > const&, rocksdb::Options*) (options_helper.cc:1327) > ==26974== by 0x2E8B18: > RocksDBStore::ParseOptionsFromString(std::string, rocksdb::Options&) > (RocksDBStore.cc:188) > ==26974== by 0x2DF360: RocksDBOption_simple_Test::TestBody() > (TestRocksdbOptionParse.cc:31) > ==26974== by 0x428B13: void > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2078) > ==26974== by 0x423C15: void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2114) > ==26974== by 0x40A88E: testing::Test::Run() (gtest.cc:2151) > ==26974== by 0x40B087: testing::TestInfo::Run() (gtest.cc:2326) > ==26974== by 0x40B74B: testing::TestCase::Run() (gtest.cc:2444) > ==26974== by 0x412213: > testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4315) > ==26974== by 0x429EF9: bool > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2078) > ==26974== by 0x424A6F: bool > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2114) > ==26974== by 0x410DE3: testing::UnitTest::Run() (gtest.cc:3929) > ==26974== by 0x29C6C5: main (gtest.h:2288) > ==26974== Uninitialised value was created by a stack allocation > ==26974== at 0x2DF1AC: RocksDBOption_simple_Test::TestBody() (options.h:1350) > > That's this line. > > 1347 // Options to control the behavior of a database (passed to DB::Open) > 1348 struct Options : public DBOptions, public ColumnFamilyOptions { > 1349 // Create an Options object with default values for all fields. > 1350 Options() : DBOptions(), ColumnFamilyOptions() {} > <----------- HERE > > It then gives two more errors before the crash. > > ==26974== Invalid read of size 8 > ==26974== at 0x94162B2: memcpy@GLIBC_2.2.5 (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26974== by 0x3B0A99: > rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(rocksdb::Options > const&) (stl_algobase.h:372) > ==26974== by 0x3B8C69: > rocksdb::GetOptionsFromString(rocksdb::Options const&, std::string > const&, rocksdb::Options*) (options_helper.cc:1327) > ==26974== by 0x2E8B18: > RocksDBStore::ParseOptionsFromString(std::string, rocksdb::Options&) > (RocksDBStore.cc:188) > ==26974== by 0x2DF360: RocksDBOption_simple_Test::TestBody() > (TestRocksdbOptionParse.cc:31) > ==26974== by 0x428B13: void > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2078) > ==26974== by 0x423C15: void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2114) > ==26974== by 0x40A88E: testing::Test::Run() (gtest.cc:2151) > ==26974== by 0x40B087: testing::TestInfo::Run() (gtest.cc:2326) > ==26974== by 0x40B74B: testing::TestCase::Run() (gtest.cc:2444) > ==26974== by 0x412213: > testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4315) > ==26974== by 0x429EF9: bool > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2078) > ==26974== by 0x424A6F: bool > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2114) > ==26974== by 0x410DE3: testing::UnitTest::Run() (gtest.cc:3929) > ==26974== by 0x29C6C5: main (gtest.h:2288) > ==26974== Address 0x39fe1038 is 8 bytes before a block of size > 4,290,772,994 alloc'd > ==26974== at 0x9412965: operator new(unsigned long) (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26974== by 0x3B0FD7: > rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(rocksdb::Options > const&) (new_allocator.h:104) > ==26974== by 0x3B8C69: > rocksdb::GetOptionsFromString(rocksdb::Options const&, std::string > const&, rocksdb::Options*) (options_helper.cc:1327) > ==26974== by 0x2E8B18: > RocksDBStore::ParseOptionsFromString(std::string, rocksdb::Options&) > (RocksDBStore.cc:188) > ==26974== by 0x2DF360: RocksDBOption_simple_Test::TestBody() > (TestRocksdbOptionParse.cc:31) > ==26974== by 0x428B13: void > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2078) > ==26974== by 0x423C15: void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2114) > ==26974== by 0x40A88E: testing::Test::Run() (gtest.cc:2151) > ==26974== by 0x40B087: testing::TestInfo::Run() (gtest.cc:2326) > ==26974== by 0x40B74B: testing::TestCase::Run() (gtest.cc:2444) > ==26974== by 0x412213: > testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4315) > ==26974== by 0x429EF9: bool > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2078) > ==26974== by 0x424A6F: bool > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2114) > ==26974== by 0x410DE3: testing::UnitTest::Run() (gtest.cc:3929) > ==26974== by 0x29C6C5: main (gtest.h:2288) > > ==26974== Invalid read of size 8 > ==26974== at 0x94162A0: memcpy@GLIBC_2.2.5 (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26974== by 0x3B0A99: > rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(rocksdb::Options > const&) (stl_algobase.h:372) > ==26974== by 0x3B8C69: > rocksdb::GetOptionsFromString(rocksdb::Options const&, std::string > const&, rocksdb::Options*) (options_helper.cc:1327) > ==26974== by 0x2E8B18: > RocksDBStore::ParseOptionsFromString(std::string, rocksdb::Options&) > (RocksDBStore.cc:188) > ==26974== by 0x2DF360: RocksDBOption_simple_Test::TestBody() > (TestRocksdbOptionParse.cc:31) > ==26974== by 0x428B13: void > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2078) > ==26974== by 0x423C15: void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2114) > ==26974== by 0x40A88E: testing::Test::Run() (gtest.cc:2151) > ==26974== by 0x40B087: testing::TestInfo::Run() (gtest.cc:2326) > ==26974== by 0x40B74B: testing::TestCase::Run() (gtest.cc:2444) > ==26974== by 0x412213: > testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4315) > ==26974== by 0x429EF9: bool > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2078) > ==26974== by 0x424A6F: bool > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2114) > ==26974== by 0x410DE3: testing::UnitTest::Run() (gtest.cc:3929) > ==26974== by 0x29C6C5: main (gtest.h:2288) > ==26974== Address 0x39fe1030 is 16 bytes before a block of size > 4,290,772,994 alloc'd > ==26974== at 0x9412965: operator new(unsigned long) (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==26974== by 0x3B0FD7: > rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(rocksdb::Options > const&) (new_allocator.h:104) > ==26974== by 0x3B8C69: > rocksdb::GetOptionsFromString(rocksdb::Options const&, std::string > const&, rocksdb::Options*) (options_helper.cc:1327) > ==26974== by 0x2E8B18: > RocksDBStore::ParseOptionsFromString(std::string, rocksdb::Options&) > (RocksDBStore.cc:188) > ==26974== by 0x2DF360: RocksDBOption_simple_Test::TestBody() > (TestRocksdbOptionParse.cc:31) > ==26974== by 0x428B13: void > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2078) > ==26974== by 0x423C15: void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (gtest.cc:2114) > ==26974== by 0x40A88E: testing::Test::Run() (gtest.cc:2151) > ==26974== by 0x40B087: testing::TestInfo::Run() (gtest.cc:2326) > ==26974== by 0x40B74B: testing::TestCase::Run() (gtest.cc:2444) > ==26974== by 0x412213: > testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4315) > ==26974== by 0x429EF9: bool > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2078) > ==26974== by 0x424A6F: bool > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2114) > ==26974== by 0x410DE3: testing::UnitTest::Run() (gtest.cc:3929) > ==26974== by 0x29C6C5: main (gtest.h:2288) > > Note, for what it's worth, that we've done two reads of size 8 at 8 > and 16 bytes before a valid block respectively. > > The last line that is not in teh STL is the line we eventually crash on. > > 1317 Status GetColumnFamilyOptionsFromString( > 1318 const ColumnFamilyOptions& base_options, > 1319 const std::string& opts_str, > 1320 ColumnFamilyOptions* new_options) { > 1321 std::unordered_map<std::string, std::string> opts_map; > 1322 Status s = StringToMap(opts_str, &opts_map); > 1323 if (!s.ok()) { > 1324 return s; > 1325 } > 1326 return GetColumnFamilyOptionsFromMap(base_options, opts_map, > new_options); > 1327 } > > I think this has something to do with the changes in the > ColumnFamilyOptions struct and the fact that there is a conversion > happening from an Options object to a ColumnFamilyOptions object. > > That's the feeling I get from poking around in this for a while but I > can't nail exactly what's happening here and, of course, I could very > well be wrong. > > HTH, > Brad > >> >> >> >> Next need to track down the commit in rocksdb. >> >> Mark >> >> >> On 06/06/2016 06:53 PM, Mark Nelson wrote: >>> >>> Looks like memory is being corrupted. One example: >>> >>> (gdb) info locals >>> >>> <snip> >>> max_bytes_for_level_multiplier_additional = std::vector of length >>> 35181114486204, capacity -33 = {<error reading variable> >>> >>> Running through valgrind now to see if I can track it down. >>> >>> Mark >>> >>> On 06/06/2016 11:38 AM, Somnath Roy wrote: >>>> >>>> Thanks Mark for confirming this ! >>>> May be we pulled latest rocksdb that are causing some incompatibility >>>> on option parsing ? >>>> >>>> Regards >>>> Somnath >>>> >>>> -----Original Message----- >>>> From: Mark Nelson [mailto:mnelson@xxxxxxxxxx] >>>> Sent: Monday, June 06, 2016 9:35 AM >>>> To: Somnath Roy; Sage Weil >>>> Cc: Jianjian Huo; ceph-devel >>>> Subject: Re: Bluestore crash in mkfs >>>> >>>> I'm hitting this after upgrading to master. Probably something that >>>> got merged on the 2nd or 3rd. I'll see if I can track it down. >>>> >>>> Mark >>>> >>>> On 06/04/2016 09:41 PM, Somnath Roy wrote: >>>>> >>>>> Here is the steps I did.. >>>>> 1. Git pull >>>>> 2. Make clean >>>>> 3. Do_autogen -j o3 >>>>> 4. Make install >>>>> 5. Mkcephfs to install, I still prefer it >>>>> >>>>> I will debug more and update >>>>> >>>>> Thanks & Regards >>>>> Somnath >>>>> >>>>> Sent from my iPhone >>>>> >>>>>> On Jun 4, 2016, at 7:35 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: >>>>>> >>>>>>> On Sat, 4 Jun 2016, Somnath Roy wrote: >>>>>>> Same conf file worked for me sometimes back. I think db path/wal >>>>>>> path is optional and if you don't mention anything, it will coexist >>>>>>> on the block partition. >>>>>>> Not sure if this is something changed recently though.. >>>>>>> The trace is also not complaining about db path , it is getting a >>>>>>> seg fault while parsing rocksdb option. >>>>>> >>>>>> >>>>>> Hmm, I haven't seen this. >>>>>> >>>>>> Note that you don't need 'devs'. And you're right that just >>>>>> specifying the block path is enough. If you're doing a manual build, >>>>>> is it possible you're dynamically linking rocksdb or something? >>>>>> >>>>>> sage >>>>>> >>>>>>> >>>>>>> Thanks & Regards >>>>>>> Somnath >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Jianjian Huo [mailto:samuel.huo@xxxxxxxxx] >>>>>>> Sent: Friday, June 03, 2016 9:29 PM >>>>>>> To: Somnath Roy >>>>>>> Cc: Sage Weil (sage@xxxxxxxxxxxx); ceph-devel >>>>>>> Subject: Re: Bluestore crash in mkfs >>>>>>> >>>>>>>> On Fri, Jun 3, 2016 at 7:16 PM, Somnath Roy >>>>>>>> <Somnath.Roy@xxxxxxxxxxx> wrote: >>>>>>>> Sage, >>>>>>>> Latest code in Bluestore is crashing on me during mkfs. Here is >>>>>>>> the dump. >>>>>>>> >>>>>>>> ceph version 10.2.0-1809-gac8c40f >>>>>>>> (ac8c40f3e3257517f71d2af8bd26dcd264887b7f) >>>>>>>> 1: (()+0x99e65e) [0x55cf5263565e] >>>>>>>> 2: (()+0x113d0) [0x7f323526f3d0] >>>>>>>> 3: >>>>>>>> (rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(rocksdb::Options >>>>>>>> const&)+0x59) [0x55cf525d4ce9] >>>>>>>> 4: (rocksdb::GetOptionsFromString(rocksdb::Options const&, >>>>>>>> std::__cxx11::basic_string<char, std::char_traits<char>, >>>>>>>> std::allocator<char> > const&, rocksdb::Options*)+0x15a) >>>>>>>> [0x55cf525dc74a] >>>>>>>> 5: >>>>>>>> (RocksDBStore::ParseOptionsFromString(std::__cxx11::basic_string<ch >>>>>>>> ar, std::char_traits<char>, std::allocator<char> >, >>>>>>>> rocksdb::Options&)+0x22e) [0x55cf524f58be] >>>>>>>> 6: (RocksDBStore::init(std::__cxx11::basic_string<char, >>>>>>>> std::char_traits<char>, std::allocator<char> >)+0x77) >>>>>>>> [0x55cf524f5f17] >>>>>>>> 7: (BlueStore::_open_db(bool)+0x274) [0x55cf5234b9e4] >>>>>>>> 8: (BlueStore::mkfs()+0x899) [0x55cf52361ad9] >>>>>>>> 9: (OSD::mkfs(CephContext*, ObjectStore*, >>>>>>>> std::__cxx11::basic_string<char, std::char_traits<char>, >>>>>>>> std::allocator<char> > const&, uuid_d, int)+0x117) [0x55cf52001427] >>>>>>>> 10: (main()+0x1003) [0x55cf51f9f2e3] >>>>>>>> 11: (__libc_start_main()+0xf0) [0x7f32331fa830] >>>>>>>> 12: (_start()+0x29) [0x55cf51fe2719] >>>>>>>> NOTE: a copy of the executable, or `objdump -rdS <executable>` is >>>>>>>> needed to interpret this. >>>>>>>> >>>>>>>> >>>>>>>> I am using the following option : >>>>>>>> >>>>>>>> devs = /dev/sdb1 >>>>>>>> bluestore_block_path = /dev/sdb2 >>>>>>>> >>>>>>>> Am I missing anything ? >>>>>>> >>>>>>> >>>>>>> This is what Mark posted last time, you forgot db path? >>>>>>> https://github.com/ceph/cbt/blob/master/example/bluestore/ceph.conf. >>>>>>> bluestore_example >>>>>>>> >>>>>>>> >>>>>>>> Thanks & Regards >>>>>>>> Somnath >>>>>>>> PLEASE NOTE: The information contained in this electronic mail >>>>>>>> message is intended only for the use of the designated >>>>>>>> recipient(s) named above. If the reader of this message is not the >>>>>>>> intended recipient, you are hereby notified that you have received >>>>>>>> this message in error and that any review, dissemination, >>>>>>>> distribution, or copying of this message is strictly prohibited. >>>>>>>> If you have received this communication in error, please notify >>>>>>>> the sender by telephone or e-mail (as shown above) immediately and >>>>>>>> destroy any and all copies of this message in your possession >>>>>>>> (whether hard copies or electronically stored copies). >>>>>>>> -- >>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>>>>>>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More >>>>>>>> majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>> >>>>>>> N?????r??y??????X??ǧv???){.n?????z?]z????ay? ʇڙ??j ??f???h????? >>>>>>> ?w??? ???j:+v???w???????? ????zZ+???????j"????i >>>>> >>>>> N r y b X ǧv ^ ){.n + z ]z {ay ʇڙ ,j f h z w >>>> >>>> j:+v w j m zZ+ ݢj" !tml= >>>>> >>>>> >>>> N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay� ʇڙ�,j ��f���h���z� �w��� ���j:+v���w�j�m���� ����zZ+�����ݢj"��!tml= >>>> >>>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > -- > Cheers, > Brad -- Cheers, Brad -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
