On Fri, Dec 23, 2016 at 11:16 AM, Allen Samuels <Allen.Samuels@xxxxxxxxxxx> wrote: > Does the unit test run or does it actually fault out or does it just report an error? It runs, completes and passes. > > > Allen Samuels > SanDisk |a Western Digital brand > 2880 Junction Avenue, San Jose, CA 95134 > T: +1 408 801 7030| M: +1 408 780 6416 > allen.samuels@xxxxxxxxxxx > > >> -----Original Message----- >> From: Brad Hubbard [mailto:bhubbard@xxxxxxxxxx] >> Sent: Thursday, December 22, 2016 5:15 PM >> To: Willem Jan Withagen <wjw@xxxxxxxxxxx> >> Cc: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; Ceph Development >> <ceph-devel@xxxxxxxxxxxxxxx> >> Subject: Re: An empty vptr in an raw object >> >> The valgrind output I posted seems to indicate an issue in the stl, which is >> unlikely and probably a false positive. Strange coincidence though... >> >> Another option for attacking this might be sanitizers? >> >> >> On Fri, Dec 23, 2016 at 11:01 AM, Willem Jan Withagen <wjw@xxxxxxxxxxx> >> wrote: >> > On 23-12-2016 01:54, Brad Hubbard wrote: >> >> Here's what I ran. >> >> >> >> $ valgrind --trace-children=yes --show-reachable=yes >> >> --track-origins=yes --read-var-info=yes --tool=memcheck >> >> --leak-check=full --num-callers=50 -v --log-file=unittest_denc.log >> >> bin/unittest_denc >> > >> > Errgh, >> > >> > Installing valgrind package also wants to install gcc. >> > Something I desperately want to avoid, because I otherwise import even >> > more diversity. >> > So I'll have to see if I can build it myself with clang. >> > >> > --WjW >> > >> >> Your stack shows /usr/srcs/Ceph/work/ceph/src/test/test_denc.cc:206 >> >> which seems to match up with this. >> >> >> >> ==10312== Mismatched free() / delete / delete [] >> >> ==10312== at 0x4C2ED4A: free (vg_replace_malloc.c:530) >> >> ==10312== by 0x162A6A: deallocate (new_allocator.h:110) >> >> ==10312== by 0x162A6A: deallocate (alloc_traits.h:442) >> >> ==10312== by 0x162A6A: _M_deallocate (stl_vector.h:178) >> >> ==10312== by 0x162A6A: void >> >> std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, >> >> std::allocator<char> >, >> >> std::allocator<std::__cxx11::basic_string<char, >> >> std::char_traits<char>, std::allocator<char> > > >:: ⤷ >> >> _M_emplace_back_aux<std::__cxx11::basic_string<char, >> >> std::char_traits<char>, std::allocator<char> > >> >>> (std::__cxx11::basic_string<char, std::char_traits<char>, >> >> std::allocator<char> >&&) (vector.tcc:438) >> >> ==10312== by 0x15E0B5: push_back (stl_vector.h:933) >> >> ==10312== by 0x15E0B5: denc_vector_Test::TestBody() >> >> (test_denc.cc:207) >> >> ==10312== by 0x19D203: >> >> HandleSehExceptionsInMethodIfSupported<testing::Test, void> >> >> (gtest.cc:2402) >> >> ==10312== by 0x19D203: void >> >> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, >> >> void>(testing::Test*, void (testing::Test::*)(), char const*) >> >> (gtest.cc:2438) >> >> ==10312== by 0x194029: testing::Test::Run() (gtest.cc:2475) >> >> ==10312== by 0x194177: testing::TestInfo::Run() (gtest.cc:2656) >> >> ==10312== by 0x194254: testing::TestCase::Run() (gtest.cc:2774) >> >> ==10312== by 0x194536: >> >> testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4649) >> >> ==10312== by 0x19D6B3: >> >> >> HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImp >> >> l, >> >> bool> (gtest.cc:2402) >> >> ==10312== by 0x19D6B3: bool >> >> testing::internal::HandleExceptionsInMethodIfSupported<testing::inter >> >> nal::UnitTestImpl, >> >> bool>(testing::internal::UnitTestImpl*, bool >> >> (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2438) >> >> ==10312== by 0x194853: testing::UnitTest::Run() (gtest.cc:4257) >> >> ==10312== by 0x15A7D8: RUN_ALL_TESTS (gtest.h:2233) >> >> ==10312== by 0x15A7D8: main (gmock_main.cc:53) >> >> ==10312== Address 0x8acba20 is 0 bytes inside a block of size 32 alloc'd >> >> ==10312== at 0x4C2E8E9: operator new[](unsigned long) >> >> (vg_replace_malloc.c:423) >> >> ==10312== by 0x16294D: allocate (new_allocator.h:104) >> >> ==10312== by 0x16294D: allocate (alloc_traits.h:416) >> >> ==10312== by 0x16294D: _M_allocate (stl_vector.h:170) >> >> ==10312== by 0x16294D: void >> >> std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, >> >> std::allocator<char> >, >> >> std::allocator<std::__cxx11::basic_string<char, >> >> std::char_traits<char>, std::allocator<char> > > >:: ⤷ >> >> _M_emplace_back_aux<std::__cxx11::basic_string<char, >> >> std::char_traits<char>, std::allocator<char> > >> >>> (std::__cxx11::basic_string<char, std::char_traits<char>, >> >> std::allocator<char> >&&) (vector.tcc:412) >> >> ==10312== by 0x15E07D: push_back (stl_vector.h:933) >> >> ==10312== by 0x15E07D: denc_vector_Test::TestBody() >> (test_denc.cc:206) >> >> ==10312== by 0x19D203: >> >> HandleSehExceptionsInMethodIfSupported<testing::Test, void> >> >> (gtest.cc:2402) >> >> ==10312== by 0x19D203: void >> >> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, >> >> void>(testing::Test*, void (testing::Test::*)(), char const*) >> >> (gtest.cc:2438) >> >> ==10312== by 0x194029: testing::Test::Run() (gtest.cc:2475) >> >> ==10312== by 0x194177: testing::TestInfo::Run() (gtest.cc:2656) >> >> ==10312== by 0x194254: testing::TestCase::Run() (gtest.cc:2774) >> >> ==10312== by 0x194536: >> >> testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4649) >> >> ==10312== by 0x19D6B3: >> >> >> HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImp >> >> l, >> >> bool> (gtest.cc:2402) >> >> ==10312== by 0x19D6B3: bool >> >> testing::internal::HandleExceptionsInMethodIfSupported<testing::inter >> >> nal::UnitTestImpl, >> >> bool>(testing::internal::UnitTestImpl*, bool >> >> (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2438) >> >> ==10312== by 0x194853: testing::UnitTest::Run() (gtest.cc:4257) >> >> ==10312== by 0x15A7D8: RUN_ALL_TESTS (gtest.h:2233) >> >> ==10312== by 0x15A7D8: main (gmock_main.cc:53) >> >> >> >> HTH. >> >> >> >> >> >> On Fri, Dec 23, 2016 at 10:51 AM, Willem Jan Withagen >> <wjw@xxxxxxxxxxx> wrote: >> >>> On 23-12-2016 01:32, Brad Hubbard wrote: >> >>>> Checked this myself and valgrind shows numerous problems of this >> type. >> >>> >> >>> I know of valgrind, and what it is suppossed to do, but have not >> >>> really used it in a real problem. So that is going to be a first. >> >>> Any chance of a simple commandline to run this on the minimized test >> >>> I now have.... >> >>> >> >>> --WjW >> >>> >> >>> >> >>>> ==10312== Mismatched free() / delete / delete [] >> >>>> ==10312== at 0x4C2ED4A: free (vg_replace_malloc.c:530) >> >>>> ==10312== by 0x160117: deallocate (new_allocator.h:110) >> >>>> ==10312== by 0x160117: deallocate (alloc_traits.h:442) >> >>>> ==10312== by 0x160117: _M_put_node (stl_list.h:387) >> >>>> ==10312== by 0x160117: _M_clear (list.tcc:80) >> >>>> ==10312== by 0x160117: ~_List_base (stl_list.h:442) >> >>>> ==10312== by 0x160117: ~list (stl_list.h:503) >> >>>> ==10312== by 0x160117: denc_list_Test::TestBody() >> (test_denc.cc:282) >> >>>> ==10312== by 0x19D203: >> >>>> HandleSehExceptionsInMethodIfSupported<testing::Test, void> >> >>>> (gtest.cc:2402) >> >>>> ==10312== by 0x19D203: void >> >>>> testing::internal::HandleExceptionsInMethodIfSupported<testing::Tes >> >>>> t, >> >>>> void>(testing::Test*, void (testing::Test::*)(), char const*) >> >>>> (gtest.cc:2438) >> >>>> ==10312== by 0x194029: testing::Test::Run() (gtest.cc:2475) >> >>>> ==10312== by 0x194177: testing::TestInfo::Run() (gtest.cc:2656) >> >>>> ==10312== by 0x194254: testing::TestCase::Run() (gtest.cc:2774) >> >>>> ==10312== by 0x194536: >> >>>> testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4649) >> >>>> ==10312== by 0x19D6B3: >> >>>> >> HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestI >> >>>> mpl, >> >>>> bool> (gtest.cc:2402) >> >>>> ==10312== by 0x19D6B3: bool >> >>>> testing::internal::HandleExceptionsInMethodIfSupported<testing::int >> >>>> ernal::UnitTestImpl, >> >>>> bool>(testing::internal::UnitTestImpl*, bool >> >>>> (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2438) >> >>>> ==10312== by 0x194853: testing::UnitTest::Run() (gtest.cc:4257) >> >>>> ==10312== by 0x15A7D8: RUN_ALL_TESTS (gtest.h:2233) >> >>>> ==10312== by 0x15A7D8: main (gmock_main.cc:53) >> >>>> ==10312== Address 0x8ad31a0 is 0 bytes inside a block of size 24 alloc'd >> >>>> ==10312== at 0x4C2E8E9: operator new[](unsigned long) >> >>>> (vg_replace_malloc.c:423) >> >>>> ==10312== by 0x15FDF1: allocate (new_allocator.h:104) >> >>>> ==10312== by 0x15FDF1: allocate (alloc_traits.h:416) >> >>>> ==10312== by 0x15FDF1: _M_get_node (stl_list.h:383) >> >>>> ==10312== by 0x15FDF1: >> _M_create_node<denc_counter_bounded_t> >> >>>> (stl_list.h:568) >> >>>> ==10312== by 0x15FDF1: _M_insert<denc_counter_bounded_t> >> (stl_list.h:1770) >> >>>> ==10312== by 0x15FDF1: emplace_back<denc_counter_bounded_t> >> (stl_list.h:1108) >> >>>> ==10312== by 0x15FDF1: decode (denc.h:716) >> >>>> ==10312== by 0x15FDF1: >> >>>> decode<std::__cxx11::list<denc_counter_bounded_t> > (denc.h:1289) >> >>>> ==10312== by 0x15FDF1: >> >>>> decode<std::__cxx11::list<denc_counter_bounded_t> > >> (encoding.h:289) >> >>>> ==10312== by 0x15FDF1: denc_list_Test::TestBody() >> (test_denc.cc:289) >> >>>> ==10312== by 0x19D203: >> >>>> HandleSehExceptionsInMethodIfSupported<testing::Test, void> >> >>>> (gtest.cc:2402) >> >>>> ==10312== by 0x19D203: void >> >>>> testing::internal::HandleExceptionsInMethodIfSupported<testing::Tes >> >>>> t, >> >>>> void>(testing::Test*, void (testing::Test::*)(), char const*) >> >>>> (gtest.cc:2438) >> >>>> ==10312== by 0x194029: testing::Test::Run() (gtest.cc:2475) >> >>>> ==10312== by 0x194177: testing::TestInfo::Run() (gtest.cc:2656) >> >>>> ==10312== by 0x194254: testing::TestCase::Run() (gtest.cc:2774) >> >>>> ==10312== by 0x194536: >> >>>> testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4649) >> >>>> ==10312== by 0x19D6B3: >> >>>> >> HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestI >> >>>> mpl, >> >>>> bool> (gtest.cc:2402) >> >>>> ==10312== by 0x19D6B3: bool >> >>>> testing::internal::HandleExceptionsInMethodIfSupported<testing::int >> >>>> ernal::UnitTestImpl, >> >>>> bool>(testing::internal::UnitTestImpl*, bool >> >>>> (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2438) >> >>>> ==10312== by 0x194853: testing::UnitTest::Run() (gtest.cc:4257) >> >>>> ==10312== by 0x15A7D8: RUN_ALL_TESTS (gtest.h:2233) >> >>>> ==10312== by 0x15A7D8: main (gmock_main.cc:53) >> >>>> >> >>>> On Fri, Dec 23, 2016 at 10:19 AM, Brad Hubbard >> <bhubbard@xxxxxxxxxx> wrote: >> >>>>> Any clue from Valgrind? >> >>>>> >> >>>>> Did you say this only happens with clang or doesn't happen with >> clang? >> >>>>> >> >>>>> On Fri, Dec 23, 2016 at 8:01 AM, Allen Samuels >> >>>>> <Allen.Samuels@xxxxxxxxxxx> wrote: >> >>>>>> I believe I mis-read the data. What I've seen before doesn't fit this >> data. >> >>>>>> >> >>>>>> If it fails in unit test, it shouldn't be hard to just set a HW breakpoint >> on the vptr and see who the culprit is. >> >>>>>> >> >>>>>> >> >>>>>> Allen Samuels >> >>>>>> SanDisk |a Western Digital brand >> >>>>>> 2880 Junction Avenue, San Jose, CA 95134 >> >>>>>> T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx >> >>>>>> >> >>>>>> >> >>>>>>> -----Original Message----- >> >>>>>>> From: Willem Jan Withagen [mailto:wjw@xxxxxxxxxxx] >> >>>>>>> Sent: Thursday, December 22, 2016 10:37 AM >> >>>>>>> To: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; Ceph >> Development >> >>>>>>> <ceph-devel@xxxxxxxxxxxxxxx> >> >>>>>>> Subject: Re: An empty vptr in an raw object >> >>>>>>> >> >>>>>>> On 22-12-2016 19:02, Allen Samuels wrote: >> >>>>>>>> I have seen cases of null vptr due to an incompletely constructed >> object, >> >>>>>>> i.e., an object that's in the middle of being constructed. >> >>>>>>> >> >>>>>>> I going to believe you right away. >> >>>>>>> But I'm having a hard time imagining such a case. >> >>>>>>> >> >>>>>>> Are you suggesting a object is referenced, whilest it is not yet >> complete. who >> >>>>>>> does the referencing then? due to threading? >> >>>>>>> That would be even harder to find. >> >>>>>>> >> >>>>>>> --WjW >> >>>>>>> >> >>>>>>> >> >>>>>>>> Allen Samuels >> >>>>>>>> SanDisk |a Western Digital brand >> >>>>>>>> 951 SanDisk Drive, Milpitas, CA 95035 >> >>>>>>>> T: +1 408 801 7030| M: +1 408 780 6416 >> allen.samuels@xxxxxxxxxxx >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>> -----Original Message----- >> >>>>>>>>> From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel- >> >>>>>>>>> owner@xxxxxxxxxxxxxxx] On Behalf Of Willem Jan Withagen >> >>>>>>>>> Sent: Thursday, December 22, 2016 9:41 AM >> >>>>>>>>> To: Ceph Development <ceph-devel@xxxxxxxxxxxxxxx> >> >>>>>>>>> Subject: An empty vptr in an raw object >> >>>>>>>>> >> >>>>>>>>>> I have a piece of code that actually seem to crash because the >> vptr >> >>>>>>>>>> is not set: >> >>>>>>>>>> (gdb) p *_raw >> >>>>>>>>>> $2 = {_vptr$raw = 0x0, data = 0x10cc000 "\003", len = 72, nref = >> >>>>>>>>>> {val = 1}, crc_spinlock = 0, crc_map = {__tree_ = { >> >>>>>>>>>> __begin_node_ = 0x10cc078, >> >>>>>>>>>> __pair1_ = >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>> >> {<std::__1::__libcpp_compressed_pair_imp<std::__1::__tree_end_node<st >> >>>>>>>>> d::__1::__tree_node_base<void*>*>, >> >>>>>>>>>> >> >>>>>>>>> >> std::__1::allocator<std::__1::__tree_node<std::__1::__value_type<std: >> >>>>>>>>> :__1 >> >>>>>>>>> ::pair<unsigned >> >>>>>>>>>> long, unsigned long>, std::__1::pair<unsigned int, unsigned int> >> >, >> >>>>>>>>>> void*> >, 2>> = >> >>>>>>>>>> >> >>>>>>>>> >> {<std::__1::allocator<std::__1::__tree_node<std::__1::__value_type<st >> >>>>>>>>> d::_ >> >>>>>>>>> _1::pair<unsigned >> >>>>>>>>>> long, unsigned long>, std::__1::pair<unsigned int, unsigned int> >> >, >> >>>>>>>>>> void*> >> = {<No data fields>}, __first_ = { >> >>>>>>>>>> __left_ = 0x0}}, <No data fields>}, >> >>>>>>>>>> >> >>>>>>>>>> The function that crashes: >> >>>>>>>>>> char *buffer::ptr::c_str() { >> >>>>>>>>>> assert(_raw); >> >>>>>>>>>> if (buffer_track_c_str) >> >>>>>>>>>> buffer_c_str_accesses.inc(); >> >>>>>>>>>> char *p = _raw->get_data(); >> >>>>>>>>>> return p + _off; >> >>>>>>>>>> } >> >>>>>>>>>> >> >>>>>>>>>> And crash is actually on the return line. >> >>>>>>>>>> >> >>>>>>>>>> Any ideas as how the vptr can be empty? >> >>>>>>>>> >> >>>>>>>>> Now the _vptr$raw point is part of the internal code of the clang >> >>>>>>>>> class function table/constructor. Overwriting that means that >> >>>>>>>>> class-function references are problematic to say the least. (in >> this >> >>>>>>> example get_data()). >> >>>>>>>>> >> >>>>>>>>> The major reason why this occurs is because an object is being >> zeroed >> >>>>>>>>> in C-style way: memset( &obj, 0, sizeof(obj)) And thus >> overwriting >> >>>>>>>>> the _vptr. >> >>>>>>>>> >> >>>>>>>>> Note that this does not bite the FreeBSD compilation, but also >> any >> >>>>>>>>> other attempts to build Ceph with clang. >> >>>>>>>>> >> >>>>>>>>> Now the strange thing is that this does not bite Clang compilation >> >>>>>>>>> much more. But the only test that fails is unittest_denc. So I >> guess >> >>>>>>>>> that most of the code is rather well behaved. >> >>>>>>>>> >> >>>>>>>>> And I'm off on a search to find the culprit. >> >>>>>>>>> >> >>>>>>>>> --WjW >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph- >> devel" >> >>>>>>>>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More >> >>>>>>> majordomo >> >>>>>>>>> info at http://vger.kernel.org/majordomo-info.html >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> Cheers, >> >>>>> Brad >> >>>> >> >>>> >> >>>> >> >>> >> >>> -- >> >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >> >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> >> >> >> >> > >> >> >> >> -- >> Cheers, >> Brad -- Cheers, Brad -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
