Re: osd/OSD.cc: 4844: FAILED assert(_get_map_bl(epoch, bl)) (ceph 0.61.7)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I just backported that one and 2 related patches to cuttlefish head.
-Sam

On Mon, Aug 26, 2013 at 2:14 PM, Stefan Priebe <s.priebe@xxxxxxxxxxxx> wrote:
> i had the same problem and backported
> 6951d2345a5d837c3b14103bd4d8f5ee4407c937 to ceph.
>
> It fixes it for me.
>
> Greets,
> Stefan
>
> Am 26.08.2013 23:10, schrieb Samuel Just:
>
>> This is the same osd, and hasn't been working in the mean time?  Can
>> your clsuter operate without that osd?
>> -Sam
>>
>> On Mon, Aug 19, 2013 at 2:05 PM, Olivier Bonvalet <ceph.list@xxxxxxxxx>
>> wrote:
>>>
>>> Le lundi 19 août 2013 à 12:27 +0200, Olivier Bonvalet a écrit :
>>>>
>>>> Hi,
>>>>
>>>> I have an OSD which crash every time I try to start it (see logs below).
>>>> Is it a known problem ? And is there a way to fix it ?
>>>>
>>>> root! taman:/var/log/ceph# grep -v ' pipe' osd.65.log
>>>> 2013-08-19 11:07:48.478558 7f6fe367a780  0 ceph version 0.61.7
>>>> (8f010aff684e820ecc837c25ac77c7a05d7191ff), process ceph-osd, pid 19327
>>>> 2013-08-19 11:07:48.516363 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount FIEMAP ioctl is supported and
>>>> appears to work
>>>> 2013-08-19 11:07:48.516380 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount FIEMAP ioctl is disabled via
>>>> 'filestore fiemap' config option
>>>> 2013-08-19 11:07:48.516514 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount did NOT detect btrfs
>>>> 2013-08-19 11:07:48.517087 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount syscall(SYS_syncfs, fd) fully
>>>> supported
>>>> 2013-08-19 11:07:48.517389 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount found snaps <>
>>>> 2013-08-19 11:07:49.199483 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount: enabling WRITEAHEAD journal
>>>> mode: btrfs not detected
>>>> 2013-08-19 11:07:52.191336 7f6fe367a780  1 journal _open /dev/sdk4 fd
>>>> 18: 53687091200 bytes, block size 4096 bytes, directio = 1, aio = 1
>>>> 2013-08-19 11:07:52.196020 7f6fe367a780  1 journal _open /dev/sdk4 fd
>>>> 18: 53687091200 bytes, block size 4096 bytes, directio = 1, aio = 1
>>>> 2013-08-19 11:07:52.196920 7f6fe367a780  1 journal close /dev/sdk4
>>>> 2013-08-19 11:07:52.199908 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount FIEMAP ioctl is supported and
>>>> appears to work
>>>> 2013-08-19 11:07:52.199916 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount FIEMAP ioctl is disabled via
>>>> 'filestore fiemap' config option
>>>> 2013-08-19 11:07:52.200058 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount did NOT detect btrfs
>>>> 2013-08-19 11:07:52.200886 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount syscall(SYS_syncfs, fd) fully
>>>> supported
>>>> 2013-08-19 11:07:52.200919 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount found snaps <>
>>>> 2013-08-19 11:07:52.215850 7f6fe367a780  0
>>>> filestore(/var/lib/ceph/osd/ceph-65) mount: enabling WRITEAHEAD journal
>>>> mode: btrfs not detected
>>>> 2013-08-19 11:07:52.219819 7f6fe367a780  1 journal _open /dev/sdk4 fd
>>>> 26: 53687091200 bytes, block size 4096 bytes, directio = 1, aio = 1
>>>> 2013-08-19 11:07:52.227420 7f6fe367a780  1 journal _open /dev/sdk4 fd
>>>> 26: 53687091200 bytes, block size 4096 bytes, directio = 1, aio = 1
>>>> 2013-08-19 11:07:52.500342 7f6fe367a780  0 osd.65 144201 crush map has
>>>> features 262144, adjusting msgr requires for clients
>>>> 2013-08-19 11:07:52.500353 7f6fe367a780  0 osd.65 144201 crush map has
>>>> features 262144, adjusting msgr requires for osds
>>>> 2013-08-19 11:08:13.581709 7f6fbdcb5700 -1 osd/OSD.cc: In function
>>>> 'OSDMapRef OSDService::get_map(epoch_t)' thread 7f6fbdcb5700 time 2013-08-19
>>>> 11:08:13.579519
>>>> osd/OSD.cc: 4844: FAILED assert(_get_map_bl(epoch, bl))
>>>>
>>>>   ceph version 0.61.7 (8f010aff684e820ecc837c25ac77c7a05d7191ff)
>>>>   1: (OSDService::get_map(unsigned int)+0x44b) [0x6f5b9b]
>>>>   2: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&,
>>>> PG::RecoveryCtx*, std::set<boost::intrusive_ptr<PG>,
>>>> std::less<boost::intrusive_ptr<PG> >,
>>>> std::allocator<boost::intrusive_ptr<PG> > >*)+0x3c8) [0x6f8f48]
>>>>   3: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> >
>>>> const&, ThreadPool::TPHandle&)+0x31f) [0x6f975f]
>>>>   4: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> >
>>>> const&, ThreadPool::TPHandle&)+0x14) [0x7391d4]
>>>>   5: (ThreadPool::worker(ThreadPool::WorkThread*)+0x68a) [0x8f8e3a]
>>>>   6: (ThreadPool::WorkThread::entry()+0x10) [0x8fa0e0]
>>>>   7: (()+0x6b50) [0x7f6fe3070b50]
>>>>   8: (clone()+0x6d) [0x7f6fe15cba7d]
>>>>   NOTE: a copy of the executable, or `objdump -rdS <executable>` is
>>>> needed to interpret this.
>>>>
>>>> full logs here : http://pastebin.com/RphNyLU0
>>>>
>>>>
>>>
>>> Hi,
>>>
>>> still same problem with Ceph 0.61.8 :
>>>
>>> 2013-08-19 23:01:54.369609 7fdd667a4780  0 osd.65 144279 crush map has
>>> features 262144, adjusting msgr requires for osds
>>> 2013-08-19 23:01:58.315115 7fdd405de700 -1 osd/OSD.cc: In function
>>> 'OSDMapRef OSDService::get_map(epoch_t)' thread 7fdd405de700 time 2013-08-19
>>> 23:01:58.313955
>>> osd/OSD.cc: 4847: FAILED assert(_get_map_bl(epoch, bl))
>>>
>>>   ceph version 0.61.8 (a6fdcca3bddbc9f177e4e2bf0d9cdd85006b028b)
>>>   1: (OSDService::get_map(unsigned int)+0x44b) [0x6f736b]
>>>   2: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&,
>>> PG::RecoveryCtx*, std::set<boost::intrusive_ptr<PG>,
>>> std::less<boost::intrusive_ptr<PG> >,
>>> std::allocator<boost::intrusive_ptr<PG> > >*)+0x3c8) [0x6fa708]
>>>   3: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> >
>>> const&, ThreadPool::TPHandle&)+0x31f) [0x6faf1f]
>>>   4: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> >
>>> const&, ThreadPool::TPHandle&)+0x14) [0x73a9b4]
>>>   5: (ThreadPool::worker(ThreadPool::WorkThread*)+0x68a) [0x8fb69a]
>>>   6: (ThreadPool::WorkThread::entry()+0x10) [0x8fc940]
>>>   7: (()+0x6b50) [0x7fdd6619ab50]
>>>   8: (clone()+0x6d) [0x7fdd646f5a7d]
>>>   NOTE: a copy of the executable, or `objdump -rdS <executable>` is
>>> needed to interpret this.
>>>
>>>
>>> (It's on Debian Wheezy, with a 3.10.5 kernel)
>>>
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@xxxxxxxxxxxxxx
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com





[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux