Re: Multiple OSD crashes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The issue should have been resolved by backport https://tracker.ceph.com/issues/40424 in nautilus, was it merged into 14.2.2 ?

Also do you think it is safe to upgrade from 14.2.1 to 14.2.2 ?

On 7/19/2019 1:05 PM, Paul Emmerich wrote:
I've also encountered a crash just like that after upgrading to 14.2.2

Looks like this issue: http://tracker.ceph.com/issues/37282

--
Paul Emmerich

Looking for help with your Ceph cluster? Contact us at https://croit.io

croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io <http://www.croit.io>
Tel: +49 89 1896585 90


On Fri, Jul 19, 2019 at 11:36 AM Daniel Aberger - Profihost AG <d.aberger@xxxxxxxxxxxx <mailto:d.aberger@xxxxxxxxxxxx>> wrote:

    Hello,

    we are experiencing crashing OSDs in multiple independent Ceph clusters.

    Each OSD has very similar log entries regarding the crash as far as I
    can tell.

    Example log: https://pastebin.com/raw/vQ2AJ5ud

    I can provide you with more log files. They are too large for pastebin
    and I'm not aware of this mailing lists email attachement policy.

    Every log consists of the following entries:

    2019-07-10 21:36:31.903886 7f322aeff700 -1 rocksdb: submit_transaction
    error: Corruption: block checksum mismatch code = 2 Rocksdb transaction:
    Put( Prefix = M key =
    0x00000000000008c1'.0000461231.00000000000125574325' Value size = 184)
    Put( Prefix = M key = 0x00000000000008c1'._fastinfo' Value size = 186)
    Put( Prefix = O key =
    0x7f80000000000000015806b4'(!rbd_data.7c012a6b8b4567.000000000000004e!='0xfffffffffffffffeffffffffffffffff6f002f0000'x'
    Value size = 325)
    Put( Prefix = O key =
    0x7f80000000000000015806b4'(!rbd_data.7c012a6b8b4567.000000000000004e!='0xfffffffffffffffeffffffffffffffff'o'
    Value size = 1608)
    Put( Prefix = L key = 0x000000000226dc7a Value size = 16440)
    2019-07-10 21:36:31.913113 7f322aeff700 -1
    /build/ceph/src/os/bluestore/BlueStore.cc: In function 'void
    BlueStore::_kv_sync_thread()' thread 7f322aeff700 time 2019-07-10
    21:36:31.903909
    /build/ceph/src/os/bluestore/BlueStore.cc: 8808: FAILED assert(r == 0)

      ceph version 12.2.12-7-g1321c5e91f
    (1321c5e91f3d5d35dd5aa5a0029a54b9a8ab9498) luminous (stable)


    Unfortunately I'm unable to interpret the dumps. I hope you can help me
    with this issue.

    Regards,
    Daniel



-- Mit freundlichen Grüßen
       Daniel Aberger
    Ihr Profihost Team

    -------------------------------
    Profihost AG
    Expo Plaza 1
    30539 Hannover
    Deutschland

    Tel.: +49 (511) 5151 8181     | Fax.: +49 (511) 5151 8282
    URL: http://www.profihost.com | E-Mail: info@xxxxxxxxxxxxx <mailto:info@xxxxxxxxxxxxx>

    Sitz der Gesellschaft: Hannover, USt-IdNr. DE813460827
    Registergericht: Amtsgericht Hannover, Register-Nr.: HRB 202350
    Vorstand: Cristoph Bluhm, Sebastian Bluhm, Stefan Priebe
    Aufsichtsrat: Prof. Dr. iur. Winfried Huck (Vorsitzender)
    _______________________________________________
    ceph-users mailing list
    ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>
    http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux