ceph can not repair itself after accidental power down, half of pgs are peering

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



hi,all
 
i have 5 osds and 3 mons. its status is ok then.
 
to be mentioned , this cluster has no any data.  i just deploy it and to be familar with some command lines.
what is the probpem and how to fix?
 
thanks
 
 
---environment-----
ceph-release-1-0.el6.noarch
ceph-deploy-1.5.11-0.noarch
ceph-0.81.0-5.el6.x86_64
ceph-libs-0.81.0-5.el6.x86_64
---------ceph -s ----------------------
[root at cephosd1-mona ~]# ceph -s
    cluster 508634f6-20c9-43bb-bc6f-b777f4bb1651
     health HEALTH_WARN 183 pgs peering; 183 pgs stuck inactive; 183 pgs stuck unclean; clock skew detected on mon.cephosd2-monb, mon.cephosd3-monc
     monmap e13: 3 mons at {cephosd1-mona=10.154.249.3:6789/0,cephosd2-monb=10.154.249.4:6789/0,cephosd3-monc=10.154.249.5:6789/0}, election epoch 74, quorum 0,1,2 cephosd1-mona,cephosd2-monb,cephosd3-monc
     osdmap e151: 5 osds: 5 up, 5 in
      pgmap v499: 384 pgs, 4 pools, 0 bytes data, 0 objects
            201 MB used, 102143 MB / 102344 MB avail
                 167 peering
                 201 active+clean
                  16 remapped+peering
 
 
--------------log--------------------------osd.0------------
2014-08-26 19:16:13.926345 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:16:13.926355 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc2a80 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d5960).accept: got bad authorizer
2014-08-26 19:16:28.928023 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:16:28.928050 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc2800 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d56a0).accept: got bad authorizer
2014-08-26 19:16:28.929139 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:16:28.929237 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38071 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:16:43.930846 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:16:43.930899 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc2580 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d0b00).accept: got bad authorizer
2014-08-26 19:16:43.932204 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:16:43.932230 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38073 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:16:58.933526 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:16:58.935094 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc2300 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d0840).accept: got bad authorizer
2014-08-26 19:16:58.936239 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:16:58.936261 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38074 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:17:13.937335 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:17:13.937368 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc2080 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d1b80).accept: got bad authorizer
2014-08-26 19:17:13.937923 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:17:13.937933 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38075 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:17:28.939439 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:17:28.939455 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc1e00 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d5540).accept: got bad authorizer
2014-08-26 19:17:28.939716 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:17:28.939731 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38076 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:17:43.941398 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:17:43.941426 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc1b80 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d0dc0).accept: got bad authorizer
2014-08-26 19:17:43.941539 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:17:43.941553 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38077 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:17:58.944113 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:17:58.944257 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc0c80 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x3c2dd80).accept: got bad authorizer
2014-08-26 19:17:58.944450 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:17:58.944466 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38078 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:18:13.946363 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:18:13.946388 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc1180 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d6b40).accept: got bad authorizer
2014-08-26 19:18:13.946509 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:18:13.946518 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38079 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:18:28.948562 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:18:28.948602 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x4dc1400 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d69e0).accept: got bad authorizer
2014-08-26 19:18:28.949006 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:18:28.949042 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38080 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:18:43.950637 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:18:43.950727 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edbe80 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d6880).accept: got bad authorizer
2014-08-26 19:18:43.952035 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:18:43.952048 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38082 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:18:58.953732 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:18:58.954129 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3ede900 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d6720).accept: got bad authorizer
2014-08-26 19:18:58.954967 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:18:58.954981 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38083 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:19:13.956551 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:19:13.956692 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edf300 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d1a20).accept: got bad authorizer
2014-08-26 19:19:13.956929 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:19:13.956942 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38084 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:19:28.958546 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:19:28.958584 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3eddf00 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d5ac0).accept: got bad authorizer
2014-08-26 19:19:28.959598 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:19:28.959623 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38085 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
2014-08-26 19:19:43.960489 7f114a8d2700  0 cephx: verify_authorizer could not decrypt ticket info: error: decryptor.MessageEnd::Exception: StreamTransformationFilter: invalid PKCS #7 block padding found
2014-08-26 19:19:43.960516 7f114a8d2700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edad00 sd=25 :6800 s=0 pgs=0 cs=0 l=0 c=0x45d2c00).accept: got bad authorizer
2014-08-26 19:19:43.961823 7f114c009700  0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption
2014-08-26 19:19:43.961851 7f114c009700  0 -- 11.154.249.2:6800/1667 >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38086 s=1 pgs=0 cs=0 l=0 c=0x45d23c0).failed verifying authorize reply
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140826/215c956a/attachment.htm>


[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux