How far out are your clocks? It's showing a clock skew, if they're too
far out it can cause issues with cephx.
Otherwise you're probably going to need to check your cephx auth keys.
-Michael
On 26/08/2014 12:26, yuelongguang wrote:
> hi,all
> i have 5 osds and 3 mons. its status is ok then.
> to be mentioned , this cluster has no any data. i just deploy it and
> to be familar with some command lines.
> what is the probpem and how to fix?
> thanks
> ---environment-----
> ceph-release-1-0.el6.noarch
> ceph-deploy-1.5.11-0.noarch
> ceph-0.81.0-5.el6.x86_64
> ceph-libs-0.81.0-5.el6.x86_64
> ---------ceph -s ----------------------
> [root at cephosd1-mona ~]# ceph -s
> cluster 508634f6-20c9-43bb-bc6f-b777f4bb1651
> health HEALTH_WARN 183 pgs peering; 183 pgs stuck inactive; 183
> pgs stuck unclean; clock skew detected on mon.cephosd2-monb,
> mon.cephosd3-monc
> monmap e13: 3 mons at
> {cephosd1-mona=10.154.249.3:6789/0,cephosd2-monb=10.154.249.4:6789/0,cephosd3-monc=10.154.249.5:6789/0},
> election epoch 74, quorum 0,1,2 cephosd1-mona,cephosd2-monb,cephosd3-monc
> osdmap e151: 5 osds: 5 up, 5 in
> pgmap v499: 384 pgs, 4 pools, 0 bytes data, 0 objects
> 201 MB used, 102143 MB / 102344 MB avail
> 167 peering
> 201 active+clean
> 16 remapped+peering
> --------------log--------------------------osd.0------------
> 2014-08-26 19:16:13.926345 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:16:13.926355 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc2a80 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d5960).accept: got bad authorizer
> 2014-08-26 19:16:28.928023 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:16:28.928050 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc2800 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d56a0).accept: got bad authorizer
> 2014-08-26 19:16:28.929139 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:16:28.929237 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38071 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:16:43.930846 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:16:43.930899 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc2580 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d0b00).accept: got bad authorizer
> 2014-08-26 19:16:43.932204 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:16:43.932230 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38073 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:16:58.933526 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:16:58.935094 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc2300 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d0840).accept: got bad authorizer
> 2014-08-26 19:16:58.936239 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:16:58.936261 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38074 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:17:13.937335 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:17:13.937368 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc2080 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d1b80).accept: got bad authorizer
> 2014-08-26 19:17:13.937923 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:17:13.937933 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38075 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:17:28.939439 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:17:28.939455 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc1e00 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d5540).accept: got bad authorizer
> 2014-08-26 19:17:28.939716 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:17:28.939731 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38076 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:17:43.941398 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:17:43.941426 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc1b80 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d0dc0).accept: got bad authorizer
> 2014-08-26 19:17:43.941539 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:17:43.941553 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38077 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:17:58.944113 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:17:58.944257 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc0c80 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x3c2dd80).accept: got bad authorizer
> 2014-08-26 19:17:58.944450 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:17:58.944466 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38078 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:18:13.946363 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:18:13.946388 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc1180 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d6b40).accept: got bad authorizer
> 2014-08-26 19:18:13.946509 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:18:13.946518 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38079 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:18:28.948562 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:18:28.948602 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x4dc1400 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d69e0).accept: got bad authorizer
> 2014-08-26 19:18:28.949006 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:18:28.949042 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38080 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:18:43.950637 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:18:43.950727 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edbe80 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d6880).accept: got bad authorizer
> 2014-08-26 19:18:43.952035 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:18:43.952048 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38082 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:18:58.953732 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:18:58.954129 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3ede900 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d6720).accept: got bad authorizer
> 2014-08-26 19:18:58.954967 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:18:58.954981 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38083 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:19:13.956551 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:19:13.956692 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edf300 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d1a20).accept: got bad authorizer
> 2014-08-26 19:19:13.956929 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:19:13.956942 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38084 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:19:28.958546 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:19:28.958584 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3eddf00 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d5ac0).accept: got bad authorizer
> 2014-08-26 19:19:28.959598 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:19:28.959623 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38085 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
> 2014-08-26 19:19:43.960489 7f114a8d2700 0 cephx: verify_authorizer
> could not decrypt ticket info: error: decryptor.MessageEnd::Exception:
> StreamTransformationFilter: invalid PKCS #7 block padding found
> 2014-08-26 19:19:43.960516 7f114a8d2700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edad00 sd=25 :6800 s=0 pgs=0 cs=0
> l=0 c=0x45d2c00).accept: got bad authorizer
> 2014-08-26 19:19:43.961823 7f114c009700 0 cephx: verify_reply
> couldn't decrypt with error: error decoding block for decryption
> 2014-08-26 19:19:43.961851 7f114c009700 0 -- 11.154.249.2:6800/1667
> >> 11.154.249.7:6800/1599 pipe(0x3edb700 sd=24 :38086 s=1 pgs=0 cs=0
> l=0 c=0x45d23c0).failed verifying authorize reply
>
>
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users at lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140826/0915d0c6/attachment.htm>
