Hello,
I did not do anything special that I know of. I was just exporting an
image from Openstack. We have recently upgraded from Jewel 10.2.3 to
Luminous 12.2.2.
Caps for admin:
client.admin
key: CENSORED
auid: 0
caps: [mgr] allow *
caps: [mon] allow *
caps: [osd] allow *
Caps for Cinder:
client.cinder
key: CENSORED
caps: [mgr] allow r
caps: [mon] profile rbd, allow command "osd blacklist"
caps: [osd] profile rbd pool=vms, profile rbd pool=volumes,
profile rbd pool=images
Caps for MGR:
mgr.0
key: CENSORED
caps: [mon] allow *
I believe this is causing the virtual machines we have running to
crash. Any advice would be appreciated. Please let me know if I need
to provide any other details. Thank you,
Cary
-Dynamic
On Mon, Jan 29, 2018 at 7:53 PM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote:
> On Fri, Jan 26, 2018 at 12:14 PM Cary <dynamic.cary@xxxxxxxxx> wrote:
>>
>> Hello,
>>
>> We are running Luminous 12.2.2. 6 OSD hosts with 12 1TB OSDs, and 64GB
>> RAM. Each host has a SSD for Bluestore's block.wal and block.db.
>> There are 5 monitor nodes as well with 32GB RAM. All servers have
>> Gentoo with kernel, 4.12.12-gentoo.
>>
>> When I export an image using:
>> rbd export pool-name/volume-name /location/image-name.raw
>>
>> Message similar to below are displayed. The signature check fails
>> randomly. And sometimes a message about a bad authorizer, but not
>> everytime.
>> The image is still exported successfully.
>>
>> 2018-01-24 17:35:15.616080 7fc8d4024700 0 cephx:
>> verify_authorizer_reply bad nonce got 4552544084014661633 expected
>> 4552499520046621785 sent 4552499520046621784
>> 2018-01-24 17:35:15.616098 7fc8d4024700 0 --
>> 172.21.32.16:0/1412094654 >> 172.21.32.6:6802/6219 conn(0x7fc8b0078a50
>> :-1 s=STATE_CONNECTING_WAIT_CONNECT_REPLY_AUTH pgs=0 cs=0
>> l=1)._process_connection failed verifying authorize reply
>> 2018-01-24 17:35:15.699004 7fc8d4024700 0 SIGN: MSG 2 Message
>> signature does not match contents.
>> 2018-01-24 17:35:15.699020 7fc8d4024700 0 SIGN: MSG 2Signature on
>> message:
>> 2018-01-24 17:35:15.699021 7fc8d4024700 0 SIGN: MSG 2 sig:
>> 8189090775647585001
>> 2018-01-24 17:35:15.699047 7fc8d4024700 0 SIGN: MSG 2Locally
>> calculated signature:
>> 2018-01-24 17:35:15.699048 7fc8d4024700 0 SIGN: MSG 2
>> sig_check:140500325643792
>> 2018-01-24 17:35:15.699049 7fc8d4024700 0 Signature failed.
>> 2018-01-24 17:35:15.699050 7fc8d4024700 0 --
>> 172.21.32.16:0/1412094654 >> 172.21.32.2:6807/153106
>> conn(0x7fc8bc020870 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH
>> pgs=26018 cs=1 l=1).process Signature check failed
>>
>> Does anyone know what could cause this, and what I can do to fix it.
>
>
> That's in the cephx authentication code and it's indicating that the secure
> signature sent with the message isn't what the local node thinks it should
> be. That's pretty odd (a bit flip or something that could actually change it
> ought to trigger the messaging checksums directly) and I'm not quite sure
> how it could happen.
>
> But, as you've noticed, it retries and apparently succeeds. How did you
> notice this?
> -Greg
>
>>
>>
>> Thank you,
>>
>> Cary
>> -Dynamic
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
