Re: MDS Issues

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



All;

When I went to check Wido's suggestion, I found the MDS daemons would start successfully.  I obviously found no significant time differences.

Sorry for making a mountain out of a mole-hill.

Thank you,

Dominic L. Hilsbos, MBA 
Director - Information Technology 
Perform Air International Inc.
DHilsbos@xxxxxxxxxxxxxx 
www.PerformAir.com



-----Original Message-----
From: Wido den Hollander [mailto:wido@xxxxxxxx] 
Sent: Friday, March 06, 2020 12:15 PM
To: Dominic Hilsbos; ceph-users@xxxxxxx
Subject:  Re: MDS Issues



On 3/6/20 7:46 PM, DHilsbos@xxxxxxxxxxxxxx wrote:
> All;
> 
> We are in the middle of upgrading our primary cluster from 14.2.5 to 14.2.8. Our cluster utilizes 6 MDSs for 3 CephFS file systems. 3 MDSs are collocated with MON/MGR, and 3 MDSs are collocated with OSDs.
> 
> At this point we have upgraded all 3 of the MON/MDS/MGR servers. The MDS on 2 of the 3 is currently not working, and we are seeing the below log messages.
> 
> 2020-03-06 11:12:56.184 <> -1 mds.<daemon> unable to obtain rotating service keys; retrying
> 2020-03-06 11:13:26.184 <>  0 monclient: wait_auth_rotating timed out after 30
> 2020-03-06 11:13:26.184 <> -1 mds.<daemon> ERROR: failed to refresh rotating keys, maximum retry time reached.
> 2020-03-06 11:13:26.184 <>  1 mds.<daemon> suicide! Wanted state up:boot
> 
> Any ideas?
> 

Double check: Is the time correct on all the machines?

cephx can have issues if there is a clock issue.

Wido

> Thank you,
> 
> Dominic L. Hilsbos, MBA 
> Director - Information Technology 
> Perform Air International Inc.
> DHilsbos@xxxxxxxxxxxxxx 
> www.PerformAir.com
> 
> 
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
> 
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux