syslog broke my cluster

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




As per my previous messages on the list, I was having a strange problem in my test cluster (Hammer 0.94.6, CentOS 6.5) where my monitors were literally crawling to a halt, preventing them to ever reach quorum and causing all sort of problems. As it turned out, to my surprise everything went back to normal as soon as I turned off syslog -- special thanks to Sean!

The slowdown with syslog on was so severe that logs were being written with a timestamp that was several minutes (and eventually up to hours) behind the system clock. The logs from my 4 monitors can be seen in the links below:

https://gist.github.com/anonymous/85213467f701c5a69c7fdb4e54bc7406
https://gist.github.com/anonymous/f30a8903e701423825fd4d5aaa651e6a
https://gist.github.com/anonymous/42a1856cc819de5b110d9f887e9859d2
https://gist.github.com/anonymous/652bc41197e83a9d76cf5b2e6a211aa2

I'm still trying to understand what is going on with my syslog servers but I was wondering... is this a known/documented issue?

Luckily this was a test cluster but I'm worried I could hit this on a production cluster any time soon, and I'm wondering how I could detect it before my support engineers loose their minds.

Thanks,

Sergio


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux