On 09/07/17 18:37, John Jasen wrote: > I have multiple servers running stock CentOS 7 rsyslog 7.4.7-16.el7, > which are configured to log locally and over TCP to a remote logserver, > also running stock CentOS 7 rsyslog. The remote server uses imptcp to > receive, and pretty basic rules to parse and commit to disk. > > I have several systems that log prolifically, but periodically, they > stop soon after the remote log server HUPs (daily logrotate). Very soon > after they stop logging (completely, even to local files), the services > on these systems block, and our monitoring system starts alerting. > Restarting rsyslog on the clients proves ineffectual. > > The situation may clear itself without intervention after 90 minutes to > several hours. > > However, this does not happen on all client systems in a similar > situation (CentOS 7, large volume of constant log data); nor does it > happen daily. > > Any ideas as to what's going on? > > Thanks in advance. > Sorry for the late answer, but can you give more details ? I remember having seen that kind of issue only when sending other logs that the default one (so when using imfile plugin, tracking other files like httpd logs as an example) What are your rules ? How is the network between all those nodes ? I had also an issue over "unreliable" network with buffer/queue and also when the receiver had his main msg queue size too small. Some parameters that can help (?) : # sender size $WorkDirectory /var/lib/rsyslog # default location for work (spool) files $ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName forwardqueue # set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down # receiver side $MainMsgQueueSize 100000 -- Fabian Arrotin The CentOS Project | http://www.centos.org gpg key: 56BEC54E | twitter: @arrfab
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ CentOS mailing list CentOS@xxxxxxxxxx https://lists.centos.org/mailman/listinfo/centos
