Re: Very chatty MON logs: Is this "normal"?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 06/17/2015 08:30 PM, Somnath Roy wrote:
> << However, I'd rather not set the level to 0/0, as that would disable all logging from the MONs
> 
> I don't think so. All the error scenarios and stack trace (in case of crash) are supposed to be logged with log level 0. But, generally, we need the highest log level (say 20) to get all the information when something to debug.
> So, I doubt how beneficial it will be to enable logging for some intermediate levels.
> Probably, there is no guideline for these log level too which developer should follow strictly.

I don't think this is documented anywhere, but for a while now we've
been using roughly this approach to debug levels:

-1  - errors.
 0  - info you really want in the log each time it happens.
 1  - info that should be outputted by default
      should be stuff that don't happen often and is quite important to
      get to the logs when it happens.
 5  - important but happens a bit often not to output as 1
10  - gross majority of debug messages in the monitor
20  - debug that could impact monitor performance severely
      (e.g., debug from inside a loop)
30  - debug that you should not need unless you're really looking for it

It is fairly common a developer will ask you for 'debug mon = 10' in
order to catch all debug messages at levels 5 and 10, because those are
the ones that usually pay off when tracking down issues.

But given this is left pretty much to the developer's criteria,
different services may use different levels of verbosity for different
things, and you may need a higher debug level to get info out of some
parts of the code than others.

In this particular case, the message that is being outputted should,
imo, be on debug level 5 instead of 1.  We used to output a lot of stuff
on debug level 1, but have been moving away from that; there are still
artifacts though, and this is one of them.

Setting 'mon debug = 0/5' should be okay.  Unless you see that setting
'/5' impacts your performance and/or memory consumption, you should
leave that be.  '0/5' means 'output only debug 0 or lower to the logs;
keep the last 1000 debug level 5 or lower in memory in case of a crash'.
Your logs will not be as heavily populated but, if for some reason the
daemon crashes, you get quite a few of debug information to help track
down the source of the problem.

HTH,

  -Joao

> 
> Thanks & Regards
> Somnath
> 
> -----Original Message-----
> From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Daniel Schneller
> Sent: Wednesday, June 17, 2015 12:11 PM
> To: ceph-users@xxxxxxxxxxxxxx
> Subject: Re:  Very chatty MON logs: Is this "normal"?
> 
> On 2015-06-17 18:52:51 +0000, Somnath Roy said:
> 
>> This is presently written from log level 1 onwards :-) So, only log
>> level 0 will not log this..
>> Try, 'debug_mon = 0/0' in the conf file..
> 
> Yeah, once I had sent the mail I realized that "1" in the log line was the level. Had overlooked that before.
> However, I'd rather not set the level to 0/0, as that would disable all logging from the MONs.
> 
>> Now, I don't have enough knowledge on that part to say whether it is
>> important enough to log at log level 1 , sorry :-(
> 
> That would indeed be an interesting to know.
> Judging from the sheer amount, at least I have my doubts, because the cluster seems to be running without any issues. So I figure at least it isn't indicative of an immediate issue.
> 
> Anyone with a little more definitve knowledge around? Should I create a bug ticket for this?
> 
> Cheers,
> Daniel
> 
> 
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> ________________________________
> 
> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
> 
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux