Re: logging with container

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Adam,

When I do "ceph tell mon.ceph-1 config set log_to_file true",
I see the log file is created. That confirms that those options in command line
can only be override by runtime config change.
Could you check mon and mgr logging on your setup?

Can we remove those options in command line and let logging to be controlled
by cluster configuration or configuration file?

Another issue is that, log keeps going to /var/lib/docker/containers/<id>/<id>-json.log,
which keeps growing up and it's not under logrotate management. How can I stop
logging to container stdout/stderr? Setting "log_to_stderr" doesn't help.


Thanks!
Tony
________________________________________
From: Tony Liu <tonyliu0592@xxxxxxxxxxx>
Sent: March 21, 2022 09:41 PM
To: Adam King
Cc: ceph-users@xxxxxxx; dev@xxxxxxx
Subject:  Re: logging with container

Hi Adam,

# ceph config get mgr log_to_file
true
# ceph config get mgr log_file
/var/log/ceph/$cluster-$name.log
# ceph config get osd log_to_file
true
# ceph config get osd log_file
/var/log/ceph/$cluster-$name.log
# ls /var/log/ceph/fa771070-a975-11ec-86c7-e4434be9cb2e/
ceph-osd.10.log  ceph-osd.13.log  ceph-osd.16.log  ceph-osd.19.log  ceph-osd.1.log  ceph-osd.22.log  ceph-osd.4.log  ceph-osd.7.log  ceph-volume.log
# ceph version
ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)

"log_to_file" and "log_file" are set the same for mgr and osd, but why there is osd log only,
but no mgr log?


Thanks!
Tony
________________________________________
From: Adam King <adking@xxxxxxxxxx>
Sent: March 21, 2022 08:26 AM
To: Tony Liu
Cc: ceph-users@xxxxxxx; dev@xxxxxxx
Subject: Re:  logging with container

Hi Tony,

Afaik those container flags just set the defaults and the config options override them. Setting the necessary flags (https://docs.ceph.com/en/latest/cephadm/operations/#logging-to-files) seemed to work for me.

[ceph: root@vm-00 /]# ceph config get osd.0 log_to_file
false
[ceph: root@vm-00 /]# ceph config show osd.0 log_to_file
false
[ceph: root@vm-00 /]# ceph config set global log_to_file true
[ceph: root@vm-00 /]# ceph config set global mon_cluster_log_to_file true
[ceph: root@vm-00 /]# ceph config get osd.0 log_to_file
true
[ceph: root@vm-00 /]# ceph config show osd.0 log_to_file
true
[ceph: root@vm-00 /]# ceph version
ceph version 16.2.7-601-g179a7bca (179a7bca8a84771b0dde09e26f7a2146a985df90) pacific (stable)
[ceph: root@vm-00 /]# exit
exit
[root@vm-00 ~]# ls /var/log/ceph/413f7ec8-a91e-11ec-9b02-52540092b5a3/
ceph.audit.log  ceph.cephadm.log  ceph.log  ceph-mgr.vm-00.ukcctb.log  ceph-mon.vm-00.log  ceph-osd.0.log  ceph-osd.10.log  ceph-osd.2.log  ceph-osd.4.log  ceph-osd.6.log  ceph-osd.8.log  ceph-volume.log



On Mon, Mar 21, 2022 at 1:06 AM Tony Liu <tonyliu0592@xxxxxxxxxxx<mailto:tonyliu0592@xxxxxxxxxxx>> wrote:
Hi,

After reading through doc, it's still not very clear to me how logging works with container.
This is with Pacific v16.2 container.

In OSD container, I see this.
```
/usr/bin/ceph-osd -n osd.16 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug
```
When check ceph configuration.
```
# ceph config get osd.16 log_file
/var/log/ceph/$cluster-$name.log
# ceph config get osd.16 log_to_file
true
# ceph config show osd.16 log_to_file
false
```
Q1, what's the intention of those log settings in command line? It's high priority and overrides
configuration in file and mon. Is there any option not doing that when deploy the container?
Q2, since log_to_file is set to false by command line, why there is still loggings in log_file?

The same for mgr and mon.

What I want is to have everything in log file and minimize the stdout and stderr from container.
Because log file is managed by logrotate, it unlikely blow up disk space. But stdout and stderr
from container is stored in a single file, not managed by logrotate. It may grow up to huge file.
Also, it's easier to check log file by vi than "podman logs". And log file is also collected and
stored by ELK for central management.

Any comments how I can achieve what I want?
Runtime override may not be the best option, cause it's not persistent.


Thanks!
Tony

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx>
To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx>

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux