Hi Adam, When I do "ceph tell mon.ceph-1 config set log_to_file true", I see the log file is created. That confirms that those options in command line can only be override by runtime config change. Could you check mon and mgr logging on your setup? Can we remove those options in command line and let logging to be controlled by cluster configuration or configuration file? Another issue is that, log keeps going to /var/lib/docker/containers/<id>/<id>-json.log, which keeps growing up and it's not under logrotate management. How can I stop logging to container stdout/stderr? Setting "log_to_stderr" doesn't help. Thanks! Tony ________________________________________ From: Tony Liu <tonyliu0592@xxxxxxxxxxx> Sent: March 21, 2022 09:41 PM To: Adam King Cc: ceph-users@xxxxxxx; dev@xxxxxxx Subject: [ceph-users] Re: logging with container Hi Adam, # ceph config get mgr log_to_file true # ceph config get mgr log_file /var/log/ceph/$cluster-$name.log # ceph config get osd log_to_file true # ceph config get osd log_file /var/log/ceph/$cluster-$name.log # ls /var/log/ceph/fa771070-a975-11ec-86c7-e4434be9cb2e/ ceph-osd.10.log ceph-osd.13.log ceph-osd.16.log ceph-osd.19.log ceph-osd.1.log ceph-osd.22.log ceph-osd.4.log ceph-osd.7.log ceph-volume.log # ceph version ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable) "log_to_file" and "log_file" are set the same for mgr and osd, but why there is osd log only, but no mgr log? Thanks! Tony ________________________________________ From: Adam King <adking@xxxxxxxxxx> Sent: March 21, 2022 08:26 AM To: Tony Liu Cc: ceph-users@xxxxxxx; dev@xxxxxxx Subject: Re: [ceph-users] logging with container Hi Tony, Afaik those container flags just set the defaults and the config options override them. Setting the necessary flags (https://docs.ceph.com/en/latest/cephadm/operations/#logging-to-files) seemed to work for me. [ceph: root@vm-00 /]# ceph config get osd.0 log_to_file false [ceph: root@vm-00 /]# ceph config show osd.0 log_to_file false [ceph: root@vm-00 /]# ceph config set global log_to_file true [ceph: root@vm-00 /]# ceph config set global mon_cluster_log_to_file true [ceph: root@vm-00 /]# ceph config get osd.0 log_to_file true [ceph: root@vm-00 /]# ceph config show osd.0 log_to_file true [ceph: root@vm-00 /]# ceph version ceph version 16.2.7-601-g179a7bca (179a7bca8a84771b0dde09e26f7a2146a985df90) pacific (stable) [ceph: root@vm-00 /]# exit exit [root@vm-00 ~]# ls /var/log/ceph/413f7ec8-a91e-11ec-9b02-52540092b5a3/ ceph.audit.log ceph.cephadm.log ceph.log ceph-mgr.vm-00.ukcctb.log ceph-mon.vm-00.log ceph-osd.0.log ceph-osd.10.log ceph-osd.2.log ceph-osd.4.log ceph-osd.6.log ceph-osd.8.log ceph-volume.log On Mon, Mar 21, 2022 at 1:06 AM Tony Liu <tonyliu0592@xxxxxxxxxxx<mailto:tonyliu0592@xxxxxxxxxxx>> wrote: Hi, After reading through doc, it's still not very clear to me how logging works with container. This is with Pacific v16.2 container. In OSD container, I see this. ``` /usr/bin/ceph-osd -n osd.16 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug ``` When check ceph configuration. ``` # ceph config get osd.16 log_file /var/log/ceph/$cluster-$name.log # ceph config get osd.16 log_to_file true # ceph config show osd.16 log_to_file false ``` Q1, what's the intention of those log settings in command line? It's high priority and overrides configuration in file and mon. Is there any option not doing that when deploy the container? Q2, since log_to_file is set to false by command line, why there is still loggings in log_file? The same for mgr and mon. What I want is to have everything in log file and minimize the stdout and stderr from container. Because log file is managed by logrotate, it unlikely blow up disk space. But stdout and stderr from container is stored in a single file, not managed by logrotate. It may grow up to huge file. Also, it's easier to check log file by vi than "podman logs". And log file is also collected and stored by ELK for central management. Any comments how I can achieve what I want? Runtime override may not be the best option, cause it's not persistent. Thanks! Tony _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
