On Mon, Jun 24, 2019 at 4:30 PM Alex Litvak <alexander.v.litvak@xxxxxxxxx> wrote: > > Jason, > > What are you suggesting to do ? Removing this line from the config database and keeping in config files instead? I think it's a hole right now in the MON config store that should be addressed. I've opened a tracker ticket [1] to support re-opening the admin socket after the MON configs are received (if not overridden in the local conf). > On 6/24/2019 1:12 PM, Jason Dillaman wrote: > > On Mon, Jun 24, 2019 at 2:05 PM Alex Litvak > > <alexander.v.litvak@xxxxxxxxx> wrote: > >> > >> Jason, > >> > >> Here you go: > >> > >> WHO MASK LEVEL OPTION VALUE RO > >> client advanced admin_socket /var/run/ceph/$name.$pid.asok * > > > > This is the offending config option that is causing your warnings. > > Since the mon configs are read after the admin socket has been > > initialized, it is ignored (w/ the warning saying setting this > > property has no effect). > > > >> global advanced cluster_network 10.0.42.0/23 * > >> global advanced debug_asok 0/0 > >> global advanced debug_auth 0/0 > >> global advanced debug_bdev 0/0 > >> global advanced debug_bluefs 0/0 > >> global advanced debug_bluestore 0/0 > >> global advanced debug_buffer 0/0 > >> global advanced debug_civetweb 0/0 > >> global advanced debug_client 0/0 > >> global advanced debug_compressor 0/0 > >> global advanced debug_context 0/0 > >> global advanced debug_crush 0/0 > >> global advanced debug_crypto 0/0 > >> global advanced debug_dpdk 0/0 > >> global advanced debug_eventtrace 0/0 > >> global advanced debug_filer 0/0 > >> global advanced debug_filestore 0/0 > >> global advanced debug_finisher 0/0 > >> global advanced debug_fuse 0/0 > >> global advanced debug_heartbeatmap 0/0 > >> global advanced debug_javaclient 0/0 > >> global advanced debug_journal 0/0 > >> global advanced debug_journaler 0/0 > >> global advanced debug_kinetic 0/0 > >> global advanced debug_kstore 0/0 > >> global advanced debug_leveldb 0/0 > >> global advanced debug_lockdep 0/0 > >> global advanced debug_mds 0/0 > >> global advanced debug_mds_balancer 0/0 > >> global advanced debug_mds_locker 0/0 > >> global advanced debug_mds_log 0/0 > >> global advanced debug_mds_log_expire 0/0 > >> global advanced debug_mds_migrator 0/0 > >> global advanced debug_memdb 0/0 > >> global advanced debug_mgr 0/0 > >> global advanced debug_mgrc 0/0 > >> global advanced debug_mon 0/0 > >> global advanced debug_monc 0/00 > >> global advanced debug_ms 0/0 > >> global advanced debug_none 0/0 > >> global advanced debug_objclass 0/0 > >> global advanced debug_objectcacher 0/0 > >> global advanced debug_objecter 0/0 > >> global advanced debug_optracker 0/0 > >> global advanced debug_osd 0/0 > >> global advanced debug_paxos 0/0 > >> global advanced debug_perfcounter 0/0 > >> global advanced debug_rados 0/0 > >> global advanced debug_rbd 0/0 > >> global advanced debug_rbd_mirror 0/0 > >> global advanced debug_rbd_replay 0/0 > >> global advanced debug_refs 0/0 > >> global basic log_file /dev/null * > >> global advanced mon_cluster_log_file /dev/null * > >> global advanced osd_pool_default_crush_rule -1 > >> global advanced osd_scrub_begin_hour 19 > >> global advanced osd_scrub_end_hour 4 > >> global advanced osd_scrub_load_threshold 0.010000 > >> global advanced osd_scrub_sleep 0.100000 > >> global advanced perf true > >> global advanced public_network 10.0.40.0/23 * > >> global advanced rocksdb_perf true > >> > >> On 6/24/2019 11:50 AM, Jason Dillaman wrote: > >>> On Sun, Jun 23, 2019 at 4:27 PM Alex Litvak > >>> <alexander.v.litvak@xxxxxxxxx> wrote: > >>>> > >>>> Hello everyone, > >>>> > >>>> I encounter this in nautilus client and not with mimic. Removing admin socket entry from config on client makes no difference > >>>> > >>>> Error: > >>>> > >>>> rbd ls -p one > >>>> 2019-06-23 12:58:29.344 7ff2710b0700 -1 set_mon_vals failed to set admin_socket = /var/run/ceph/$name.$pid.asok: Configuration option 'admin_socket' may not be modified at runtime > >>>> 2019-06-23 12:58:29.348 7ff2708af700 -1 set_mon_vals failed to set admin_socket = /var/run/ceph/$name.$pid.asok: Configuration option 'admin_socket' may not be modified at runtime > >>>> > >>>> I have no issues running other ceph clients (no messages on the screen with ceph -s or ceph iostat from the same box.) > >>>> I connected to a few other client nodes and as root I can do the same string > >>>> rbd ls -p one > >>>> > >>>> > >>>> On all the nodes with user libvirt I have seen the admin_socket messages > >>>> > >>>> oneadmin@virt3n1-la:~$ rbd ls -p one --id libvirt > >>>> 2019-06-23 13:16:41.626 7f9ea0ff9700 -1 set_mon_vals failed to set admin_socket = /var/run/ceph/$name.$pid.asok: Configuration option 'admin_socket' may not be modified at runtime > >>>> 2019-06-23 13:16:41.626 7f9e8bfff700 -1 set_mon_vals failed to set admin_socket = /var/run/ceph/$name.$pid.asok: Configuration option 'admin_socket' may not be modified at runtime > >>>> > >>>> I can execute all rbd operations on the cluster from client otherwise. Commenting client in config file makes no difference > >>>> > >>>> This is an optimiised config distributed across the clients it is almost the same as on servers (no libvirt on servers) > >>>> > >>>> [client] > >>>> admin_socket = /var/run/ceph/$name.$pid.asok > >>>> > >>>> [client.libvirt] > >>>> admin socket = /var/run/ceph/$cluster-$type.$id.$pid.$cctid.asok # must be writable by QEMU and allowed by SELinux or AppArmor > >>>> log file = /var/log/ceph/qemu-guest-$pid.log # must be writable by QEMU and allowed by SELinux or AppArmor > >>>> > >>>> # Please do not change this file directly since it is managed by Ansible and will be overwritten > >>>> [global] > >>>> cluster network = 10.0.42.0/23 > >>>> fsid = 3947ba2d-1b01-4909-8e3a-f9714f427483 > >>>> log file = /dev/null > >>>> mon cluster log file = /dev/null > >>>> mon host = [v2:10.0.40.121:3300,v1:10.0.40.121:6789],[v2:10.0.40.122:3300,v1:10.0.40.122:6789],[v2:10.0.40.123:3300,v1:10.0.40.123:6789] > >>>> perf = True > >>>> public network = 10.0.40.0/23 > >>>> rocksdb_perf = True > >>>> > >>>> > >>>> Here is config from mon > >>>> > >>>> NAME VALUE SOURCE OVERRIDES IGNORES > >>>> cluster_network 10.0.42.0/23 file (mon[10.0.42.0/23]) > >>>> daemonize false override > >>>> debug_asok 0/0 mon > >>>> debug_auth 0/0 mon > >>>> debug_bdev 0/0 mon > >>>> debug_bluefs 0/0 mon > >>>> debug_bluestore 0/0 mon > >>>> debug_buffer 0/0 mon > >>>> debug_civetweb 0/0 mon > >>>> debug_client 0/0 mon > >>>> debug_compressor 0/0 mon > >>>> debug_context 0/0 mon > >>>> debug_crush 0/0 mon > >>>> debug_crypto 0/0 mon > >>>> debug_dpdk 0/0 mon > >>>> debug_eventtrace 0/0 mon > >>>> debug_filer 0/0 mon > >>>> debug_filestore 0/0 mon > >>>> debug_finisher 0/0 mon > >>>> debug_fuse 0/0 mon > >>>> debug_heartbeatmap 0/0 mon > >>>> debug_javaclient 0/0 mon > >>>> debug_journal 0/0 mon > >>>> debug_journaler 0/0 mon > >>>> debug_kinetic 0/0 mon > >>>> debug_kstore 0/0 mon > >>>> debug_leveldb 0/0 mon > >>>> debug_lockdep 0/0 mon > >>>> debug_mds 0/0 mon > >>>> debug_mds_balancer 0/0 mon > >>>> debug_mds_locker 0/0 mon > >>>> debug_mds_log 0/0 mon > >>>> debug_mds_log_expire 0/0 mon > >>>> debug_mds_migrator 0/0 mon > >>>> debug_memdb 0/0 mon > >>>> debug_mgr 0/0 mon > >>>> debug_mgrc 0/0 mon > >>>> debug_mon 0/0 mon > >>>> debug_monc 0/00 mon > >>>> debug_ms 0/0 mon > >>>> debug_none 0/0 mon > >>>> debug_objclass 0/0 mon > >>>> debug_objectcacher 0/0 mon > >>>> debug_objecter 0/0 mon > >>>> debug_optracker 0/0 mon > >>>> debug_osd 0/0 mon > >>>> debug_paxos 0/0 mon > >>>> debug_perfcounter 0/0 mon > >>>> debug_rados 0/0 mon > >>>> debug_rbd 0/0 mon > >>>> debug_rbd_mirror 0/0 mon > >>>> debug_rbd_replay 0/0 mon > >>>> debug_refs 0/0 mon > >>>> err_to_stderr true override > >>>> keyring $mon_data/keyring default > >>>> leveldb_block_size 65536 default > >>>> leveldb_cache_size 536870912 default > >>>> leveldb_compression false default > >>>> leveldb_log default > >>>> leveldb_write_buffer_size 33554432 default > >>>> log_file override file[/dev/null],mon[/dev/null] > >>>> log_stderr_prefix debug cmdline > >>>> log_to_stderr true override > >>>> log_to_syslog false override > >>>> mon_allow_pool_delete true mon > >>>> mon_cluster_log_file /dev/null file (mon[/dev/null]) > >>>> mon_cluster_log_to_stderr true cmdline > >>>> mon_data /var/lib/ceph/mon/ceph-storage2n2-la cmdline > >>>> mon_host [v2:10.0.40.121:3300,v1:10.0.40.121:6789],[v2:10.0.40.122:3300,v1:10.0.40.122:6789],[v2:10.0.40.123:3300,v1:10.0.40.123:6789] file > >>>> mon_initial_members storage2n1-la,storage2n2-la,storage2n3-la file > >>>> mon_osd_down_out_interval 300 mon > >>>> osd_pool_default_crush_rule -1 file (mon[-1]) > >>>> osd_scrub_begin_hour 19 mon > >>>> osd_scrub_end_hour 4 mon > >>>> osd_scrub_load_threshold 0.010000 mon > >>>> osd_scrub_sleep 0.100000 mon > >>>> perf true file (mon[true]) > >>>> public_addr v2:10.0.40.122:0/0 cmdline > >>>> public_network 10.0.40.0/23 file (mon[10.0.40.0/23]) > >>>> rbd_default_features 61 default > >>>> rocksdb_perf true file (mon[true]) > >>>> setgroup ceph cmdline > >>>> setuser ceph cmdline > >>> > >>> What's the mon config for the "client.admin" user? "ceph config get > >>> client.admin" > >>> > >>>> > >>>> I am not sure why I am getting this messages and why are they inconsistent across the nodes. For example I am not getting those when I execute rbd in containers running ceph daemons on server > >>>> cluster nodes. Any clue would be appreciated. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> ceph-users mailing list > >>>> ceph-users@xxxxxxxxxxxxxx > >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>> > >>> > >>> > >> > >> > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users@xxxxxxxxxxxxxx > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com [1] http://tracker.ceph.com/issues/40554 -- Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |