On Fri, Jan 5, 2018 at 10:24 AM, Sage Weil <sweil@xxxxxxxxxx> wrote:
> On Fri, 5 Jan 2018, Gregory Farnum wrote:
>> On Fri, Jan 5, 2018 at 9:35 AM, Sage Weil <sweil@xxxxxxxxxx> wrote:
>> > My mon-based config branch is coming together. The last bit I did was the
>> > CLI commands to set and show config. It didn't come out quite like I
>> > expected it would. Here's what there is:
>> >
>> > config dump Show all configuration option(s)
>> > config get <who> {<key>} Show configuration option(s) for an entity
>> > config rm <who> <name> Clear a configuration option for one or more
>> > entities
>> > config set <who> <name> <value> Cet a configuration option for one or more
>> > entities
>> > config show <who> Show running configuration
>> >
>> > The show command is the oddball here: it reports what the *daemon* reports
>> > as the running config, which includes other inputs (conf file, overrides,
>> > etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
>> > if there is a better command than 'show'? Maybe 'running' or 'active' or
>> > 'report' or something?
>>
>> I don't have a good suggestion for this name, but I was a little
>> puzzled until I realized it seems to only show the options which are
>> non-default? Given that defaults can change across software versions,
>> I'm not sure that's a feasible long-term strategy. (In addition to
>> dealing with whether we're comparing against the defaults on the OSD
>> version or the monitor version, if I get a dump of options from
>> somebody to do debugging with I'm not going to remember that a timeout
>> changed between point releases and they're operating differently!)
>>
>> Although I definitely get the desire to make the config legible when
>> dumped, I'm not sure there's much utility there as a default anyway —
>> won't admins mostly be querying for a specific value or else running
>> diffs between different dumps? Perhaps we can deal with the bad naming
>> by having "ceph config show non-default <who>", "ceph config show
>> current <who>", and whatever else we like.
>
> Yeah, defaults are excluded because there are 1000+ options and the output
> becomes unusable. I'd prefer to make the less-standard command the one
> that dumps everything (including defaults), like show-all, or similar.
> Note though that currently the daemons aren't reporting their default
> values to the mgr at all--this keeps the MMgrReport message smaller.
Hmm. Like I said, this is going to kill somebody on debugging someday.
I think we just need to eat it. (We could only send the config options
when one changes, if we're worried about their size or parsing cost.)
>
>> (Maybe options for showing configs under USER, ADVANCED, and DEV?)
>
> Oh yeah, I forgot about these! I guess I should add columns for these for
> both 'dump' and 'get'?
>
> Also, I guess the other main missing piece is 'ceph config help <option>',
> which should present a friendly summary of the option, its description,
> defaults, and so on?
>
>> Looking at the output, the WHO versus MASK bit is a little thorny. Can
>> you show what it looks like when there are multiple masks on the same
>> option?
>
> gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd/class:ssd/host:gnit debug_bluestore 20
> gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
> WHO MASK OPTION VALUE
> osd host:gnit/class:ssd debug_bluestore 20/20
> gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd.0/class:ssd/host:gnit debug_bluestore 21
> gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
> WHO MASK OPTION VALUE
> osd.0 host:gnit/class:ssd debug_bluestore 21/21
I was unclear. I mean, what's it look like when there's a setting for
osd/class:ssd and a different setting for osd/class:hdd? ...which I
now realize you did have, but I missed it and that's a good
illustration for what I meant. See end of email.
>> It looks like things are dumped in config declaration order, right?
>> Given that, maybe OPTION should be the "top" (leftmost) column and
>> then WHO and MASK can be filed underneath it, with nicely indented
>> output to make it more like a tree.
>> Or else make it more like a tree in the other direction, with either
>> WHO or MASK as the root and with output order controlled that way. But
>> that probably involved a lot more repetition of the output.
>
> They're sorted by section (global, type, daemon), then by option name.
> Indenting is a great idea... I'd probably indent just the 'who' portion,
> like so? Is that what you mean?
>
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> global mon_pg_warn_min_per_osd 3
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> client admin_socket /tmp/ceph-asok.A5CIZs/$name.$pid.asok
> client log_file /home/sage/src/ceph6/build/out/$name.$pid.log
> client.rgw rgw_crypt_require_ssl false
> client.rgw rgw_crypt_s3_kms_encryption_keys testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=
> client.rgw rgw_frontends civetweb port=8000
> client.rgw rgw_lc_debug_interval 10
So that's not what I meant, but I like it if we're going this way.
> mds debug_mds 20/20
> mds debug_mgrc 20/20
> osd bluestore_block_create true
> osd bluestore_block_db_create true
> osd debug_bluestore 30/30
> osd host:gnit/class:hdd debug_bluestore 20/20
> osd host:gnit/class:ssd debug_bluestore 20/20
> osd.0 host:gnit/class:ssd debug_bluestore 21/21
>
> I'm not sure sorting by the option name would work well here since you'd
> have, say, all of the debug_* options grouped together but applying to
> different things. The idea was to easily see what the osd configs are,
> similar to what you get when looking at the old ini file. And then also
> see any specializations per daemon below it.
Here's (a bad manually-typed illustration of) what I was suggesting:
OPTION WHO MASK
VALUE
log_file client
/tmp/ceph-asok.A5CIZs/$name.$pid.asok
mds
/tmp/ceph-asok.A5CIZs/mds.$id.$pid.asok
host:gnit
/tmp/ceph-asok.special_gnit/mds.$id.$pid.asok
bluestore_block_create osd
true
bluestore_block_db_create osd
true
debug_bluestore osd
host:gnit/class:hdd 20/20
host:gnit/class:ssd 20/20
osd.0
host:gnit/class:ssd 21/21
debug_filestore osd
10/10
So this way the options which have different settings can be visually
identified; and the same for masks versus daemon-type. (Although I'm
not sure which order we want those columns to go in.)
-Greg
>
> sage
>
>
>> -Greg
>>
>> >
>> > A sample from my vstart cluster (vstart is only putting some of its
>> > options in the mon so far, so this shows both conf and mon as sources):
>> >
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > global mon_pg_warn_min_per_osd 3
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > mds mds_debug_auth_pins true
>> > mds mds_debug_frag true
>> > mds mds_debug_subtrees true
>> > mon mon_allow_pool_deletes true
>> > mon mon_data_avail_crit 1
>> > mon mon_data_avail_warn 2
>> > mon mon_osd_reporter_subtree_level osd
>> > mon osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config set osd.0 debug_osd 33
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0 debug_osd
>> > WHO MASK OPTION VALUE
>> > osd.0 debug_osd 33/33
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config set host:gnit debug_monc 15
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > global host:gnit debug_monc 15/15
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config show osd.0
>> > NAME VALUE SOURCE OVERRIDES
>> > admin_socket /tmp/ceph-asok.r90bKw/$name.asok file
>> > bluestore_block_create 1 file
>> > bluestore_block_db_create 1 file
>> > bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file file
>> > bluestore_block_db_size 67108864 file
>> > bluestore_block_wal_create 1 file
>> > bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file file
>> > bluestore_block_wal_size 1048576000 file
>> > bluestore_fsck_on_mount 1 file
>> > chdir file
>> > debug_bdev 20/20 file
>> > debug_bluefs 20/20 file
>> > debug_bluestore 30/30 file
>> > debug_filestore 20/20 file
>> > debug_journal 20/20 file
>> > debug_mgrc 20/20 file
>> > debug_monc 20/20 file
>> > debug_ms 1/1 file
>> > debug_objclass 20/20 file
>> > debug_objecter 20/20 file
>> > debug_osd 25/25 file mon
>> > debug_reserver 10/10 file
>> > debug_rocksdb 10/10 file
>> > enable_experimental_unrecoverable_data_corrupting_features * file
>> > erasure_code_dir /home/sage/src/ceph6/build/lib file
>> > filestore_fd_cache_size 32 file
>> > filestore_wbthrottle_btrfs_inodes_hard_limit 30 file
>> > filestore_wbthrottle_btrfs_ios_hard_limit 20 file
>> > filestore_wbthrottle_btrfs_ios_start_flusher 10 file
>> > filestore_wbthrottle_xfs_inodes_hard_limit 30 file
>> > filestore_wbthrottle_xfs_ios_hard_limit 20 file
>> > filestore_wbthrottle_xfs_ios_start_flusher 10 file
>> > heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
>> > keyring $osd_data/keyring default
>> > leveldb_log default
>> > lockdep 1 file
>> > log_file /home/sage/src/ceph6/build/out/$name.log file
>> > mon_osd_backfillfull_ratio 0.99 file
>> > mon_osd_full_ratio 0.99 file
>> > mon_osd_nearfull_ratio 0.99 file
>> > mon_pg_warn_min_per_osd 3 mon
>> > osd_check_max_object_name_len_on_startup 0 file
>> > osd_class_default_list * file
>> > osd_class_dir /home/sage/src/ceph6/build/lib file
>> > osd_class_load_list * file
>> > osd_copyfrom_max_chunk 524288 mon
>> > osd_data /home/sage/src/ceph6/build/dev/osd$id file
>> > osd_debug_misdirected_ops 1 mon
>> > osd_debug_op_order 1 mon
>> > osd_failsafe_full_ratio 0.99 file
>> > osd_journal /home/sage/src/ceph6/build/dev/osd$id/journal file
>> > osd_journal_size 100 file
>> > osd_objectstore bluestore override file
>> > osd_pool_default_min_size 1 mon
>> > osd_pool_default_size 1 mon
>> > osd_scrub_load_threshold 2000 mon
>> > pid_file /home/sage/src/ceph6/build/out/$name.pid file
>> > plugin_dir /home/sage/src/ceph6/build/lib file
>> > run_dir /home/sage/src/ceph6/build/out file
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config set osd/host:gnit debug_filestore 20
>> > gnit:build (wip-config) 11:32 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > osd host:gnit debug_filestore 20/20
>> > global host:gnit debug_monc 15/15
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:32 AM $ bin/ceph config set osd/class:ssd debug_bluestore 0
>> > gnit:build (wip-config) 11:33 AM $ bin/ceph config set osd/class:hdd debug_bluestore 20
>> > gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > osd class:ssd debug_bluestore 0/0
>> > osd host:gnit debug_filestore 20/20
>> > global host:gnit debug_monc 15/15
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> >
>> > (osd.0 is an ssd)
>> >
>> > Thoughts?
>> > sage
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> > the body of a message to majordomo@xxxxxxxxxxxxxxx
>> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
