hi, no risk no fun 😂 okay I have reset the settings you mentioned to standard. what you exactly mean with taking offline the osd? ceph orch daemon stop osd.2? or mark down? for the command which path i use? you mean: bluestore-kv /var/lib/ceph/$fsid/osd.2 compact??? Igor Fedotov <ifedotov@xxxxxxx> schrieb am Mi., 6. Okt. 2021, 13:33: > On 10/6/2021 2:16 PM, José H. Freidhof wrote: > > Hi Igor, > > > > yes i have some osd settings set :-) here are my ceph config dump. those > > settings are from a redhat document for bluestore devices > > maybe it is that setting causing this problem? "advanced > > mon_compact_on_trim false"??? > > OMG!!! > > No - mon_compact_on_trim has nothing to deal with bluestore. > > Highly likely it's bluestore_rocksdb_options which hurts... > Documentations tend to fall behind the best practices.... I would > strongly discourage you from using non-default settings unless it's > absolutely clear why this is necessary. > > Even at the first glance the following settings (just a few ones I'm > completely aware) are suboptimal/non-recommended: > > rocksdb_perf > > bluefs_sync_write > > bluefs_csum_type > > > Not to mention bluestore_rocksdb_options which hasn't got much adoption > so far and apparently greatly alters rocksdb behavior... > > > So I would suggest to revert rocksdb options back to default, run the > compaction and if it succeeds monitor the OSD for a while. Then if it > works fine - apply the same for others > > > Hope this helps, > > Igor > > > > > > > i will test it this afternoon... at the moment are everything semi > > prodcuctive and i need to repair one osd node.. because i think of this > > reason the osds crashed on the node and the osd container crashes with a > > dump while coming up now. > > need first to replicate all between all three nodes and then i can take > > offline the osd.2.and test your command. i will inform you later... > > > > root@cd88-ceph-osdh-01:/# ceph config dump > > WHO MASK > > LEVEL OPTION VALUE > > > > > > > > > > > > RO > > global > > advanced leveldb_max_open_files 131072 > > > > > > > > > > global > > advanced mon_compact_on_trim false > > > > > > > > > > global > > dev ms_crc_data false > > > > > > > > > > global > > advanced osd_deep_scrub_interval 1209600.000000 > > > > > > > > > > global > > advanced osd_max_scrubs 16 > > > > > > > > > > global > > advanced osd_scrub_load_threshold 0.010000 > > > > > > > > > > global > > advanced osd_scrub_max_interval 1209600.000000 > > > > > > > > > > global > > advanced osd_scrub_min_interval 86400.000000 > > > > > > > > > > global > > advanced perf true > > > > > > > > > > global > > advanced rbd_readahead_disable_after_bytes 0 > > > > > > > > > > global > > advanced rbd_readahead_max_bytes 4194304 > > > > > > > > > > global > > advanced rocksdb_perf true > > > > > > > > > > global > > advanced throttler_perf_counter false > > > > > > > > > > mon > > advanced auth_allow_insecure_global_id_reclaim false > > > > > > > > > > mon > > advanced cluster_network 10.50.50.0/24 > > > > > > > > > > > > * > > mon > > advanced mon_osd_down_out_interval 300 > > > > > > > > > > mon > > advanced public_network 10.50.50.0/24 > > > > > > > > > > > > * > > mgr > > advanced mgr/cephadm/container_init True > > > > > > > > > > > > * > > mgr > > advanced mgr/cephadm/device_enhanced_scan true > > > > > > > > > > > > * > > mgr > > advanced mgr/cephadm/migration_current 2 > > > > > > > > > > > > * > > mgr > > advanced mgr/cephadm/warn_on_stray_daemons false > > > > > > > > > > > > * > > mgr > > advanced mgr/cephadm/warn_on_stray_hosts false > > > > > > > > > > > > * > > osd > > advanced bluefs_sync_write true > > > > > > > > > > osd > > dev bluestore_cache_autotune true > > > > > > > > > > osd > > dev bluestore_cache_kv_ratio 0.200000 > > > > > > > > > > osd > > dev bluestore_cache_meta_ratio 0.800000 > > > > > > > > > > osd > > dev bluestore_cache_size 2147483648 > > > > > > > > > > osd > > dev bluestore_cache_size_hdd 2147483648 > > > > > > > > > > osd > > advanced bluestore_csum_type none > > > > > > > > > > osd > > dev bluestore_extent_map_shard_max_size 200 > > > > > > > > > > osd > > dev bluestore_extent_map_shard_min_size 50 > > > > > > > > > > osd > > dev bluestore_extent_map_shard_target_size 100 > > > > > > > > > > osd > > advanced bluestore_rocksdb_options > > > compression=kNoCompression,max_write_buffer_number=64,min_write_buffer_number_to_merge=32,recycle_log_file_num=64,compaction_style=kCompactionStyleLevel,write_buffer_size=4MB,target_file_size_base=4MB,max_background_compactions=64,level0_file_num_compaction_trigger=64,level0_slowdown_writes_trigger=128,level0_stop_writes_trigger=256,max_bytes_for_level_base=6GB,compaction_threads=32,flusher_threads=8,compaction_readahead_size=2MB > > * > > osd > > advanced mon_osd_cache_size 1024 > > > > > > > > > > osd > > dev ms_crc_data false > > > > > > > > > > osd > > advanced osd_map_share_max_epochs 5 > > > > > > > > > > osd > > advanced osd_max_backfills 1 > > > > > > > > > > osd > > dev osd_max_pg_log_entries 10 > > > > > > > > > > osd > > dev osd_memory_cache_min 3000000000 > > > > > > > > > > osd > host:cd133-ceph-osdh-01 > > basic osd_memory_target 5797322383 > > > > > > > > > > osd > host:cd133k-ceph-osdh-01 > > basic osd_memory_target 9402402385 > > > > > > > > > > osd host:cd88-ceph-osdh-01 > > basic osd_memory_target 5797322096 > > > > > > > > > > osd > > advanced osd_memory_target_autotune true > > > > > > > > > > osd > > dev osd_min_pg_log_entries 10 > > > > > > > > > > osd > > advanced osd_op_num_shards 8 > > > > > > > > > > > > * > > osd > > advanced osd_op_num_threads_per_shard 2 > > > > > > > > > > > > * > > osd > > dev osd_pg_log_dups_tracked 10 > > > > > > > > > > osd > > dev osd_pg_log_trim_min 10 > > > > > > > > > > osd > > advanced osd_recovery_max_active 3 > > > > > > > > > > osd > > advanced osd_recovery_max_single_start 1 > > > > > > > > > > osd > > advanced osd_recovery_sleep 0.000000 > > > > > > Am Mi., 6. Okt. 2021 um 12:55 Uhr schrieb Igor Fedotov <ifedotov@xxxxxxx > >: > > > >> Jose, > >> > >> In fact 48GB is a way too much for WAL drive - usually the write ahead > log > >> tend to be 2-4 GBs. > >> > >> But in your case it's ~150GB, while DB itself is very small (146MB!!!): > >> > >> WAL 45 GiB 111 GiB 0 B 0 B 0 B > >> 154 GiB 2400 > >> > >> DB 0 B 164 MiB 0 B 0 B 0 B > >> 146 MiB 30 > >> > >> > >> which means that there are some issues with RocksDB's WAL processing, > >> which needs some troubleshooting... > >> > >> Curious if other OSDs are suffering from the same and whether you have > any > >> custom settings for your OSD(s)? > >> > >> Additionally you might want to try the following command to compact this > >> specific OSD manually and check if this would normalize the DB layout - > the > >> majority of data has to be at DB level not WAL. Please share the > resulting > >> layout (reported by "ceph daemon osd.2 bluefs stats" command) after the > >> compaction is fulfiled and OSD is restarted. > >> > >> The compaction command to be applied on an offline OSD: > "ceph-kvstore-tool > >> bluestore-kv <path-to-osd> compact" > >> > >> Even if the above works great please refrain from applying that > compaction > >> to every OSD - let's see how that "compacted" OSD evolves.Would WAL grow > >> again or not? > >> > >> Thanks, > >> > >> Igor > >> > >> > >> > >> > >> > >> > >> On 10/6/2021 1:35 PM, José H. Freidhof wrote: > >> > >> Hello Igor, > >> > >> yes the volume is nvme wal partitions for the bluestore devicegroups > are > >> only 48gb each > >> > >> on each osd node are 1 nvme with 1tb splitted in 20 lvs with 48gb (WAL) > >> on each osd node are 4 ssd with 1tb splitted in 5 lvs with 175gb > (rock.db) > >> on each osd node are 20 hdd with 5.5tb with 1 lvs (block.db) > >> > >> each blustore have 1 partition nvme,ssd and hdd like described in the > >> documentation > >> > https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/ > >> > >> is this to small or can i adjust the max allocation on the wal nvme > device > >> in the ceph configuration? > >> i know that the ssd and nvme are to small for those 5.5tb disk... its 1% > >> only ot the rotation disk. > >> i am new in ceph and still or always learning, but we are in a little > >> hurry because our other datastores are old and full. > >> > >> root@cd88-ceph-osdh-01:/# ceph daemon osd.2 bluestore bluefs device > info > >> { > >> "dev": { > >> "device": "BDEV_WAL", > >> "total": 48318377984, > >> "free": 1044480, > >> "bluefs_used": 48317333504 > >> }, > >> "dev": { > >> "device": "BDEV_DB", > >> "total": 187904811008, > >> "free": 68757217280, > >> "bluefs_used": 119147593728 > >> }, > >> "dev": { > >> "device": "BDEV_SLOW", > >> "total": 6001172414464, > >> "free": 5624912359424, > >> "bluefs_used": 0, > >> "bluefs max available": 5624401231872 > >> } > >> } > >> root@cd88-ceph-osdh-01:/# ceph daemon osd.2 bluefs stats > >> 0 : device size 0xb3ffff000 : using 0xb3ff00000(45 GiB) > >> 1 : device size 0x2bbfffe000 : using 0x1bbeb00000(111 GiB) > >> 2 : device size 0x57541c00000 : using 0x579b592000(350 GiB) > >> RocksDBBlueFSVolumeSelector: wal_total:45902462976, > db_total:178509578240, > >> slow_total:5701113793740, db_avail:103884521472 > >> Usage matrix: > >> DEV/LEV WAL DB SLOW * * > >> REAL FILES > >> LOG 124 MiB 2.3 GiB 0 B 0 B 0 B > >> 7.5 MiB 1 > >> WAL 45 GiB 111 GiB 0 B 0 B 0 B > >> 154 GiB 2400 > >> DB 0 B 164 MiB 0 B 0 B 0 B > >> 146 MiB 30 > >> SLOW 0 B 0 B 0 B 0 B 0 B > 0 > >> B 0 > >> TOTALS 45 GiB 113 GiB 0 B 0 B 0 B > 0 > >> B 2431 > >> MAXIMUMS: > >> LOG 124 MiB 2.3 GiB 0 B 0 B 0 B > 17 > >> MiB > >> WAL 45 GiB 149 GiB 0 B 0 B 0 B > >> 192 GiB > >> DB 0 B 762 MiB 0 B 0 B 0 B > >> 741 MiB > >> SLOW 0 B 0 B 0 B 0 B 0 B > 0 B > >> TOTALS 45 GiB 150 GiB 0 B 0 B 0 B > 0 B > >> > >> Am Mi., 6. Okt. 2021 um 11:45 Uhr schrieb Igor Fedotov < > ifedotov@xxxxxxx>: > >> > >>> Hey Jose, > >>> > >>> it looks like your WAL volume is out of space which looks weird given > >>> its capacity = 48Gb. > >>> > >>> Could you please share the output of the following commands: > >>> > >>> ceph daemon osd.N bluestore bluefs device info > >>> > >>> ceph daemon osd.N bluefs stats > >>> > >>> > >>> Thanks, > >>> > >>> Igor > >>> > >>> > >>> On 10/6/2021 12:24 PM, José H. Freidhof wrote: > >>>> Hello together > >>>> > >>>> we have a running ceph pacific 16.2.5 cluster and i found this > messages > >>> in > >>>> the service logs of the osd daemons. > >>>> > >>>> we have three osd nodes .. each node has 20osds as bluestore with > >>>> nvme/ssd/hdd > >>>> > >>>> is this a bug or maybe i have some settings wrong? > >>>> > >>>> > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:25.821+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x100000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:29.857+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x100000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.073+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x400000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.405+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x100000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.465+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x100000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.529+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x100000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f38eebd4700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [L] New > >>> memtable > >>>> created with log file: #9588. Immutable memtables: 1. > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f38eebd4700 1 bluefs _allocate unable to allocate 0x100000 on bdev > 0, > >>>> allocator name bluefs-wal, allocator type hybrid, capacity > 0xb40000000, > >>>> block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f3905c02700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.547575) > >>>> [db_impl/db_impl_compaction_flush.cc:2198] Calling > >>>> FlushMemTableToOutputFile with column family [L], flush slots > available > >>> 1, > >>>> compaction slots available 1, flush slots scheduled 1, compaction > slots > >>>> scheduled 0 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f3905c02700 4 rocksdb: [flush_job.cc:321] [L] [JOB 5709] Flushing > >>>> memtable with next log file: 9587 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f3905c02700 4 rocksdb: [flush_job.cc:321] [L] [JOB 5709] Flushing > >>>> memtable with next log file: 9588 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f3905c02700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850547916, > >>>> "job": 5709, "event": "flush_started", "num_memtables": 2, > >>> "num_entries": > >>>> 4146, "num_deletes": 0, "total_data_size": 127203926, "memory_usage": > >>>> 130479920, "flush_reason": "Write Buffer Full"} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.545+0000 > >>>> 7f3905c02700 4 rocksdb: [flush_job.cc:350] [L] [JOB 5709] Level-0 > flush > >>>> table #9589: started > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850559292, > >>>> "cf_name": "L", "job": 5709, "event": "table_file_creation", > >>> "file_number": > >>>> 9589, "file_size": 3249934, "table_properties": {"data_size": 3247855, > >>>> "index_size": 1031, "index_partitions": 0, "top_level_index_size": 0, > >>>> "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, > >>>> "filter_size": 197, "raw_key_size": 1088, "raw_average_key_size": 16, > >>>> "raw_value_size": 3246252, "raw_average_value_size": 47739, > >>>> "num_data_blocks": 36, "num_entries": 68, "num_deletions": 32, > >>>> "num_merge_operands": 0, "num_range_deletions": 0, "format_version": > 0, > >>>> "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", > >>>> "column_family_name": "L", "column_family_id": 10, "comparator": > >>>> "leveldb.BytewiseComparator", "merge_operator": "nullptr", > >>>> "prefix_extractor_name": "nullptr", "property_collectors": "[]", > >>>> "compression": "NoCompression", "compression_options": > "window_bits=-14; > >>>> level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; > >>>> enabled=0; ", "creation_time": 1633511730, "oldest_key_time": > >>> 1633511730, > >>>> "file_creation_time": 1633511850}} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: [flush_job.cc:401] [L] [JOB 5709] Level-0 > flush > >>>> table #9589: 3249934 bytes OK > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.559362) > >>>> [memtable_list.cc:447] [L] Level-0 commit table #9589 started > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.559583) > >>>> [memtable_list.cc:503] [L] Level-0 commit table #9589: memtable #1 > done > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.559586) > >>>> [memtable_list.cc:503] [L] Level-0 commit table #9589: memtable #2 > done > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.559601) > >>>> EVENT_LOG_v1 {"time_micros": 1633511850559593, "job": 5709, "event": > >>>> "flush_finished", "output_compression": "NoCompression", "lsm_state": > >>> [8, > >>>> 1, 0, 0, 0, 0, 0], "immutable_memtables": 0} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f3905c02700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.559638) > >>>> [db_impl/db_impl_compaction_flush.cc:205] [L] Level summary: files[8 1 > >>> 0 0 > >>>> 0 0 0] max score 1.00 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f38fb3ed700 4 rocksdb: [compaction/compaction_job.cc:1676] [L] [JOB > >>> 5710] > >>>> Compacting 8@0 + 1@1 files to L1, score 1.00 > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f38fb3ed700 4 rocksdb: [compaction/compaction_job.cc:1680] [L] > >>> Compaction > >>>> start summary: Base version 3090 Base level 0, inputs: [9589(3173KB) > >>>> 9586(4793KB) 9583(1876KB) 9580(194KB) 9576(6417KB) 9573(1078KB) > >>> 9570(405KB) > >>>> 9567(29KB)], [9564(1115KB)] > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.557+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850559956, > >>>> "job": 5710, "event": "compaction_started", "compaction_reason": > >>>> "LevelL0FilesNum", "files_L0": [9589, 9586, 9583, 9580, 9576, 9573, > >>> 9570, > >>>> 9567], "files_L1": [9564], "score": 1, "input_data_size": 19542092} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: [compaction/compaction_job.cc:1349] [L] [JOB > >>> 5710] > >>>> Generated table #9590: 36 keys, 3249524 bytes > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850582987, > >>>> "cf_name": "L", "job": 5710, "event": "table_file_creation", > >>> "file_number": > >>>> 9590, "file_size": 3249524, "table_properties": {"data_size": 3247449, > >>>> "index_size": 1031, "index_partitions": 0, "top_level_index_size": 0, > >>>> "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, > >>>> "filter_size": 197, "raw_key_size": 576, "raw_average_key_size": 16, > >>>> "raw_value_size": 3246252, "raw_average_value_size": 90173, > >>>> "num_data_blocks": 36, "num_entries": 36, "num_deletions": 0, > >>>> "num_merge_operands": 0, "num_range_deletions": 0, "format_version": > 0, > >>>> "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", > >>>> "column_family_name": "L", "column_family_id": 10, "comparator": > >>>> "leveldb.BytewiseComparator", "merge_operator": "nullptr", > >>>> "prefix_extractor_name": "nullptr", "property_collectors": "[]", > >>>> "compression": "NoCompression", "compression_options": > "window_bits=-14; > >>>> level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; > >>>> enabled=0; ", "creation_time": 1633471854, "oldest_key_time": 0, > >>>> "file_creation_time": 1633511850}} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: [compaction/compaction_job.cc:1415] [L] [JOB > >>> 5710] > >>>> Compacted 8@0 + 1@1 files to L1 => 3249524 bytes > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.583469) > >>>> [compaction/compaction_job.cc:760] [L] compacted to: files[0 1 0 0 0 0 > >>> 0] > >>>> max score 0.01, MB/sec: 846.1 rd, 140.7 wr, level 1, files in(8, 1) > >>> out(1) > >>>> MB in(17.5, 1.1) out(3.1), read-write-amplify(1.2) write-amplify(0.2) > >>> OK, > >>>> records in: 376, records dropped: 340 output_compression: > NoCompression > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: (Original Log Time > 2021/10/06-09:17:30.583498) > >>>> EVENT_LOG_v1 {"time_micros": 1633511850583485, "job": 5710, "event": > >>>> "compaction_finished", "compaction_time_micros": 23098, > >>>> "compaction_time_cpu_micros": 20039, "output_level": 1, > >>> "num_output_files": > >>>> 1, "total_output_size": 3249524, "num_input_records": 376, > >>>> "num_output_records": 36, "num_subcompactions": 1, > "output_compression": > >>>> "NoCompression", "num_single_delete_mismatches": 0, > >>>> "num_single_delete_fallthrough": 0, "lsm_state": [0, 1, 0, 0, 0, 0, > 0]} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583615, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9589} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583648, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9586} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583675, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9583} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583709, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9580} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583739, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9576} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583769, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9573} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583804, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9570} > >>>> cd88-ceph-osdh-01 bash[6283]: debug 2021-10-06T09:17:30.581+0000 > >>>> 7f38fb3ed700 4 rocksdb: EVENT_LOG_v1 {"time_micros": > 1633511850583835, > >>>> "job": 5710, "event": "table_file_deletion", "file_number": 9567} > >>>> _______________________________________________ > >>>> ceph-users mailing list -- ceph-users@xxxxxxx > >>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx > >> > >> -- > >> > >> Mit freundlichen Grüßen, > >> > >> - > >> > >> José H. Freidhof > >> > >> Reyerhütterstrasse 130b > >> 41065 Mönchengladbach > >> eMail: harald.freidhof@xxxxxxxxx > >> mobil: +49 (0) 1523 – 717 7801 > >> > >> > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |