Hi,
tl;dr why are my osds still spilling?
I've recently upgraded to 16.2.14 from 16.2.9 and started receiving bluefs
spillover warnings (due to the "fix spillover alert" per the 16.2.14
release notes). E.g. from 'ceph health detail', the warning on one of
these (there are a few):
osd.76 spilled over 128 KiB metadata from 'db' device (56 GiB used of 60 GiB) to slow device
This is a 15T HDD with only a 60G SSD for the db so it's not surprising it
spilled as it's way below the recommendation for rbd usage at db size 1-2%
of the storage size.
There was some spare space on the db ssd so I increased the size of the db
LV up over 400G and did an bluefs-bdev-expand.
However, days later, I'm still getting the spillover warning for that osd,
including after running a manual compact:
# ceph tell osd.76 compact
See attached perf-dump-76 for the perf dump output:
# cephadm enter --name 'osd.76' ceph daemon 'osd.76' perf dump" | jq -r '.bluefs'
In particular, if my understanding is correct, that's telling me the db
available size is 487G (i.e. the LV expand worked), of which it's using
59G, and there's 128K spilled to the slow device:
"db_total_bytes": 512309059584, # 487G
"db_used_bytes": 63470305280, # 59G
"slow_used_bytes": 131072, # 128K
A "bluefs stats" also says the db is using 128K of slow storage (although
perhaps it's getting the info from the same place as the perf dump?):
# ceph tell osd.76 bluefs stats
1 : device size 0x7747ffe000 : using 0xea6200000(59 GiB)
2 : device size 0xe8d7fc00000 : using 0x6554d689000(6.3 TiB)
RocksDBBlueFSVolumeSelector Usage Matrix:
DEV/LEV WAL DB SLOW * * REAL FILES
LOG 0 B 10 MiB 0 B 0 B 0 B 8.8 MiB 1
WAL 0 B 2.5 GiB 0 B 0 B 0 B 751 MiB 8
DB 0 B 56 GiB 128 KiB 0 B 0 B 50 GiB 842
SLOW 0 B 0 B 0 B 0 B 0 B 0 B 0
TOTAL 0 B 58 GiB 128 KiB 0 B 0 B 0 B 850
MAXIMUMS:
LOG 0 B 22 MiB 0 B 0 B 0 B 18 MiB
WAL 0 B 3.9 GiB 0 B 0 B 0 B 1.0 GiB
DB 0 B 71 GiB 282 MiB 0 B 0 B 62 GiB
SLOW 0 B 0 B 0 B 0 B 0 B 0 B
TOTAL 0 B 74 GiB 282 MiB 0 B 0 B 0 B
SIZE << 0 B 453 GiB 14 TiB
I had a look at the "DUMPING STATS" output in the logs bug I don't know
how to interpret it. I did try calculating the total of the sizes on the
"Sum" lines but that comes to 100G so I don't know what that all means.
See attached log-stats-76.
I also tried "ceph-kvstore-tool bluestore-kv ... stats":
$ {
cephadm unit --fsid $clusterid --name osd.76 stop
cephadm shell --fsid $clusterid --name osd.76 -- ceph-kvstore-tool bluestore-kv /var/lib/ceph/osd/ceph-76 stats
cephadm unit --fsid $clusterid --name osd.76 start
}
Output attached as bluestore-kv-stats-76. I can't see anything interesting
in there, although again I don't really know how to interpret it.
So... why is this osd db still spilling onto slow storage, and how do I fix
things so it's no longer using the slow storage?
And a bonus issue... on another osd that hasn't yet been resized (i.e.
again with a grossly undersized 60G db on SSD with a 15T HDD) I'm also
getting a spillover warning. The "bluefs stats" seems to be saying the db
is NOT currently spilling (i.e. "0 B" the DB/SLOW position in the matrix),
but there's "something" currently using 59G on the slow device:
$ ceph tell osd.85 bluefs stats
1 : device size 0xeffffe000 : using 0x3a3900000(15 GiB)
2 : device size 0xe8d7fc00000 : using 0x7aea7434000(7.7 TiB)
RocksDBBlueFSVolumeSelector Usage Matrix:
DEV/LEV WAL DB SLOW * * REAL FILES
LOG 0 B 10 MiB 0 B 0 B 0 B 7.4 MiB 1
WAL 0 B 564 MiB 0 B 0 B 0 B 132 MiB 2
DB 0 B 11 GiB 0 B 0 B 0 B 8.1 GiB 177
SLOW 0 B 3.0 GiB 59 GiB 0 B 0 B 56 GiB 898
TOTAL 0 B 13 GiB 59 GiB 0 B 0 B 0 B 1072
MAXIMUMS:
LOG 0 B 24 MiB 0 B 0 B 0 B 20 MiB
WAL 0 B 2.8 GiB 0 B 0 B 0 B 1.0 GiB
DB 0 B 22 GiB 448 KiB 0 B 0 B 18 GiB
SLOW 0 B 3.3 GiB 62 GiB 0 B 0 B 62 GiB
TOTAL 0 B 27 GiB 62 GiB 0 B 0 B 0 B
SIZE << 0 B 57 GiB 14 TiB
Is there anywhere that describes how to interpret this output, and
specifically, what stuff is going into the SLOW row? Seemingly there's
898 "files" there, but not LOG, WAL or DB files - so what are they?
Cheers,
Chris
{
"db_total_bytes": 512309059584,
"db_used_bytes": 63470305280,
"wal_total_bytes": 0,
"wal_used_bytes": 0,
"slow_total_bytes": 16000896466944,
"slow_used_bytes": 131072,
"num_files": 849,
"log_bytes": 4034560,
"log_compactions": 4362,
"logged_bytes": 64043409408,
"files_written_wal": 2870,
"files_written_sst": 8909,
"bytes_written_wal": 544374063104,
"bytes_written_sst": 263840120832,
"bytes_written_slow": 0,
"max_bytes_wal": 0,
"max_bytes_db": 79799779328,
"max_bytes_slow": 0,
"read_random_count": 20805699,
"read_random_bytes": 4541739952520,
"read_random_disk_count": 20286102,
"read_random_disk_bytes": 4519039577501,
"read_random_buffer_count": 541279,
"read_random_buffer_bytes": 22700375019,
"read_count": 46723,
"read_bytes": 3785087200,
"read_prefetch_count": 18100,
"read_prefetch_bytes": 2921384858,
"compact_lat": {
"avgcount": 4368,
"sum": 54.132304331,
"avgtime": 0.012392926
},
"compact_lock_lat": {
"avgcount": 4362,
"sum": 6.781485867,
"avgtime": 0.001554673
},
"alloc_slow_fallback": 0,
"alloc_slow_size_fallback": 0,
"read_zeros_candidate": 0,
"read_zeros_errors": 0
}
Oct 6 10:58:10 k3 conmon[2099276]: debug 2023-10-05T23:58:10.154+0000 7f65569c9700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS -------
Oct 6 10:58:10 k3 conmon[2099276]: debug 2023-10-05T23:58:10.154+0000 7f65569c9700 4 rocksdb: [db_impl/db_impl.cc:851]
Oct 6 10:58:10 k3 conmon[2099276]: ** DB Stats **
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative writes: 44M writes, 437M keys, 44M commit groups, 1.0 writes per commit group, ingest: 446.32 GB, 0.89 MB/s
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative WAL: 44M writes, 15M syncs, 2.85 writes per sync, written: 446.32 GB, 0.89 MB/s
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Oct 6 10:58:10 k3 conmon[2099276]: Interval writes: 49K writes, 427K keys, 49K commit groups, 1.0 writes per commit group, ingest: 345.26 MB, 0.58 MB/s
Oct 6 10:58:10 k3 conmon[2099276]: Interval WAL: 49K writes, 18K syncs, 2.73 writes per sync, written: 0.34 MB, 0.58 MB/s
Oct 6 10:58:10 k3 conmon[2099276]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [default] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 4.48 MB 0.8 0.0 0.0 0.0 0.6 0.6 0.0 1.0 0.0 9.3 63.17 59.97 447 0.141 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 2/0 119.44 MB 0.6 10.0 0.6 9.4 9.9 0.5 0.0 17.4 24.6 24.4 417.41 360.21 112 3.727 451M 2197K
Oct 6 10:58:10 k3 conmon[2099276]: L2 15/0 936.23 MB 0.4 1.9 0.4 1.5 1.5 0.0 0.0 4.0 37.5 30.0 51.97 39.10 4 12.993 68M 21M
Oct 6 10:58:10 k3 conmon[2099276]: Sum 20/0 1.04 GB 0.0 11.9 1.0 11.0 12.0 1.1 0.0 21.0 22.9 23.2 532.55 459.28 563 0.946 519M 23M
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [default] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 11.9 1.0 11.0 11.5 0.5 0.0 0.0 26.0 25.0 469.38 399.31 116 4.046 519M 23M
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.6 0.6 0.0 0.0 0.0 9.3 63.05 59.97 446 0.141 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.1 0.11 0.00 1 0.115 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.572, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 12.04 GB write, 0.02 MB/s write, 11.92 GB read, 0.02 MB/s read, 532.6 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [default] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [m-0] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [m-1] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(M
Oct 6 10:58:10 k3 conmon[2099276]: B/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [m-2] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [p-0] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 183.13 KB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.6 0.42 0.23 204 0.002 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 3.42 MB 0.0 0.6 0.0 0.5 0.6 0.0 0.0 101.6 57.1 57.1 9.89 6.11 51 0.194 3248K 0
Oct 6 10:58:10 k3 conmon[2099276]: L3 6/0 314.85 MB 0.0 0.3 0.0 0.3 0.3 0.0 0.0 14.1 53.5 53.5 5.88 3.31 1 5.884 1820K 27
Oct 6 10:58:10 k3 conmon[2099276]: Sum 10/0 318.45 MB 0.0 0.9 0.0 0.8 0.9 0.0 0.0 154.4 54.3 54.7 16.19 9.65 256 0.063 5069K 27
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 0.9 0.0 0.8 0.9 0.0 0.0 0.0 55.8 55.8 15.77 9.42 52 0.303 5069K 27
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 13.6 0.42 0.23 204 0.002 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.006, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.86 GB write, 0.00 MB/s write, 0.86 GB read, 0.00 MB/s read, 16.2 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [p-1] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 37.91 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Sum 1/0 37.91 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [p-2] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 8.58 MB 0.8 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 61.9 23.64 16.72 447 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 21.65 MB 0.6 3.1 1.4 1.7 2.9 1.2 0.0 2.1 64.3 60.2 49.76 33.68 112 0.444 20M 249K
Oct 6 10:58:10 k3 conmon[2099276]: L2 9/0 330.23 MB 1.0 10.9 1.2 9.7 10.7 1.0 0.0 8.9 56.5 55.7 197.02 122.95 30 6.567 69M 187K
Oct 6 10:58:10 k3 conmon[2099276]: L3 251/0 15.72 GB 0.6 43.5 1.4 42.1 42.3 0.1 0.0 30.7 64.1 62.3 695.46 331.87 24 28.978 65M 8854K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 264/0 16.07 GB 0.0 57.5 4.0 53.5 57.4 3.8 0.0 40.1 61.0 60.8 965.89 505.21 613 1.576 154M 9291K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 57.5 4.0 53.5 55.9 2.4 0.0 0.0 62.5 60.8 942.24 488.50 166 5.676 154M 9291K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 61.9 23.60 16.72 446 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 63.5 0.04 0.00 1 0.041 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.430, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 57.35 GB write, 0.11 MB/s write, 57.51 GB read, 0.11 MB/s read, 965.9 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [O-0] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 8.23 MB 0.8 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 61.9 23.59 16.76 448 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 0/0 0.00 KB 0.0 3.2 1.4 1.7 3.0 1.2 0.0 2.1 68.3 64.1 47.56 33.35 112 0.425 20M 248K
Oct 6 10:58:10 k3 conmon[2099276]: L2 9/0 312.89 MB 0.9 11.1 1.2 9.9 11.0 1.1 0.0 8.8 56.4 55.6 202.18 125.33 31 6.522 72M 190K
Oct 6 10:58:10 k3 conmon[2099276]: L3 251/0 15.72 GB 0.6 41.0 1.4 39.5 39.7 0.1 0.0 27.6 63.9 61.9 656.08 310.34 24 27.337 62M 9277K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 263/0 16.04 GB 0.0 55.3 4.1 51.2 55.0 3.9 0.0 38.6 60.9 60.6 929.41 485.78 615 1.511 154M 9716K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.5 0.04 0.03 1 0.045 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 55.3 4.1 51.2 53.6 2.4 0.0 0.0 62.5 60.6 905.83 469.02 167 5.424 154M 9716K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 61.9 23.54 16.76 447 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 61.8 0.04 0.00 1 0.042 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.426, interval 0.002
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 55.04 GB write, 0.11 MB/s write, 55.27 GB read, 0.11 MB/s read, 929.4 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [O-1] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 2/0 5.38 MB 0.5 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 62.0 23.60 16.67 447 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 0/0 0.00 KB 0.0 3.1 1.4 1.7 2.9 1.2 0.0 2.1 66.1 62.0 48.47 33.14 112 0.433 20M 246K
Oct 6 10:58:10 k3 conmon[2099276]: L2 9/0 314.58 MB 0.9 11.2 1.2 10.0 11.0 1.1 0.0 8.9 55.8 55.0 205.07 126.68 31 6.615 71M 191K
Oct 6 10:58:10 k3 conmon[2099276]: L3 250/0 15.72 GB 0.6 42.9 1.4 41.5 41.7 0.1 0.0 30.2 63.7 61.9 689.41 324.93 25 27.577 64M 8994K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 261/0 16.04 GB 0.0 57.2 4.0 53.2 57.0 3.9 0.0 39.9 60.6 60.4 966.55 501.41 615 1.572 156M 9433K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 57.2 4.0 53.2 55.6 2.4 0.0 0.0 62.1 60.4 942.95 484.74 168 5.613 156M 9433K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 62.0 23.56 16.67 446 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 62.9 0.04 0.00 1 0.042 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.429, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: A
Oct 6 10:58:10 k3 conmon[2099276]: ddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 57.04 GB write, 0.11 MB/s write, 57.20 GB read, 0.11 MB/s read, 966.6 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [O-2] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [L] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 1/0 459.28 KB 0.2 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 65.0 21.65 14.38 1564 0.014 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 85.54 KB 0.0 1.7 1.4 0.3 0.3 -0.0 0.0 0.2 241.6 46.6 7.23 5.01 391 0.018 170K 151K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 2/0 544.82 KB 0.0 1.7 1.4 0.3 1.7 1.4 0.0 1.2 60.5 60.4 28.88 19.39 1955 0.015 170K 151K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 38.5 0.01 0.01 1 0.012 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [L] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 1.7 1.4 0.3 0.3 -0.0 0.0 0.0 241.6 46.6 7.23 5.01 391 0.018 170K 151K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 65.0 21.64 14.38 1563 0.014 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 21.5 0.01 0.00 1 0.006 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.375, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 1.70 GB write, 0.00 MB/s write, 1.70 GB read, 0.00 MB/s read, 28.9 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [L] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [P] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 116.29 MB 0.8 0.0 0.0 0.0 16.6 16.6 0.0 1.0 0.0 87.3 195.04 118.26 447 0.436 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 3/0 155.14 MB 0.9 31.4 16.6 14.8 25.1 10.3 0.0 1.5 88.7 70.9 362.17 227.31 112 3.234 114M 11M
Oct 6 10:58:10 k3 conmon[2099276]: L2 5/0 277.25 MB 0.1 28.2 10.2 18.0 18.0 0.0 0.0 1.8 101.9 65.0 283.20 156.26 170 1.666 78M 45M
Oct 6 10:58:10 k3 conmon[2099276]: Sum 11/0 548.69 MB 0.0 59.6 26.8 32.8 59.7 26.9 0.0 3.6 72.6 72.7 840.41 501.83 729 1.153 193M 57M
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [P] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 59.6 26.8 32.8 43.0 10.3 0.0 0.0 94.5 68.3 645.37 383.57 282 2.289 193M 57M
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 16.6 16.6 0.0 0.0 0.0 87.3 194.68 118.26 446 0.437 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 91.6 0.36 0.00 1 0.356 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 600.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 16.629, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 59.67 GB write, 0.12 MB/s write, 59.57 GB read, 0.12 MB/s read, 840.4 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [P] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [default] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 4.48 MB 0.8 0.0 0.0 0.0 0.6 0.6 0.0 1.0 0.0 9.3 63.17 59.97 447 0.141 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 2/0 119.44 MB 0.6 10.0 0.6 9.4 9.9 0.5 0.0 17.4 24.6 24.4 417.41 360.21 112 3.727 451M 2197K
Oct 6 10:58:10 k3 conmon[2099276]: L2 15/0 936.23 MB 0.4 1.9 0.4 1.5 1.5 0.0 0.0 4.0 37.5 30.0 51.97 39.10 4 12.993 68M 21M
Oct 6 10:58:10 k3 conmon[2099276]: Sum 20/0 1.04 GB 0.0 11.9 1.0 11.0 12.0 1.1 0.0 21.0 22.9 23.2 532.55 459.28 563 0.946 519M 23M
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [default] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 11.9 1.0 11.0 11.5 0.5 0.0 0.0 26.0 25.0 469.38 399.31 116 4.046 519M 23M
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.6 0.6 0.0 0.0 0.0 9.3 63.05 59.97 446 0.141 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB
Oct 6 10:58:10 k3 conmon[2099276]: 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.1 0.11 0.00 1 0.115 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.572, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 12.04 GB write, 0.02 MB/s write, 11.92 GB read, 0.02 MB/s read, 532.6 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [default] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [m-0] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [m-1] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [m-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [m-2] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfile
Oct 6 10:58:10 k3 conmon[2099276]: s, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [p-0] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 183.13 KB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.6 0.42 0.23 204 0.002 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 3.42 MB 0.0 0.6 0.0 0.5 0.6 0.0 0.0 101.6 57.1 57.1 9.89 6.11 51 0.194 3248K 0
Oct 6 10:58:10 k3 conmon[2099276]: L3 6/0 314.85 MB 0.0 0.3 0.0 0.3 0.3 0.0 0.0 14.1 53.5 53.5 5.88 3.31 1 5.884 1820K 27
Oct 6 10:58:10 k3 conmon[2099276]: Sum 10/0 318.45 MB 0.0 0.9 0.0 0.8 0.9 0.0 0.0 154.4 54.3 54.7 16.19 9.65 256 0.063 5069K 27
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 0.9 0.0 0.8 0.9 0.0 0.0 0.0 55.8 55.8 15.77 9.42 52 0.303 5069K 27
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 13.6 0.42 0.23 204 0.002 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.006, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.86 GB write, 0.00 MB/s write, 0.86 GB read, 0.00 MB/s read, 16.2 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [p-1] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 37.91 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Sum 1/0 37.91 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [p-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [p-2] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 8.58 MB 0.8 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 61.9 23.64 16.72 447 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 21.65 MB 0.6 3.1 1.4 1.7 2.9 1.2 0.0 2.1 64.3 60.2 49.76 33.68 112 0.444 20M 249K
Oct 6 10:58:10 k3 conmon[2099276]: L2 9/0 330.23 MB 1.0 10.9 1.2 9.7 10.7 1.0 0.0 8.9 56.5 55.7 197.02 122.95 30 6.567 69M 187K
Oct 6 10:58:10 k3 conmon[2099276]: L3 251/0 15.72 GB 0.6 43.5 1.4 42.1 42.3 0.1 0.0 30.7 64.1 62.3 695.46 331.87 24 28.978 65M 8854K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 264/0 16.07 GB 0.0 57.5 4.0 53.5 57.4 3.8 0.0 40.1 61.0 60.8 965.89 505.21 613 1.576 154M 9291K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-0] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 57.5 4.0 53.5 55.9 2.4 0.0 0.0 62.5 60.8 942.24 488.50 166 5.676 154M 9291K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 61.9 23.60 16.72 446 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 63.5 0.04 0.00 1 0.041 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.430, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 57.35 GB write, 0.11 MB/s write, 57.51 GB read, 0.11 MB/s read, 965.9 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [O-0] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Writ
Oct 6 10:58:10 k3 conmon[2099276]: e(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 8.23 MB 0.8 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 61.9 23.59 16.76 448 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 0/0 0.00 KB 0.0 3.2 1.4 1.7 3.0 1.2 0.0 2.1 68.3 64.1 47.56 33.35 112 0.425 20M 248K
Oct 6 10:58:10 k3 conmon[2099276]: L2 9/0 312.89 MB 0.9 11.1 1.2 9.9 11.0 1.1 0.0 8.8 56.4 55.6 202.18 125.33 31 6.522 72M 190K
Oct 6 10:58:10 k3 conmon[2099276]: L3 251/0 15.72 GB 0.6 41.0 1.4 39.5 39.7 0.1 0.0 27.6 63.9 61.9 656.08 310.34 24 27.337 62M 9277K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 263/0 16.04 GB 0.0 55.3 4.1 51.2 55.0 3.9 0.0 38.6 60.9 60.6 929.41 485.78 615 1.511 154M 9716K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-1] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 55.3 4.1 51.2 53.6 2.4 0.0 0.0 62.5 60.6 905.83 469.02 167 5.424 154M 9716K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 61.9 23.54 16.76 447 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 61.8 0.04 0.00 1 0.042 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.426, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 55.04 GB write, 0.11 MB/s write, 55.27 GB read, 0.11 MB/s read, 929.4 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [O-1] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 2/0 5.38 MB 0.5 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 62.0 23.60 16.67 447 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 0/0 0.00 KB 0.0 3.1 1.4 1.7 2.9 1.2 0.0 2.1 66.1 62.0 48.47 33.14 112 0.433 20M 246K
Oct 6 10:58:10 k3 conmon[2099276]: L2 9/0 314.58 MB 0.9 11.2 1.2 10.0 11.0 1.1 0.0 8.9 55.8 55.0 205.07 126.68 31 6.615 71M 191K
Oct 6 10:58:10 k3 conmon[2099276]: L3 250/0 15.72 GB 0.6 42.9 1.4 41.5 41.7 0.1 0.0 30.2 63.7 61.9 689.41 324.93 25 27.577 64M 8994K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 261/0 16.04 GB 0.0 57.2 4.0 53.2 57.0 3.9 0.0 39.9 60.6 60.4 966.55 501.41 615 1.572 156M 9433K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [O-2] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 57.2 4.0 53.2 55.6 2.4 0.0 0.0 62.1 60.4 942.95 484.74 168 5.613 156M 9433K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 62.0 23.56 16.67 446 0.053 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 62.9 0.04 0.00 1 0.042 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.429, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 57.04 GB write, 0.11 MB/s write, 57.20 GB read, 0.11 MB/s read, 966.6 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [O-2] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [L] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 1/0 459.28 KB 0.2 0.0 0.0 0.0 1.4 1.4 0.0 1.0 0.0 65.0 21.65 14.38 1564 0.014 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 1/0 85.54 KB 0.0 1.7 1.4 0.3 0.3 -0.0 0.0 0.2 241.6 46.6 7.23 5.01 391 0.018 170K 151K
Oct 6 10:58:10 k3 conmon[2099276]: Sum 2/0 544.82 KB 0.0 1.7 1.4 0.3 1.7 1.4 0.0 1.2 60.5 60.4 28.88 19.39 1955 0.015 170K 151K
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [L] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 1.7 1.4 0.3 0.3 -0.0 0.0 0.0 241.6 46.6 7.23 5.01 391 0.018 170K 151K
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.4 1.4 0.0 0.0 0.0 65.0 21.64 14.38 1563 0.014 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 21.5 0.01 0.00 1 0.006 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 1.375, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumul
Oct 6 10:58:10 k3 conmon[2099276]: ative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 1.70 GB write, 0.00 MB/s write, 1.70 GB read, 0.00 MB/s read, 28.9 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [L] **
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [P] **
Oct 6 10:58:10 k3 conmon[2099276]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: L0 3/0 116.29 MB 0.8 0.0 0.0 0.0 16.6 16.6 0.0 1.0 0.0 87.3 195.04 118.26 447 0.436 0 0
Oct 6 10:58:10 k3 conmon[2099276]: L1 3/0 155.14 MB 0.9 31.4 16.6 14.8 25.1 10.3 0.0 1.5 88.7 70.9 362.17 227.31 112 3.234 114M 11M
Oct 6 10:58:10 k3 conmon[2099276]: L2 5/0 277.25 MB 0.1 28.2 10.2 18.0 18.0 0.0 0.0 1.8 101.9 65.0 283.20 156.26 170 1.666 78M 45M
Oct 6 10:58:10 k3 conmon[2099276]: Sum 11/0 548.69 MB 0.0 59.6 26.8 32.8 59.7 26.9 0.0 3.6 72.6 72.7 840.41 501.83 729 1.153 193M 57M
Oct 6 10:58:10 k3 conmon[2099276]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** Compaction Stats [P] **
Oct 6 10:58:10 k3 conmon[2099276]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
Oct 6 10:58:10 k3 conmon[2099276]: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oct 6 10:58:10 k3 conmon[2099276]: Low 0/0 0.00 KB 0.0 59.6 26.8 32.8 43.0 10.3 0.0 0.0 94.5 68.3 645.37 383.57 282 2.289 193M 57M
Oct 6 10:58:10 k3 conmon[2099276]: High 0/0 0.00 KB 0.0 0.0 0.0 0.0 16.6 16.6 0.0 0.0 0.0 87.3 194.68 118.26 446 0.437 0 0
Oct 6 10:58:10 k3 conmon[2099276]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 91.6 0.36 0.00 1 0.356 0 0
Oct 6 10:58:10 k3 conmon[2099276]: Uptime(secs): 512408.1 total, 0.0 interval
Oct 6 10:58:10 k3 conmon[2099276]: Flush(GB): cumulative 16.629, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(GB): cumulative 0.000, interval 0.000
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Total Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(L0 Files): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: AddFile(Keys): cumulative 0, interval 0
Oct 6 10:58:10 k3 conmon[2099276]: Cumulative compaction: 59.67 GB write, 0.12 MB/s write, 59.57 GB read, 0.12 MB/s read, 840.4 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Oct 6 10:58:10 k3 conmon[2099276]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
Oct 6 10:58:10 k3 conmon[2099276]:
Oct 6 10:58:10 k3 conmon[2099276]: ** File Read Latency Histogram By Level [P] **
Oct 6 10:58:10 k3 conmon[2099276]:
db_statistics {
"rocksdb_compaction_statistics": "",
"": "",
"": "** Compaction Stats [default] **",
"": "Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop",
"": "----------------------------------------------------------------------------------------------------------------------------------------------------------------------------",
"": " L0 2/0 20.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0",
"": " L1 2/0 128.34 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0",
"": " L2 15/0 936.23 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0",
"": " Sum 19/0 1.04 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0",
"": " Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0",
"": "",
"": "** Compaction Stats [default] **",
"": "Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop",
"": "-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------",
"": "User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0",
"": "Uptime(secs): 0.1 total, 0.1 interval",
"": "Flush(GB): cumulative 0.000, interval 0.000",
"": "AddFile(GB): cumulative 0.000, interval 0.000",
"": "AddFile(Total Files): cumulative 0, interval 0",
"": "AddFile(L0 Files): cumulative 0, interval 0",
"": "AddFile(Keys): cumulative 0, interval 0",
"": "Cumulative compaction: 0.00 GB write, 0.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds",
"": "Interval compaction: 0.00 GB write, 0.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds",
"": "Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count",
"": "",
"": "** File Read Latency Histogram By Level [default] **",
"": "** Level 0 read latency histogram (micros):",
"": "Count: 10 Average: 1.7000 StdDev: 1.73",
"": "Min: 0 Median: 1.0000 Max: 6",
"": "Percentiles: P50: 1.00 P75: 1.83 P99: 5.80 P99.9: 5.98 P99.99: 6.00",
"": "------------------------------------------------------",
"": "[ 0, 1 ] 5 50.000% 50.000% ##########",
"": "( 1, 2 ] 3 30.000% 80.000% ######",
"": "( 2, 3 ] 1 10.000% 90.000% ##",
"": "( 4, 6 ] 1 10.000% 100.000% ##",
"": "",
"": "** Level 1 read latency histogram (micros):",
"": "Count: 6 Average: 5.6667 StdDev: 6.29",
"": "Min: 0 Median: 2.0000 Max: 19",
"": "Percentiles: P50: 2.00 P75: 5.50 P99: 19.00 P99.9: 19.00 P99.99: 19.00",
"": "------------------------------------------------------",
"": "[ 0, 1 ] 1 16.667% 16.667% ###",
"": "( 1, 2 ] 2 33.333% 50.000% #######",
"": "( 4, 6 ] 2 33.333% 83.333% #######",
"": "( 15, 22 ] 1 16.667% 100.000% ###",
"": "",
"": "** Level 2 read latency histogram (micros):",
"": "Count: 45 Average: 2.0889 StdDev: 1.90",
"": "Min: 0 Median: 1.1154 Max: 8",
"": "Percentiles: P50: 1.12 P75: 1.98 P99: 8.00 P99.9: 8.00 P99.99: 8.00",
"": "------------------------------------------------------",
"": "[ 0, 1 ] 21 46.667% 46.667% #########",
"": "( 1, 2 ] 13 28.889% 75.556% ######",
"": "( 2, 3 ] 2 4.444% 80.000% #",
"": "( 4, 6 ] 8 17.778% 97.778% ####",
"": "( 6, 10 ] 1 2.222% 100.000% ",
"": "",
"": "",
"": "** DB Stats **",
"": "Uptime(secs): 0.1 total, 0.1 interval",
"": "Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s",
"": "Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s",
"": "Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent",
"": "Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s",
"": "Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s",
"": "Interval stall: 00:00:0.000 H:M:S, 0.0 percent"
}
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
