Hi all,
I have small cluster and yesterday I tried to mount older RBD snapshot torecover data. (I have approx. 230 daily snapshots of one RBD image
on my small ceph).
After I did mount and ls operation, cluster was stuck and I notice that 2of my OSD's eaten CPU and raise in memory usage (more than 4x more
memory than usual). Cluster was unusable after that and it didn't help stopping rbd mount operation and restarting OSD's etc.
Strange is, that it affects only 2 of my OSD's - one is on NVME and has cache PG's mapped, other one is rotational HDD. IOPS are normal,
only CPU and memory usage are unusual.
I noticed hundred thousands of slow requests (on idle cluster) so I startto debug OSD operations. You can find OSD 8 log below. OSD 9
contains nvme cached PG's and there is nothing unusual in standard log. In the middle of log I raised debug level to 10.
It look's to me, that it is trying to do one operation with one snapshotted RBD object in PG 2.b
(rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3) in a loop.
Is there any way how to stop that? What is causing the problem? Can I somehow flush OSD's "journals" or "queues" to stop current operation
and make cluster usable again? Is there any way how I can access old snapshot data?
Problem started on Luminous 12.2.12 and upgrade to 12.2.13 didn't help.
Thank you
Jan Pekar
2020-02-03 15:09:22.470277 7f51f6af7700 0 log_channel(cluster) log [WRN] : 30966 slow requests, 5 included below; oldest blocked for >
37.505482 secs
2020-02-03 15:09:22.470336 7f51f6af7700 0 log_channel(cluster) log [WRN] : slow request 37.503404 seconds old, received at 2020-02-03
15:08:44.956422: osd_op(osd.9.59923:2670991 2.b 2.162d8f8b (undecoded)
ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) currently queued_for_pg
2020-02-03 15:09:22.470360 7f51f6af7700 0 log_channel(cluster) log [WRN] : slow request 37.500695 seconds old, received at 2020-02-03
15:08:44.959132: osd_op(osd.9.59923:2671023 2.b 2.162d8f8b (undecoded)
ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) currently queued_for_pg
2020-02-03 15:09:22.470366 7f51f6af7700 0 log_channel(cluster) log [WRN] : slow request 37.498013 seconds old, received at 2020-02-03
15:08:44.961814: osd_op(osd.9.59923:2671055 2.b 2.162d8f8b (undecoded)
ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) currently queued_for_pg
2020-02-03 15:09:22.470371 7f51f6af7700 0 log_channel(cluster) log [WRN] : slow request 37.494836 seconds old, received at 2020-02-03
15:08:44.964990: osd_op(osd.9.59923:2671087 2.b 2.162d8f8b (undecoded)
ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) currently queued_for_pg
2020-02-03 15:09:22.470376 7f51f6af7700 0 log_channel(cluster) log [WRN] : slow request 37.491950 seconds old, received at 2020-02-03
15:08:44.967877: osd_op(osd.9.59923:2671119 2.b 2.162d8f8b (undecoded)
ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) currently queued_for_pg
...
2020-02-03 15:09:35.537259 7f51ddac5700 0 osd.8 59939 do_command r=0
2020-02-03 15:09:35.588753 7f51ddac5700 0 osd.8 59939 do_command r=0 Set debug_osd to 10/10
2020-02-03 15:09:35.588814 7f51ddac5700 0 log_channel(cluster) log [INF] : Set debug_osd to 10/10
2020-02-03 15:09:35.597667 7f51f0945700 2 osd.8 59939 ms_handle_reset con 0x5647736ac800 session 0x564785519e00
2020-02-03 15:09:35.599056 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] dropping ondisk_read_lock
2020-02-03 15:09:35.599209 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d5800 finish
2020-02-03 15:09:35.599250 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d59c0 prio 63 cost 29 latency 22.224501
osd_op(osd.9.59923:2779946 2.b 2.162d8f8b (undecoded) ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected
e59939) v8 pg pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227] local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f
59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227 lcod 0'0 mlcod 0'0 active+clean]
2020-02-03 15:09:35.599293 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] _handle_message: 0x5647873d59c0
2020-02-03 15:09:35.599347 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_op osd_op(osd.9.59923:2779946 2.b 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get
max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) v8 may_read ->
write-ordered flags ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected
2020-02-03 15:09:35.599398 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
map_snapid_to_clone=true
2020-02-03 15:09:35.599506 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
maps to 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3
2020-02-03 15:09:35.599606 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] get_object_context: found obc in cache: 0x564785fb3880
2020-02-03 15:09:35.613724 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] get_object_context: 0x564785fb3880 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 rwstate(none
n=0 w=0) oi: 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3(58682'4215046 osd.8.0:393237 dirty|data_digest|omap_digest s 4194304
uv 7633387 dd 41a45f29 od ffffffff alloc_hint [0 0 0]) exists: 1 ssc: 0x564783df4160 snapset:
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
2020-02-03 15:09:35.613881 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 HIT
2020-02-03 15:09:35.614024 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] execute_ctx 0x5647843a3000
2020-02-03 15:09:35.614053 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] execute_ctx 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get max 8388608] ov 58682'4215046
2020-02-03 15:09:35.614077 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] taking ondisk_read_lock
2020-02-03 15:09:35.614095 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_osd_op 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get max 8388608]
2020-02-03 15:09:35.614112 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_osd_op copy-get max 8388608
2020-02-03 15:09:35.637123 7f51f72f8700 10 osd.8 59939 tick
2020-02-03 15:09:35.637149 7f51f72f8700 10 osd.8 59939 do_waiters -- start
2020-02-03 15:09:35.637152 7f51f72f8700 10 osd.8 59939 do_waiters -- finish
2020-02-03 15:09:35.645761 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] dropping ondisk_read_lock
2020-02-03 15:09:35.645887 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d59c0 finish
2020-02-03 15:09:35.645924 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d5b80 prio 63 cost 29 latency 22.271117
osd_op(osd.9.59923:2779947 2.b 2.162d8f8b (undecoded) ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected
e59939) v8 pg pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227] local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f
59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227 lcod 0'0 mlcod 0'0 active+clean]
2020-02-03 15:09:35.645960 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] _handle_message: 0x5647873d5b80
2020-02-03 15:09:35.646007 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_op osd_op(osd.9.59923:2779947 2.b 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get
max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) v8 may_read ->
write-ordered flags ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected
2020-02-03 15:09:35.646047 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
map_snapid_to_clone=true
2020-02-03 15:09:35.646145 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
maps to 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3
2020-02-03 15:09:35.646239 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] get_object_context: found obc in cache: 0x564785fb3880
2020-02-03 15:09:35.646255 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] get_object_context: 0x564785fb3880 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 rwstate(none
n=0 w=0) oi: 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3(58682'4215046 osd.8.0:393237 dirty|data_digest|omap_digest s 4194304
uv 7633387 dd 41a45f29 od ffffffff alloc_hint [0 0 0]) exists: 1 ssc: 0x564783df4160 snapset:
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
2020-02-03 15:09:35.646359 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 HIT
2020-02-03 15:09:35.646469 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] execute_ctx 0x5647843a3000
2020-02-03 15:09:35.646494 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] execute_ctx 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get max 8388608] ov 58682'4215046
2020-02-03 15:09:35.646513 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] taking ondisk_read_lock
2020-02-03 15:09:35.646530 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_osd_op 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get max 8388608]
2020-02-03 15:09:35.646547 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_osd_op copy-get max 8388608
2020-02-03 15:09:35.681653 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] dropping ondisk_read_lock
2020-02-03 15:09:35.681780 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d5b80 finish
2020-02-03 15:09:35.681817 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d5d40 prio 63 cost 29 latency 22.306952
osd_op(osd.9.59923:2779948 2.b 2.162d8f8b (undecoded) ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected
e59939) v8 pg pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227] local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f
59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227 lcod 0'0 mlcod 0'0 active+clean]
2020-02-03 15:09:35.681851 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] _handle_message: 0x5647873d5d40
2020-02-03 15:09:35.681898 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_op osd_op(osd.9.59923:2779948 2.b 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get
max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e59939) v8 may_read ->
write-ordered flags ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected
2020-02-03 15:09:35.681937 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
map_snapid_to_clone=true
2020-02-03 15:09:35.682035 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
maps to 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3
2020-02-03 15:09:35.682128 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] get_object_context: found obc in cache: 0x564785fb3880
2020-02-03 15:09:35.682144 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] get_object_context: 0x564785fb3880 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 rwstate(none
n=0 w=0) oi: 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3(58682'4215046 osd.8.0:393237 dirty|data_digest|omap_digest s 4194304
uv 7633387 dd 41a45f29 od ffffffff alloc_hint [0 0 0]) exists: 1 ssc: 0x564783df4160 snapset:
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
2020-02-03 15:09:35.682245 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] find_object_context 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 @7e3 snapset
c07=[c07,c04,c01,bfe,bf9,bf6,bf3,bf0,bed,bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab,ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d,957,951,94b,945,93f,939,933,92d,927,921,91b,915,90f,909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f,849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9,7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff,9]:{11=[9],7e3=[7e3,7dd,7d7,7d1,7cb,7c5,7bf,7b9,7b3,7ad,7a7,7a1,79b,795,78f,789,783,77d,777,771,76b,765,75f,759,753,74d,747,741,73b,735,72f,729,723,71d,717,711,70b,705,6ff],849=[849,843,83d,837,831,82b,825,81f,819,813,80d,807,801,7fb,7f5,7ef,7e9],909=[909,903,8fd,8f7,8f1,8eb,8e5,8df,8d9,8d3,8cd,8c7,8c1,8bb,8b5,8af,8a9,8a3,89d,897,891,88b,885,87f,879,873,86d,867,861,85b,855,84f],915=[915,90f],957=[957,951,94b,945,93f,939,933,92d,927,921,91b],ba5=[ba5,b9f,b99,b93,b8d,b87,b81,b7b,b75,b6f,b69,b63,b5d,b57,b51,b4b,b45,b3f,b39,b33,b2d,b27,b21,b1b,b15,b0d,b07,b01,afb,af5,aef,ae9,ae3,add,ad7,ad1,acb,ac5,abf,ab9,ab3,aad,aa7,aa1,a9b,a95,a8f,a89,a83,a7d,a77,a71,a6b,a65,a5f,a59,a53,a4d,a47,a41,a3b,a35,a2f,a29,a23,a1d,a17,a11,a0b,a05,9ff,9f9,9f3,9ed,9e7,9e1,9db,9d5,9cf,9c9,9c3,9bd,9b7,9b1,9ab,9a5,99f,999,993,98d,987,981,97b,975,96f,969,963,95d],bea=[bea,be7,be4,be1,bde,bdb,bd6,bd3,bd0,bcd,bca,bc7,bc4,bc1,bbe,bbb,bb8,bb5,bab],bf9=[bf9,bf6,bf3,bf0,bed],c07=[c07,c04,c01,bfe]}
2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 HIT
2020-02-03 15:09:35.682356 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] execute_ctx 0x5647843a3000
2020-02-03 15:09:35.682385 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] execute_ctx 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get max 8388608] ov 58682'4215046
2020-02-03 15:09:35.682404 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] taking ondisk_read_lock
2020-02-03 15:09:35.682420 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_osd_op 2:d1f1b468:::rbd_data.9e87fa74b0dc51.000000000001d1a9:7e3 [copy-get max 8388608]
2020-02-03 15:09:35.682437 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] do_osd_op copy-get max 8388608
2020-02-03 15:09:35.714076 7f51e0acb700 10 osd.8 pg_epoch: 59939 pg[2.b( v 59908'4218227 (58731'4216723,59908'4218227]
local-lis/les=59932/59934 n=4693 ec=1/1 lis/c 59932/59932 les/c/f 59934/59934/38237 59932/59932/59932) [8,0] r=0 lpr=59932 crt=59908'4218227
lcod 0'0 mlcod 0'0 active+clean] dropping ondisk_read_lock
2020-02-03 15:09:35.714196 7f51e0acb700 10 osd.8 59939 dequeue_op 0x5647873d5d40 finish
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
