Hi guys,
I am testing ceph with bcache, I found some I/O with O_SYNC writeback to
HDD, which caused large latency on HDD, I trace the I/O with iosnoop:
./iosnoop -Q -ts -d '8,192
Tracing block I/O for 1 seconds (buffered)...
STARTs ENDs COMM PID TYPE DEV
BLOCK BYTES LATms
1809296.292350 1809296.319052 tp_osd_tp 22191 R 8,192
4578940240 16384 26.70
1809296.292330 1809296.320974 tp_osd_tp 22191 R 8,192
4577938704 16384 28.64
1809296.292614 1809296.323292 tp_osd_tp 22191 R 8,192
4600404304 16384 30.68
1809296.292353 1809296.325300 tp_osd_tp 22191 R 8,192
4578343088 16384 32.95
1809296.292340 1809296.328013 tp_osd_tp 22191 R 8,192
4578055472 16384 35.67
1809296.292606 1809296.330518 tp_osd_tp 22191 R 8,192
4578581648 16384 37.91
1809295.169266 1809296.334041 bstore_kv_fi 17266 WS 8,192
4244996360 4096 1164.78
1809296.292618 1809296.336349 tp_osd_tp 22191 R 8,192
4602631760 16384 43.73
1809296.292618 1809296.338812 tp_osd_tp 22191 R 8,192
4602632976 16384 46.19
1809296.030103 1809296.342780 tp_osd_tp 22180 WS 8,192
4741276048 131072 312.68
1809296.292347 1809296.345045 tp_osd_tp 22191 R 8,192
4609037872 16384 52.70
1809296.292620 1809296.345109 tp_osd_tp 22191 R 8,192
4609037904 16384 52.49
1809296.292612 1809296.347251 tp_osd_tp 22191 R 8,192
4578937616 16384 54.64
1809296.292621 1809296.351136 tp_osd_tp 22191 R 8,192
4612654992 16384 58.51
1809296.292341 1809296.353428 tp_osd_tp 22191 R 8,192
4578220656 16384 61.09
1809296.292342 1809296.353864 tp_osd_tp 22191 R 8,192
4578220880 16384 61.52
1809295.167650 1809296.358510 bstore_kv_fi 17266 WS 8,192
4923695960 4096 1190.86
1809296.292347 1809296.361885 tp_osd_tp 22191 R 8,192
4607437136 16384 69.54
1809296.029363 1809296.367313 tp_osd_tp 22180 WS 8,192
4739824400 98304 337.95
1809296.292349 1809296.370245 tp_osd_tp 22191 R 8,192
4591379888 16384 77.90
1809296.292348 1809296.376273 tp_osd_tp 22191 R 8,192
4591289552 16384 83.92
1809296.292353 1809296.378659 tp_osd_tp 22191 R 8,192
4578248656 16384 86.31
1809296.292619 1809296.384835 tp_osd_tp 22191 R 8,192
4617494160 65536 92.22
1809295.165451 1809296.393715 bstore_kv_fi 17266 WS 8,192
1355703120 4096 1228.26
1809295.168595 1809296.401560 bstore_kv_fi 17266 WS 8,192
1122200 4096 1232.96
1809295.165221 1809296.408018 bstore_kv_fi 17266 WS 8,192
960656 4096 1242.80
1809295.166737 1809296.411505 bstore_kv_fi 17266 WS 8,192
57682504 4096 1244.77
1809296.292352 1809296.418123 tp_osd_tp 22191 R 8,192
4579459056 32768 125.77
I'm confused why write with O_SYNC must writeback on the backend storage
device? And when I used bcache for a time,
the latency increased a lot.(The SSD is not very busy), There's some
best practices on configuration?
