On 2/25/21 9:00 PM, Norman.Kern wrote: > I made a test: BTW, what is the version of your kernel, and your bcache-tool, and which distribution is running ? > > - Stop writing and wait for dirty data writen back > > $ lsblk > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT > sdf 8:80 0 7.3T 0 disk > └─bcache0 252:0 0 7.3T 0 disk > └─ceph--32a481f9--313c--417e--aaf7--bdd74515fd86-osd--data--2f670929--3c8a--45dd--bcef--c60ce3ee08e1 253:1 0 7.3T 0 lvm > sdd 8:48 0 7.3T 0 disk > sdb 8:16 0 7.3T 0 disk > sdk 8:160 0 893.8G 0 disk > └─bcache0 252:0 0 7.3T 0 disk > └─ceph--32a481f9--313c--417e--aaf7--bdd74515fd86-osd--data--2f670929--3c8a--45dd--bcef--c60ce3ee08e1 253:1 0 7.3T 0 lvm > $ cat /sys/block/bcache0/bcache/dirty_data > 0.0k > > root@WXS0106:~# bcache-super-show /dev/sdf > sb.magic ok > sb.first_sector 8 [match] > sb.csum 71DA9CA968B4A625 [match] > sb.version 1 [backing device] > > dev.label (empty) > dev.uuid d07dc435-129d-477d-8378-a6af75199852 > dev.sectors_per_block 8 > dev.sectors_per_bucket 1024 > dev.data.first_sector 16 > dev.data.cache_mode 1 [writeback] > dev.data.cache_state 1 [clean] > cset.uuid d87713c6-2e76-4a09-8517-d48306468659 > > - check the available cache > > # cat /sys/fs/bcache/d87713c6-2e76-4a09-8517-d48306468659/cache_available_percent > 27 > What is the content from /sys/fs/bcache/<cache-set-uuid>/cache0/priority_stats ? Can you past here too. There is no dirty blocks, but cache is occupied 78% buckets, if you are using 5.8+ kernel, then a gc is probably desired. You may try to trigger a gc by writing to sys/fs/bcache/<cache-set-uuid>/internal/trigger_gc > > As the doc described: > > cache_available_percent > Percentage of cache device which doesn’t contain dirty data, and could potentially be used for writeback. This doesn’t mean this space isn’t used for clean cached data; the unused statistic (in priority_stats) is typically much lower. > When all dirty data writen back, why cache_available_percent was not 100? > > And when I start the write I/O, the new writen didn't replace the clean cache(it think the cache is diry now?), so it cause the hdd with large latency: > > ./bin/iosnoop -Q -d '8,80' > > <...> 73338 WS 8,80 3513701472 4096 217.69 > <...> 73338 WS 8,80 3513759360 4096 448.80 > <...> 73338 WS 8,80 3562211912 4096 511.69 > <...> 73335 WS 8,80 3562212528 4096 505.08 > <...> 73339 WS 8,80 3562213376 4096 501.19 > <...> 73336 WS 8,80 3562213992 4096 511.16 > <...> 73343 WS 8,80 3562214016 4096 511.74 > <...> 73340 WS 8,80 3562214128 4096 512.95 > <...> 73329 WS 8,80 3562214208 4096 510.48 > <...> 73338 WS 8,80 3562214600 4096 518.64 > <...> 73341 WS 8,80 3562214632 4096 519.09 > <...> 73342 WS 8,80 3562214664 4096 518.28 > <...> 73336 WS 8,80 3562214688 4096 519.27 > <...> 73343 WS 8,80 3562214736 4096 528.31 > <...> 73339 WS 8,80 3562214784 4096 530.13 > I just wondering why gc thread does not run up .... Thanks. Coly Li
