Hello,
I've tried using bcache with a large 6.4TB NVMe device, but found it
takes long time to register after clean reboot -- around 10 minutes.
That's even with idle machine reboot.
Things look like this soon after reboot:
root@node420:~# ps axuww |grep md12
root 9768 88.1 0.0 2268 744 pts/0 D+ 16:20 0:25
/lib/udev/bcache-register /dev/md12
Device r/s w/s rMB/s wMB/s rrqm/s wrqm/s
%rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
nvme0n1 420.00 0.00 52.50 0.00 0.00 0.00 0.00
0.00 0.30 0.00 1.04 128.00 0.00 2.38 99.87
nvme1n1 417.67 0.00 52.21 0.00 0.00 0.00 0.00
0.00 0.30 0.00 1.03 128.00 0.00 2.39 100.00
md12 838.00 0.00 104.75 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 128.00 0.00 0.00 0.00
As you can see nvme1n1, which is Micron 9200, is reading with the humble
52MB/s (417r/s), and that is very far bellow it's capabilities of
3500MB/s & 840K IOPS.
At the same time it seems like the bcache-register process is saturating
the CPU core it's running on, so maybe that's the bottleneck?
Tested with kernels 4.9 and 4.19.
1. Is this current state of affairs -- i.e. this known/expected
behaviour with such a large cache?
2. If this isn't expected -- any ideas how to debug or fix it?
3. What is max recommended cache size?
--
Best regards,
Teodor Milkov
