Hi linux-bcache,
I was evaluating the performance and stability of bcache.
In my test environment, I was using bcache as a read cache in
writearound cache mode running on OS ubuntu 20.04 with linux kernel
5.15.
However, I found that sometimes the data was inconsistent, and here is
how to reproduce:
1. Set up bcache with 1 hard drive and 1 read cache
$ make-bcache -B /dev/sdb -C /dev/sdc
2. Set as writearound mode
$ echo "writearound" > /sys/block/bcache0/bcache/cache_mode
3. Create XFS on /dev/bcache0
$ mkfs.xfs -m reflink=0 -f /dev/bcache0
4. Mount the file system
$ mkdir /srv/node/ -p
$ mount -t xfs /dev/bcache0 /srv/node -o
rw,nofail,noexec,nodev,noatime,nodiratime,inode64,logbsize=256k
5. Prepare a 50G file with random content
$ head -c 50G < /dev/urandom > 50G
6. Get md5sum of this 50G file
$ md5sum 50G
99c91956af4eec68707b329902062bfd
7. Copy the 50G file to the mount point
$ cp 50G /srv/node/
8. During the copy operation, remove the cache device manually
$ echo 1 > /sys/block/sdc/device/delete
9. Wait for the copy operation done, then calculate md5sum of 50G file
in the destination
$ md5sum /srv/node/50G
bb178bbb9c0ca07c202580e3ea9e62c5
The result shows the md5sum of two 50G file are not the same.
As far as I know, no data is written to cache device in writearound
mode. Therefore a failed cache device should not cause any data
inconsistency.
There are some further observations,
1. I can reproduce it with kernel 5.04 / 5.14.
2. I can reproduce it with ubuntu 22.04.
3. I cannot reproduce if the file is not large enough, like 10G and 20G.
Please advise me how to fix it. Thank you.
Sincerely,
Norris Hsu
