Hi folks,
For now most of the obvious bcache deadlock/race/panic bugs are fixed,
bcache may survive under heavy I/O loads for 12+ hours on my testing
machine (Lenovo SR650 with 96G memory, 16T NVMe ssd and 24x2 CPU cores).
Now new problem shows up. After running for such long time, the bcache
in-memory btree node cache occupies too many system memory (more than
32GB anonymous memory) and finally makes the whole system hang or panic.
On small or slow machine, it won't happen. But on fast and powerful
machine, when new I/O requests coming too fast, the new allocated in-
memory btree node cache is allocating faster than it is shrinking. After
continously running for 12+ hours for such high I/O loads, finally the
btree in-memory btree node cache eats up all system memory.
When such prolbem does not show up before Linux v5.3?
Because the bcache code could not survive for more than 40 minutes
before Linux v5.3 on my testing machine.
We need a method to limit memory consumption by the bcache in-memory
btree node cache. This patch set is an effort to limit the total size
of btree node cache memory and avoid further system hang or panic by
out of memory condition.
Any comments are welcome, and I am still testing the patches in the mean
time.
Thanks in advance.
Coly Li
---
Coly Li (7):
bcache: remove member accessed from struct btree
bcache: reap c->btree_cache_freeable from the tail in bch_mca_scan()
bcache: reap from tail of c->btree_cache in bch_mca_scan()
bcache: add __bch_mca_scan() with parameter "bool reap_flush"
bcache: limit bcache btree node cache memory consumption by I/O
throttle
bcache: remove unnecessary mca_cannibalize()
bcache: add cond_resched() in bch_btree_node_get() if mca_alloc()
fails
drivers/md/bcache/bcache.h | 3 ++
drivers/md/bcache/btree.c | 127 +++++++++++++++++++++++++++++----------------
drivers/md/bcache/btree.h | 5 +-
drivers/md/bcache/super.c | 3 ++
4 files changed, 91 insertions(+), 47 deletions(-)
--
2.16.4
