Hi,
I have encountered an issue when using fscache where the Linux page
cache consumes memory on the system that then cannot be cleared. This
eventually leaves the system in a crippled swapping state.
To outline what I am doing:
[With a data set of ~35GB in ~2000 files]
1. Start four processes that each read the entire contents of a file
then move to the next etc.
2. After the first read from NFS we are constantly reading ~250MB/s from
the cache.
3. Very high system CPU utilisation is observed on (kswapd0 & flush-0).
4. After ~30min we find that part of the page cache can't be cleared (by
echo 3 > /proc/sys/vm/drop_caches)
5. Eventually all RAM becomes consumed and the system starts swapping
heavily.
6. The four reading processes are terminated.
7. The page cache still will not clear, even after unmounting the NFS
filesystem and restarting cachefilesd.
Many messages of this form are seen in /var/log/messages, but not sure
if this is connected:
FS-Cache: Assertion failed
1 == 0 is false
FS-Cache: Assertion failed
0 >= 1 is false
When performing the same test without fscache this does not occur,
however the test without fscache never read anything like the quantity
of data from the real NFS server as from the cache when testing with
fscache.
Sorry I don't have much data to provide, but I hope this will still be
of some use.
thanks,
Stef
Kernel: 2.6.39.4 (x86_64)
RAM: 4GB
CPUs: 4
cachefilesd: bdec8cc (Bump the revision to 0.10.2)
Kernel / fscache:
4dc2606 3.0.3 64-bit Crash running fscache/cachefilesd
3b27dea NFS: Use FS-Cache invalidation
7438164 CacheFiles: Implement invalidation
9f27574 VFS: Make more complete truncate operation available to CacheFiles
7b4919b FS-Cache: Provide proper invalidation
5dacc0f FS-Cache: Fix operation state management and accounting
b83629c fscache: remove dead code under CONFIG_WORKQUEUE_DEBUGFS
e3249f6 FS-Cache: Make cookie relinquishment wait for outstanding reads
0e2f15f CacheFiles: Make some debugging statements conditional
c2934b8 FS-Cache: Check cookie is still correct in
__fscache_read_or_alloc_pages()
94ed160 FS-Cache: Check that there are no read ops when cookie relinquished
7f29d79 CacheFiles: Downgrade the requirements passed to the allocator
f6fb3c4 FS-Cache: Validate page mapping pointer value
1999e47 CacheFiles: Fix the marking of cached pages
c540312 Noisefs: A predictable noise producing fs for testing things
e0b2ac6 ALSA: snd_usb_caiaq: track submitted output urbs
fa01632 ALSA: snd-usb-caiaq: Correct offset fields of outbound
iso_frame_desc
78e0e10 ALSA: snd-usb-caiaq: Fix keymap for RigKontrol3
ea0dc0d Linux 2.6.39.4
--
Linux-cachefs mailing list
Linux-cachefs@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cachefs
