On Tue, 23 Aug 2011 14:38:49 +0900 (JST), Ryusuke Konishi
<konishi.ryusuke@xxxxxxxxxxxxx> wrote:
Hi,
On Wed, 10 Aug 2011 12:17:45 +0100, Gordan Bobic wrote:
Another performance related problem I am seeing due to
nilfs_cleanerd
is that it causes unhealthy amounts of cache churn. It's reads and
writes are buffered, which inevitably means that things it reads
will
get cached. Since it is going through all the blocks on the fs that
have
any garbage to collect, it will eat through all the available
memory
pretty quickly. It also means that it will push out of caches
things
that really should stay in caches.
Interesting report. nilfs_cleanerd only reads log header and does
not
read payload blocks. Data blocks are instead read and copied by the
nilfs kernel code, and they are freed every time reclamation call of
a
few segments has ended.
I guess the abnormal cache churn arose from other causes, seems that
DAT file access is suspicious. (The DAT file holds metadata used to
convert virtual block addresses to real disk block addresses).
Since cleanerd's actual disk I/O is going to have no correlation
with
actual file access pattern, is there a way to make cleanerd always
operate with something like the O_DIRECT flag so that is's reads
won't
fill up the page cache?
If the problem comes from internal metadata accesses like the DAT
file
access, O_DIRECT is not applicable.
This is a pretty serious problem on small machines running of cheap
flash (think ARM machines with 512MB of RAM and slow flash media).
The quick and dirty workaround I am pondering at the moment is to
set
up a cron job that runs once/minute, checks df, and starts/kills
nilfs_cleanerd depending on how much free space is available, but
that's
not really a solution.
Gordan
Does your kernel version equal to or newer than v2.6.37 ?
I am running 2.6.38.8 + chromos patches (running on Tegra2 ARM).
Last year, we changed cache usage for the DAT file on that kernel.
This might influence the issue.
I am running 2.0.23 nilfs-utils.
The cache churn issue is trivial to reproduce:
1) On an otherwise idle machine, set the thresholds appropriately to
make nilfs_cleanerd reclaim some space
2) echo 3 > /proc/sys/vm/drop_caches
3) Observe top and iotop to establish that:
- nilfs_cleanerd is the only thing running and doing anything
- cache memory is growing at the same rate at which iotop is saying
nilfs_cleanerd is doing I/O
Gordan
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
