I have come across a fairly large memory leak(?) in glusterfsd using
the supplied configuration when doing many rewrites (read a block of
data, change it, write block back). This results in one of the
glusterfsd processes consuming very large amounts of the machines
memory (95% ish).
It seems to occur when readahead and writebehind are loaded above
Unify on the client, which connects to two remote AFR volumes.
This occurs in glusterfs--main--2.5--patch-616.
I have been able to replicate the problem on Debian Etch machines,
using a fuse-2.7.0 compiled from source and a recent kernel and the
following configurations.
The attached benchmark program can be used to cause the problem
rewrite_test --blocksize 8192 --count 8192 /mnt/foobar
should do it. Bonnie will also cause the problem when it does its rewrite test.
Sam Douglas
-- Config Files --
--- Client ---
volume clientA
type protocol/client
option transport-type tcp/client
option remote-host 10.0.0.19
option remote-subvolume afr
end-volume
volume clientB
type protocol/client
option transport-type tcp/client
option remote-host 10.0.0.20
option remote-subvolume afr
end-volume
volume namespace
type protocol/client
option transport-type tcp/client
option remote-host 10.0.0.19
option remote-subvolume namespace
end-volume
volume unify
type cluster/unify
option namespace namespace
option scheduler rr
subvolumes clientA clientB
end-volume
volume unify-wb
type performance/write-behind
option aggregate-size 128K
subvolumes unify
end-volume
volume unify-ra
type performance/read-ahead
option page-size 65536
option page-count 32
subvolumes unify-wb
end-volume
--- Server (10.0.0.19) ---
volume brick1
type storage/posix
option directory /stuff/test1
end-volume
volume brick2
type storage/posix
option directory /stuff/test2
end-volume
volume namespace
type storage/posix
option directory /stuff/test-namespace
end-volume
volume client1
type protocol/client
option transport-type tcp/client
option remote-host 10.0.0.20
option remote-subvolume brick1
end-volume
volume afr
type cluster/afr
subvolumes brick1 client1
end-volume
volume server
type protocol/server
option transport-type tcp/server
option auth.ip.afr.allow *
option auth.ip.brick1.allow *
option auth.ip.brick2.allow *
option auth.ip.namespace.allow *
subvolumes afr brick1 brick2 namespace
end-volume
--- Server (10.0.0.20) ---
volume brick1
type storage/posix
option directory /stuff/test1
end-volume
volume brick2
type storage/posix
option directory /stuff/test2
end-volume
volume client2
type protocol/client
option transport-type tcp/client
option remote-host 10.0.0.19
option remote-subvolume brick2
end-volume
volume afr
type cluster/afr
subvolumes client2 brick2
end-volume
volume server
type protocol/server
option transport-type tcp/server
option auth.ip.afr.allow *
option auth.ip.brick1.allow *
option auth.ip.brick2.allow *
subvolumes afr brick1 brick2
end-volume
