PS You also need a xlators/encryption//Makefile.am for autogen.sh to
succeed.
Thanks,
Brent
On Fri, 6 Apr 2007, Brent A Nelson wrote:
glusterfsd dies on both nodes almost immediately (I can ls succesfully once
before it dies, but cd in and they're dead). The glusterfs processes are
still running, but I of course have "Transport endpoint is not connected."
Also, glusterfsd and glusterfs no longer seem to know where to log by default
and refuse to start unless I give the -l option on each.
Thanks,
Brent
On Fri, 6 Apr 2007, Anand Avati wrote:
Brent,
the fix has been committed. can you please check if it works for you?
regards,
avati
On Thu, Apr 05, 2007 at 02:09:26AM -0400, Brent A Nelson wrote:
That's correct. I had commented out unify when narrowing down the mtime
bug (which turned out to be writebehind) and then decided I had no reason
to put it back in for this two-brick filesystem. It was mounted without
unify when this issue occurred.
Thanks,
Brent
On Wed, 4 Apr 2007, Anand Avati wrote:
Can you confirm that you were NOT using unify int he setup??
regards,
avati
On Thu, Apr 05, 2007 at 01:09:16AM -0400, Brent A Nelson wrote:
Awesome!
Thanks,
Brent
On Wed, 4 Apr 2007, Anand Avati wrote:
Brent,
thank you so much for your efforts of sending the output!
from the log it is clear the leak fd's are all for directories. Indeed
there was an issue with releasedir() call reaching all the nodes. The
fix should be committed today to tla.
Thanks!!
avati
On Wed, Apr 04, 2007 at 09:18:48PM -0400, Brent A Nelson wrote:
I avoided restarting, as this issue would take a while to reproduce.
jupiter01 and jupiter02 are mirrors of each other. All performance
translators are in use, except for writebehind (due to the mtime bug).
jupiter01:
ls -l /proc/26466/fd |wc
65536 655408 7358168
See attached for ls -l output.
jupiter02:
ls -l /proc/3651/fd |wc
ls -l /proc/3651/fd
total 11
lrwx------ 1 root root 64 2007-04-04 20:43 0 -> /dev/null
lrwx------ 1 root root 64 2007-04-04 20:43 1 -> /dev/null
lrwx------ 1 root root 64 2007-04-04 20:43 10 -> socket:[2565251]
lrwx------ 1 root root 64 2007-04-04 20:43 2 -> /dev/null
l-wx------ 1 root root 64 2007-04-04 20:43 3 ->
/var/log/glusterfs/glusterfsd.log
lrwx------ 1 root root 64 2007-04-04 20:43 4 -> socket:[2255275]
lrwx------ 1 root root 64 2007-04-04 20:43 5 -> socket:[2249710]
lr-x------ 1 root root 64 2007-04-04 20:43 6 -> eventpoll:[2249711]
lrwx------ 1 root root 64 2007-04-04 20:43 7 -> socket:[2255306]
lr-x------ 1 root root 64 2007-04-04 20:43 8 ->
/etc/glusterfs/glusterfs-client.vol
lr-x------ 1 root root 64 2007-04-04 20:43 9 ->
/etc/glusterfs/glusterfs-client.vol
Note that it looks like all those extra directories listed on
jupiter01
were locally rsynched from jupiter01's Lustre filesystems onto the
glusterfs client on jupiter01. A very large rsync from a different
machine to jupiter02 didn't go nuts.
Thanks,
Brent
On Wed, 4 Apr 2007, Anand Avati wrote:
Brent,
I hope the system is still in the same state to dig some info out.
To verify that it is a file descriptor leak, can you please run this
test. On the server, run ps ax and get the PID of glusterfsd. then do
an ls -l on /proc/<pid>/fd/ and please mail the output of that. That
should give a precise idea of what is happening.
If the system has been reset out of the state, please give us the
spec file you are using and the commands you ran (of some major jobs
like heavy rsync) so that we will try to reproduce the error in our
setup.
regards,
avati
On Wed, Apr 04, 2007 at 01:12:33PM -0400, Brent A Nelson wrote:
I put a 2-node GlusterFS mirror into use internally yesterday, as
GlusterFS was looking pretty solid, and I rsynced a whole bunch of
stuff
to it. Today, however, an ls on any of the three clients gives me:
ls: /backup: Too many open files
It looks like glusterfsd hit a limit. Is this a bug
(glusterfs/glusterfsd
forgetting to close files; essentially, a file descriptor leak), or
do I
just need to increase the limit somewhere?
Thanks,
Brent
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx
http://lists.nongnu.org/mailman/listinfo/gluster-devel
--
Shaw's Principle:
Build a system that even a fool can use,
and only a fool will want to use it.
--
Shaw's Principle:
Build a system that even a fool can use,
and only a fool will want to use it.
--
Shaw's Principle:
Build a system that even a fool can use,
and only a fool will want to use it.
--
Shaw's Principle:
Build a system that even a fool can use,
and only a fool will want to use it.
