Hi all,
I do hope my email gets through, I tried to subscribe yesterday but withno effect so far.
We are running a large cluster and do a lot of cross-mounting betweenthe nodes. To get this running we are running a lot of nfsd (196) anduse mountd with 64 threads, just in case we get a massive number of hitsonto a single node. All this is on Debian Etch with a recent 2.6.24kernel using autofs4 at the moment to do the automounts.
When running these two not nice scripts:
$ cat test_mount#!/bin/sh
n_node=1000
for i in `seq 1 $n_node`;do n=`echo $RANDOM%1342+10001 | bc| sed -e "s/1/n/"` $HOME/bin/mount.sh $n& echo -n .done
$ cat mount.sh#!/bin/sh
dir="/distributed/spray/data/EatH/S5R1"
ping -c1 -w1 $1 > /dev/null&& file="/atlas/node/$1$dir/"`ls -f/atlas/node/$1$dir/|head -n 50 | tail -n 1`md5sum ${file}
With that we encounter different problems:
Running this gives this in syslog:Jul 1 07:37:19 n1312 rpc.idmapd[2309]: nfsopen:open(/var/lib/nfs/rpc_pipefs/nfs/clntaa58/idmap): Too many open filesJul 1 07:37:19 n1312 rpc.idmapd[2309]: nfsopen:open(/var/lib/nfs/rpc_pipefs/nfs/clntaa58/idmap): Too many open filesJul 1 07:37:19 n1312 rpc.idmapd[2309]: nfsopen:open(/var/lib/nfs/rpc_pipefs/nfs/clntaa5e/idmap): Too many open filesJul 1 07:37:19 n1312 rpc.idmapd[2309]: nfsopen:open(/var/lib/nfs/rpc_pipefs/nfs/clntaa5e/idmap): Too many open filesJul 1 07:37:19 n1312 rpc.idmapd[2309]: nfsopen:open(/var/lib/nfs/rpc_pipefs/nfs/clntaa9c/idmap): Too many open files
Which is not surprising to me. However, there are a few things I'mwondering about.
(1) All our mounts use nfsvers=3 why is rpc.idmapd involved at all?(2) Why is this daemon growing so extremely large?# ps aux|grep rpc.idmapdroot 2309 0.1 16.2 2037152 1326944 ? Ss Jun30 1:24/usr/sbin/rpc.idmapd(3) The script maxes out at about 340 concurrent mounts, any idea how toincrease this number? We are already running all servers with theinsecure option, thus low ports should not be a restriction.(4) After running this script /etc/mtab and /proc/mounts are out ofsync. Ian Kent from autofs fame suggested a broken local mountimplementation which does not lock mtab well enough. Any idee about that?
TIA for any help you might give us.
Cheers
Carsten
-- Dr. Carsten Aulbert - Max Planck Institut für GravitationsphysikCallinstraße 38, 30167 Hannover, GermanyFon: +49 511 762 17185, Fax: +49 511 762 17193http://www.top500.org/system/9234 | http://www.top500.org/connfam/6/list/31
-------------------------------------------------------------------------Check out the new SourceForge.net Marketplace.It's the best place to buy or sell services forjust about anything Open Source.http://sourceforge.net/services/buy/index.php_______________________________________________NFS maillist - NFS@xxxxxxxxxxxxxxxxxxxxxxxxxx://lists.sourceforge.net/lists/listinfo/nfs_______________________________________________Please note that nfs@xxxxxxxxxxxxxxxxxxxxx is being discontinued.Please subscribe to linux-nfs@xxxxxxxxxxxxxxx instead. http://vger.kernel.org/vger-lists.html#linux-nfs��.n��������+%����;��w��{.n�����{��w��� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
