Hello Domingos, thank you very much for this in depth answer. It helped me to better understand NFS in general. Should that probably be documented in the apache manual (or is it already?) Regards Markus On Fri, 2006-07-28 at 14:19 -0300, Domingos Parra Novo wrote: > Hiyas, > > Markus Schiltknecht escreveu: > > Hi, > > > > We are puzzled by an apache2 hanging in an 'Uninterrubtable sleep'. The > > error.log tells us about a 'Stale NFS handle'. Unfortunately the log > > does not say which file it tried to access, only which request it was > > trying to serve. > > When reading content from a NFS filesystem, most operational systems > try to "cache" the data just read for an amount of time. On Linux, for > example, each time you read a block/inode from a NFS mountpoint, it will > cache its content for 30 seconds (value is aproximate, I'm not sure of > the exact value). > > Usually, apache will use this "inode cache", when reading data from > NFS. Now, lets say an apache "child" reads a file for the first time > (one request is made to the NFS server). During 30 seconds, any > subsequent read from this child to the same file will use the "inode > cache" (which means the NFS server will not be contacted in any way). > After this period, the cache is discarted, and the child will try to > read that same inode again. If something changed/deleted this inode in > the mean time, the SO will give a 'Stale NFS handle' error to the child > (after all, it tried to access an invalid file handler). > > > I've already tried with hard/soft mounted NFS, I tried disabling MMAP as > > well as Sendfile. None of that helped. The LockFile remains local > > on /var/lock... Further googling did not reveal other hints. > > In any case, you should always disable Sendfile and MMAP when serving > content from a NFS mounted filesystem. Also, your lockfile should always > point to a local filesystem (/var/lock usually is a good choice), and > the same rule is valid for your scoreboard file (either you point it to > a local file, or you comment the "ScoreBoardFile" keyword, which will > force apache to use a shared memory segment for the scoreboard). > > > We are on debian testing, linux kernel 2.6.11-1-k7, apache 2.0.55-4 > > (prefork) with mod_php 4.4.2-1.1. We are working on the php files, which > > might therefore be edited or (re)moved under apache, but normally that > > doesn't pose any problems, does it? > > Yes, it usually does. You gotta be really careful when modifying files > on an NFS partition, if something else is reading the same data from it > (your apache server, in this case). > > There is no real solution for this issue. The basic problem lies in the > fact that you "reuse" the inode, when modifying a file. You can try one > of the following: > > - If you're serving just "static" content (html, images and the such), > try to always create new files, with new names. Do not try to reuse > filenames at all, if possible; > > - If you're serving dynamic content (and in this case, modifying PHP > scripts), you must somehow modify the way you "publish" your content on > your NFS filesystem. You must always try to use "atomic" operations. > That could be done in the following way: > 1 - Lets say you're modifying a file called login.php. First, you copy > it to a new name (which in turn will create a new inode). For example, > ".new.login.php.<pid>.<timestamp>"; > 2 - Then, you modify the file as usual, but using the new file (the old > file is still valid); > 3 - After you finished modifying the new file, you must find a way to > not "invalidate" any inode in the "move" process. This is done in 3 steps: > 3.1 - Create a hardlink of the old file to a temporary directory (temp > dir must be on the same NFS partition). Something like that: "ln -f > login.php temp/login.php.<pid>.<timestamp>" > 3.2 - Overwrite the old file with the new file, using an atomic > operation. This is done like that: "ln -f > .new.login.php.<pid>.<timestamp> login.php" > 3.3 - After a few minutes (you _must_ wait at least 30 seconds), remove > the temporary file "temp/login.php.<pid>.<timestamp>". It is safer to > use a tool like "tmpwatch" (via cron) to clean files automatically in > the "temp" directory (for example, all files created after 1 hour). > > I know this is not simple to understand, and most important, it is > really hard to implement. Its easier if you have your own system do > publish content, but could be done too if you use samba or ftp to > publish your files (look in the configuration files for those two > daemons, and see if you find a way to create temp files on the upload > process, and then move the temp file into the old one). > > > Help greatly appreciated. > > > > Regards > > Regards, > > > Markus > > Domingos. > > --------------------------------------------------------------------- > The official User-To-User support forum of the Apache HTTP Server Project. > See <URL:http://httpd.apache.org/userslist.html> for more info. > To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx > " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx > For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx > --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|