Hiyas, Markus Schiltknecht escreveu:
Hi, We are puzzled by an apache2 hanging in an 'Uninterrubtable sleep'. The error.log tells us about a 'Stale NFS handle'. Unfortunately the log does not say which file it tried to access, only which request it was trying to serve.
When reading content from a NFS filesystem, most operational systems try to "cache" the data just read for an amount of time. On Linux, for example, each time you read a block/inode from a NFS mountpoint, it will cache its content for 30 seconds (value is aproximate, I'm not sure of the exact value).
Usually, apache will use this "inode cache", when reading data from NFS. Now, lets say an apache "child" reads a file for the first time (one request is made to the NFS server). During 30 seconds, any subsequent read from this child to the same file will use the "inode cache" (which means the NFS server will not be contacted in any way). After this period, the cache is discarted, and the child will try to read that same inode again. If something changed/deleted this inode in the mean time, the SO will give a 'Stale NFS handle' error to the child (after all, it tried to access an invalid file handler).
I've already tried with hard/soft mounted NFS, I tried disabling MMAP as well as Sendfile. None of that helped. The LockFile remains local on /var/lock... Further googling did not reveal other hints.
In any case, you should always disable Sendfile and MMAP when serving content from a NFS mounted filesystem. Also, your lockfile should always point to a local filesystem (/var/lock usually is a good choice), and the same rule is valid for your scoreboard file (either you point it to a local file, or you comment the "ScoreBoardFile" keyword, which will force apache to use a shared memory segment for the scoreboard).
We are on debian testing, linux kernel 2.6.11-1-k7, apache 2.0.55-4 (prefork) with mod_php 4.4.2-1.1. We are working on the php files, which might therefore be edited or (re)moved under apache, but normally that doesn't pose any problems, does it?
Yes, it usually does. You gotta be really careful when modifying files on an NFS partition, if something else is reading the same data from it (your apache server, in this case).
There is no real solution for this issue. The basic problem lies in the fact that you "reuse" the inode, when modifying a file. You can try one of the following:
- If you're serving just "static" content (html, images and the such), try to always create new files, with new names. Do not try to reuse filenames at all, if possible;
- If you're serving dynamic content (and in this case, modifying PHP scripts), you must somehow modify the way you "publish" your content on your NFS filesystem. You must always try to use "atomic" operations. That could be done in the following way: 1 - Lets say you're modifying a file called login.php. First, you copy it to a new name (which in turn will create a new inode). For example, ".new.login.php.<pid>.<timestamp>"; 2 - Then, you modify the file as usual, but using the new file (the old file is still valid); 3 - After you finished modifying the new file, you must find a way to not "invalidate" any inode in the "move" process. This is done in 3 steps: 3.1 - Create a hardlink of the old file to a temporary directory (temp dir must be on the same NFS partition). Something like that: "ln -f login.php temp/login.php.<pid>.<timestamp>" 3.2 - Overwrite the old file with the new file, using an atomic operation. This is done like that: "ln -f .new.login.php.<pid>.<timestamp> login.php" 3.3 - After a few minutes (you _must_ wait at least 30 seconds), remove the temporary file "temp/login.php.<pid>.<timestamp>". It is safer to use a tool like "tmpwatch" (via cron) to clean files automatically in the "temp" directory (for example, all files created after 1 hour).
I know this is not simple to understand, and most important, it is really hard to implement. Its easier if you have your own system do publish content, but could be done too if you use samba or ftp to publish your files (look in the configuration files for those two daemons, and see if you find a way to create temp files on the upload process, and then move the temp file into the old one).
Help greatly appreciated. Regards
Regards,
Markus
Domingos. --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|