On March 11, 2011 13:48 , Brian Hirt <bhirt@xxxxxx> wrote:
We have a bunch of rewrite rules that use four standard plain text rewrite maps (ie RewriteMap mymap txt:/some/file). These rewrite maps only change about once a month, but when they do change, apache picks up the chances pretty much immediately. I'm assuming that apache is doing a stat on the file every so often but I'm not really sure how ofter. Does it do it per request or does it limit it to a time frame like once per second?
This question could be answered by using a tool like strace, truss, DTrace, or SystemTap (depending on which OS you're using). However, looking in the source code for Apache HTTP Server 2.2.17, modules/mapper/mod_rewrite.c:
hook_fixup() and hook_uri2file handle rewriting in directory and server contexts, respectively. Both call apply_rewrite_list(), which in turn calls apply_rewrite_rule(), which calls do_expand(), which calls lookup_map(), which calls apr_stat() if the map type is txt or rnd. The file modification time from the apr_stat() call is then passed to get_cache_value(), which will fail -- forcing lookup_map() to re-read the map file -- if the modification time is later than the modification time associated with the cached map entry (or if the map entry does not exist in the cache). Both positive and negative map results are cached. See lines 1539 - 1572 of mod_rewrite.c.
So, the answer to your question is: a stat() is done for each request for which there is a RewriteRule in the applicable context whose substitution contains a map expression AND whose pattern matches the current request. If the rewrite rule does not apply to or does not match the current request, then no stat() will be done.
What happens if these maps are on a shared NFS filesystem. Will apache be able to pick up changes made to the file?
I know of no reason why httpd would not pick up the changes in this situation, assuming the NFS client sees changes on the NFS server immediately.
Will i start getting 100s of stats per second to the NFS filesystem?
If you have 100s of requests per second that match rewrite rules that apply maps, then yes. You might want to look into how your NFS client handles stats in this context: does each stat() call result in one or more RPC calls to the NFS server, or does the NFS client cache and/or optimize things in some way?
Do you have httpd configured to serve content from your NFS filesystem? If so, you're probably already doing multiple stat() calls per request, and even more if you have AllowOverride set for those paths, or if don't have FollowSymlinks set. ( See http://httpd.apache.org/docs/2.2/misc/perf-tuning.html ) In other words, before assuming that 100s of stat() per second to the NFS filesystem for rewrite maps will be a problem, you should see how many stat() per second you're already doing.
If you determine it is worth optimizing around using NFS to store rewrite maps, set up a cron job that runs once a minute on each machine in your cluster that checks to see if each rewrite map in NFS is newer than a copy on the web server's local filesystem and, if it is, copies it from NFS to the local disk. Then configure httpd to use the copies on local disk.
I hope this helps. -- Mark Montague mark@xxxxxxxxxxx --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|