Hi Joshua-- On Tue, Jun 06, 2006 at 11:12:17AM -0400, Joshua Slive wrote: > On 6/6/06, John Morrissey <jwm@xxxxxxxxx> wrote: > > I'm having a problem with Apache 2.0.54 (we're running the Debian > > packages for 2.0.54-5). Periodically, a single Apache child will rapidly > > consume all available memory on the machine - once the memory > > consumption starts, it only takes five or ten minutes (at most) for it > > to exhaust available memory. > > Change your LogFormat to include the PID of the serving process. Then > you can go back and correlate teh PID of the memory hog with the > requests it recently served. (See also mod_log_forensic, which will > be necessary if the bad request never finishes.) Thanks for your help. I've tried both, and it seems the errant children are only serving static content and mod_rewrite-generated redirects (returning 302s). The forensic logs back this up; there are only a handful of uncompleted requests each day, all for static content. Interestingly, it seems some children continue to serve requests while leaking huge amounts of RAM. Here's a recent example. Our errant process killer activated at 23:36:24. sar(1) says: time kbramused %swpused 22:45:04 645516 0.48 22:55:05 2010532 0.48 23:05:04 2025148 50.44 23:15:02 2025696 55.02 23:36:26 769572 3.58 So sometime between 22:45 and 22:55, this child began leaking. The access logs indicate this child continued to serve static content (except for one exception, see below) during this period, ~375 requests more or less evenly spaced over time. The only exception to this even spacing is thus: at 23:25:41, it served a redirect, then at 23:36:20 apparently served its last request (484 bytes of static HTML) before being killed. I suspect this gap is because the machine became so RAM-tight that it was swapping heavily. The only dynamic content served by this child during this time period is a call to wwwcount (http://www.muquit.com/muquit/software/Count/Count.html) at 22:46:46, but I'm inclined to discount this as coincidence since the same child served identical wwwcount requests at 22:25:35, 22:23:52, 22:03:40, 22:01:21 (... ad infinitum). Also, wwwcount requests account for a nontrivial amount of these machines' traffic, but we're only seeing a couple errant processes a day, at most. I'm at a loss. FWIW, here's a link to my original post, since it's been about a month: http://marc.theaimsgroup.com/?l=apache-httpd-users&m=114960657316006&w=2 Any other ideas, or things I can try? thanks, john -- John Morrissey _o /\ ---- __o jwm@xxxxxxxxx _-< \_ / \ ---- < \, www.horde.net/ __(_)/_(_)________/ \_______(_) /_(_)__ --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|