Hi Sander, Thanks for your very precise comments. Below are mine... > Chirouze, My actual firstname is Olivier :-) > > On Jan 26, 2007, at 8:41 AM, Chirouze Olivier wrote: > > > Thanks for your prompt reply. We will definitively upgrade > soon, just > > "to be up to date"... But because nothing is said about > that point in > > docs I'm wondering if that will make any difference... > > I doubt it. The Apache 2.0.x branch at this moment mainly gets > security fixes, and not a lot else. The difference between > subsequent versions tends to be fairly small, and we tend to > tell you > what the fixes are. > > You tell us that you are seeing an unusually high number of children > sitting in 'R' stage, which means they are attempting to read a full > request. Reading requests *should* not take very long (this also > depends on the kind of requests you expect to get), and it may be > that you are under attack. This may be deliberate or accidental. Yes, we think some were deliberate attacks. > > Do you have something like a Firewall in front of your server that > drops idle TCP connections? If this happens, your Apache > server will > not notice this and keep trying to read the incoming data. > Thanks for pointing to this. This exists, but the timeout is approx. 1 hour... Much higher than the Unix timeout... > In any case, you may try to reduce the value of the Timeout > directive > in your configuration file. > > The high number of FIN_WAIT_2 connections you observe may be normal, > but it may be the result of the situation sketched above. Once > Apache is done with a connection, it'll close that and forget about > it. The Apache child will then get ready to accept a new > connection, > and the kernel takes care of closing the TCP connection. It does > this by sending a FIN packet to the client, who sends an ACK back. > Then the client is supposed to send its own FIN, and while this > happens the server will have that socket in FIN_WAIT_2 state. After > the client sends its FIN, the server ACKnowledges and puts > the socket > into TIME_WAIT before it can be re-used. You can see a full diagram > of this process in Figure 18.12 in TCP Illustrated, Volume 1 by W. > Richard Stevens. > > If you are dealing with clients that don't bother sending their FIN, > or have something in-between that messes with the connection and its > termination, you may be stuck with this. You can use kernel > tunables > like the ones you mention below to reduce the overhead. Thanks for the nice explanation... That confirms what I knew ;-) The thing is, why do I find the connections Apache describes as "reading" in a "FIN_WAIT_2" state in Unix? I have good hope that setting a lower value for FIN_WAIT timeout in Unix will fix the pb, but not explain this... > > Once again, Apache has nothing to do with the TCP handshake > process. > However, you may be able to reduce the effect of these 'R' > connections by reducing the timeout and raising the number of > available child processes (if your log file tells you that > the server > reached MaxClients). What kind of timeout are you talking about here? About the MaxClients, we did reach it but when 40 concurrent requests over 150 are "reading", I'm feeling like the pb is on these connections, not on the maximum... Would anybody have an idea of a "reasonable value"? I've heard that 150 is already a pretty big value... Thanks again! Olivier > > Regards, > > Sander > > > Anyway, we're providing both HTTP and HTTPS. Might be interesting > > to try > > recognize if this happens on both? I will have a look at it. > > > > Do you think you might give me the values of the following > Unix params > > on your Solaris 9 installs? > > > > tcp_fin_wait_2_flush_interval > > tcp_keepalive_interval > > tcp_ip_abort_interval > > > > Thanks in advance, > > > > Olivier > > > > Olivier CHIROUZE > > I&0 Infrastructure > > Volvo Information Technology > > > > -----Original Message----- > > From: Richard de Vries [mailto:richard_devries@xxxxxxxxx] > > Sent: 26 January 2007 17:35 > > To: users@xxxxxxxxxxxxxxxx > > Subject: Re: Apache 2.0.58 + Solaris 5.9: status > > "...reading..." & TCP state "FIN_WAIT_2" > > > > Interesting problem. > > > > I am running Apache 2.0.59 as a reverse proxy on multiple > Solaris 9 > > and > > AIX servers and have never encountered these types of > issues. Perhaps > > you should try upgrading to 2.0.59 on one of your > development machines > > and see if that makes a difference. If not, it is most likely an OS > > and/or configuration issue. > > > > What other plugins are you running? Also, is this HTTP proxying, or > > HTTPS? > > > > ----- Original Message ---- > > From: Chirouze Olivier <olivier.chirouze@xxxxxxxxx> > > To: users@xxxxxxxxxxxxxxxx > > Sent: Friday, January 26, 2007 9:56:46 AM > > Subject: Apache 2.0.58 + Solaris 5.9: status > > "...reading..." & TCP state "FIN_WAIT_2" > > > > > > Hi all, > > > > I'm facing a quite tricky situation with Apache 2.0.58 running on > > Solaris 5.9. > > > > Apache is running as a reverse proxy (mod_proxy + mod_rewrite). > > The maximum concurrent connections is set to 150. > > > > Because we reached the maximum a few times and got the reverse proxy > > saturated, we started monitoring the Apache status page (/status). > > We noticed that many requests were in the "..reading.." state (up to > > 40!), and they block a lot of slots. > > > > At first, we upgraded from 2.0.47 to 2.0.58 because it > seemed there > > was > > a security hole in the earlier, fixed in 2.0.48. > > I found some explanation here: > > http://www.monkeybrains.net/~rudy/example/server_busy_state.html. > > > > The thing is, the situation is starting to appear again with 2.0.58. > > > > We've gone down to Unix and found that most of these > requests were in > > "FIN_WAIT_2" TCP state, and for a while (approx. 8min!!). > > > > We found this: > http://httpd.apache.org/docs/2.0/misc/fin_wait_2.html. > > What it says, in a word, is that these things can happen and are > > "normal": the connection stays in "FIN_WAIT_2" state until the > > timeout, > > if clients do not close it properly. They just say it can > be a problem > > on the Unix point of view because. > > I don't know if this is still true for 2.0 because the article was > > just > > copied from 1.3. The thing is, it says that "The connections in > > FIN_WAIT_2 do not tie up an httpd process". For us, IT DOES! Every > > "..reading.." request happend to be in the "FIN_WAIT_2" state. > > > > We have contacted Sun to get their opinion. The short answer is > > "you can > > change the FIN_WAIT_2 timeout but be careful because wrong > tuning will > > have negative impact. Maybe you should wonder why these > connections > > stay > > alive". As far as I understood, the connection is not closed by the > > client. The server (Apache) does nothing wrong. But maybe it does, > > as it > > doesn't leave the process free? > > > > My questions are: > > Does anyone have heard about similar problems? > > Why do these connections hold a process of Apache while the > > documentation says it doesn't? > > Do you recon tuning the Unix timeout would help? (current value of > > tcp_fin_wait_2_flush_interval: 675000 ms - 11min!! This looks just > > huge!) > > > > Thanks in advance, > > > > Olivier > > > > Olivier CHIROUZE > > I&0 Infrastructure > > > > Volvo Information Technology > > > > > > > > > > > --------------------------------------------------------------------- > > The official User-To-User support forum of the Apache HTTP Server > > Project. > > See <URL:http://httpd.apache.org/userslist.html> for more info. > > To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx > > " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx > > For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx > > > > > > > > > ______________________________________________________________ > ________ > > __ > > ____________ > > Do you Yahoo!? > > Everyone is raving about the all-new Yahoo! Mail beta. > > http://new.mail.yahoo.com > > > > > --------------------------------------------------------------------- > > The official User-To-User support forum of the Apache HTTP Server > > Project. > > See <URL:http://httpd.apache.org/userslist.html> for more info. > > To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx > > " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx > > For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx > > > > > --------------------------------------------------------------------- > > The official User-To-User support forum of the Apache HTTP Server > > Project. > > See <URL:http://httpd.apache.org/userslist.html> for more info. > > To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx > > " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx > > For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx > > > > > > > > > > -- > sctemme@xxxxxxxxxx http://www.temme.net/sander/ > Open Source Software Consultant > PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF > > > > --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|