Once I understood what the problem was, I was able to test it more thoroughly. I took a copy of Star Downloader and configured it to split up a single file into 10 chunks for faster downloading. Then I took a freshly-started Apache 2.0.54 server and told Star Downloader to being downloading a single file. It immediately broke it into several chunks and made 10 requests of the Apache server, each with a different byterange.
The effects on my server were dramatic, as you can see from this "vmstat 1" (every line represents one second)
procs -----------memory---------- ---swap-- --system-- ----cpu---- 2 0 128368 728244 12044 105432 0 0 1358 9823 6 14 50 29 0 0 128368 626192 12048 108288 0 0 1303 13096 8 19 63 9 10 0 128368 470044 12048 108288 0 0 1283 49044 18 38 45 0 12 0 128368 169204 12048 108548 0 0 1228 106594 29 71 0 0 8 4 127664 12260 5496 71652 0 204 1298 73347 22 69 0 8 0 17 178472 11996 5544 37556 32 51780 1380 7073 8 43 0 49 0 16 179200 12368 5564 38704 32 1712 1319 1277 1 3 0 96 0 14 180056 12244 5612 41284 568 968 1332 4709 2 4 0 94 0 14 180056 12120 5636 41884 696 1484 1394 1692 1 4 0 96 0 14 182116 11988 5652 43704 176 4772 1488 2841 2 5 0 93 0 16 198168 12236 5676 44920 128 17276 1466 873 0 4 0 95 0 17 218620 12416 5680 45568 260 22432 1455 694 1 5 0 95 0 19 221760 12108 5688 47040 224 5560 1396 3476 2 6 0 92 0 18 224692 11988 5724 48012 612 3744 1403 1796 0 3 0 96 0 20 233944 11980 5752 48544 0 10452 1355 811 1 3 0 97 0 17 234612 12096 5808 50344 204 1456 1356 2878 2 4 0 94 0 16 234876 12096 5868 51816 300 1524 1377 1578 1 3 0 97 0 19 271892 12092 5896 51888 384 39996 1518 2744 2 11 0 88 0 17 314812 12156 5920 51572 752 43876 1467 959 0 9 0 90Within 5 seconds, I had dropped from 728,244b free RAM to only 12,260b free. Immediately after that and the swap space usage began to rise at an alarming rate. All that just from a SINGLE download request split up into 10 smaller requests! No wonder the server was choking so often.
I then restarted the server and tested the same process with Star Downloader, but this time requesting the file directly rather than going through a download CGI script. This time Apache didn't blink an eye - it handled all 10 split-up requests with barely a flicker in memory usage. So your theory was right - the problem only occurred when a file was requested via a download accelerator and only when an intermediary CGI was handling the file transfer on the server side.
Next, I applied the patch, rebuilt and restarted Apache, and made sure Star Downloader could still download the file directly (without the CGI download script). Everything was still fine - the request was split into 10 requests and Apache handled them all easily.
Finally, the big test. I again requested the file with Star Downloader using the server's CGI download script. This time, though, the request was not split into 10 smaller requests. Instead, Star Downloader had to content itself with a single downloading process, and Apache's memory usage held steady.
So, hooray! Thanks again, and if you have any influence to get that script backported to the Apache 2.0.x branch, that'd be wonderful.
--------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|