[users@httpd] varying 2.0.55 worker MPM process/threads/memory behavior

[edward.dunkle@xxxxxxxxxxx]

We are attempting to upgrade our Apache web servers
to 2.0.55.  We are 
running on several different platforms with different versions of Red 
Hat Linux.  We are using the worker module compiled in.

In our test env, on a 512 Meg RAM box, we can start with the desired 512
threads.  Our config looks like this:
ServerLimit 1
ThreadLimit 512
StartServers 1
MaxClients 512
MinSpareThreads 1
MaxSpareThreads 512
ThreadsPerChild 512
MaxRequestsPerChild  0

These all show up with a ps -aef | grep httpd.  On this box, free
is 
192420 with httpd stopped and 189000 with it running.  Not very much
difference.

In UAT, on a 1 Gig RAM box, we get an error message "unable to create
worker thread" after about 300 threads.  

Why would the threads require more memory when there is twice as much 
memory available?  
And how could the same config start OK on a smaller box?  

We can start with limits set at 256 instead of 512.  But only the
root 
process and a couple other processes (listener and starter) show up with
ps -aef.  We can get the others with ps -aefm.  So why are these
threads 
treated like full processes on some boxes but threads on others?  Are
there some OS level settings that affect this behavior?

One final bit of info, if that isn't enough to confuse you, is that we
have another build with the worker module from version 2.0.50.  We
can 
copy this build over to the UAT box and run it with limits set at 550.
 

Any ideas as to what is going on?  I have examined the config logs
from 
the various builds and don't see anything that would affect this 
behavior.  They all have -pthread.  

I did some testing and tracking of memory usage on another box but it 
doesn't seem to add up.  The starting point is this:
total=513208 used=479884   free=36008  shared=0

Starting with limits at 128, free drops to 33324.  This doesn't seem
like very much.
with limits at 250, free drops to 28764  - not much more
But on this box we can't even run 256, we get the out of memory error 
after 253 threads.  

I obviously don't understand Apache memory requirements or usage patterns.
 
When I get it running with128 and look at threads with top H they have
RSS=4456 and SHARE=2412
When I get it running with 250 and look at threads with top H they have
RSS=5708 and SHARE=2426

If someone could explain this or point me to docs that explain this I 
would greatly appreciate it.

Thanks,
Ed