Linda started this thread with huge performance problems
when Squid with a size of 12 GB forks 15 times.
Linda emailed me that she is doing a test with
vm.pmap.pg_ps_enabled set to 1 (the kernel will
transparently transform 4K pages into superpages)
which gives a big relief for TLB management
and will most likely reduce performance problems with
forks of very large processes.
I am very interested in the result of the test...
But independently if the transparent allocation of huge pages
will give the needed performance gain, forking 12 GB
is not nothing. Doing it 15 times is ... a lot of housekeeping.
Therefore I have just a silly question:
can we modify Squid to fork at a slower pace?
It would be interesting to see if a 0.2 second gap
between each fork gives the system a enough
time to make the fork of 12 GB happen before the next fork
is done. Perhaps with a little "intelligence":
no delay between forks with a cache_mem < 512 MB,
0.1 sec delay with cache_mem < 4 GB etc.
Any ideas ?
Marcus
Robert Collins wrote:
On Wed, 2009-11-25 at 02:11 +0100, Henrik Nordstrom wrote:
ons 2009-11-25 klockan 09:07 +1100 skrev Robert Collins:
On Tue, 2009-11-24 at 13:45 +0100, Henrik Nordstrom wrote:
tis 2009-11-24 klockan 15:06 +1100 skrev Robert Collins:
http://www.netbsd.org/docs/kernel/vfork.html has some interesting notes
from the BSD world about this.
vfork is fundamentally broken.
Beyond the obvious (that it doesn't separate the memory out?)
Undefined results if any of the following is used:
- threads
- signals
- any form of output
- pretty much any other syscall than an successful execve
In the 'vchild', yes. However I don't see why that would cause us
problems: in the child side of the vfork we would only be calling
execve / execvp - which will be fine.
there is other alternatives coming, getting around the virtual memory
issue when starting new processes.
What are they called?
Searching.. posix_spawn() and it's posix_spawnp() wrapper seems to be
the one.
They use vfork (POSIX_SPAWN_USEVFORK) or fork internally. vfork has been
available for longer and should be trivial to drop in as an experiment,
whereas posix_spawn is still not widely available (and has precisely the
same issues for us).
-Rob
