Error AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit

[Mike Dewhirst <miked@xxxxxxxxxxxxxxx>]

    Can anyone suggest some mpm_event.conf settings for a lightly loaded
    DigitalOcean Ubuntu 2022.04 LTS droplet with 8GB of RAM and a
    "dedicated CPU"?
    
    I just had to restart Apache which crashed around 22:22 UTC. There
    is 8Mb of the above error in the log.
    
    This is the plain Apache2 error.log rather than the named website
    error log which only shows expected errors such as reCaptcha errors.
    
    
    It seems to have started filling the scoreboard 24 minutes after
    logrotate. The first ten log entries after logrotate are ...
    
    [Tue Nov 22 00:00:01.934987 2022] [mpm_event:notice] [pid 49892:tid
    140450581641088] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2
    mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations
    [Tue Nov 22 00:00:01.935032 2022] [core:notice] [pid 49892:tid
    140450581641088] AH00094: Command line: '/usr/sbin/apache2'
    [Tue Nov 22 00:20:13.411015 2022] [mpm_event:notice] [pid 49892:tid
    140450581641088] AH00493: SIGUSR1 received.  Doing graceful restart
    [Tue Nov 22 00:20:13.514771 2022] [mpm_event:notice] [pid 49892:tid
    140450581641088] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2
    mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations
    [Tue Nov 22 00:20:13.514804 2022] [core:notice] [pid 49892:tid
    140450581641088] AH00094: Command line: '/usr/sbin/apache2'
    [Tue Nov 22 00:23:57.862865 2022] [mpm_event:notice] [pid 49892:tid
    140450581641088] AH00493: SIGUSR1 received.  Doing graceful restart
    [Tue Nov 22 00:23:57.937161 2022] [mpm_event:notice] [pid 49892:tid
    140450581641088] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2
    mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations
    [Tue Nov 22 00:23:57.937192 2022] [core:notice] [pid 49892:tid
    140450581641088] AH00094: Command line: '/usr/sbin/apache2'
    [Tue Nov 22 00:23:59.938800 2022] [mpm_event:error] [pid 49892:tid
    140450581641088] AH03490: scoreboard is full, not at
    MaxRequestWorkers.Increase ServerLimit.
    [Tue Nov 22 00:24:00.939954 2022] [mpm_event:error] [pid 49892:tid
    140450581641088] AH03490: scoreboard is full, not at
    MaxRequestWorkers.Increase ServerLimit.
    
    Then later ...
    
    [Tue Nov 22 22:28:52.526681 2022] [mpm_event:error] [pid 111175:tid
    140011041032064] AH03490: scoreboard is full, not at
    MaxRequestWorkers.Increase ServerLimit.
    [Tue Nov 22 22:28:52.526899 2022] [mpm_event:notice] [pid 111175:tid
    140011041032064] AH00492: caught SIGWINCH, shutting down gracefully
    [Tue Nov 22 22:28:55.888384 2022] [core:warn] [pid 111175:tid
    140011041032064] AH00045: child process 113420 still did not exit,
    sending a SIGTERM
    
    ... more of the same ...
    
    [Tue Nov 22 22:28:59.893773 2022] [core:warn] [pid 111175:tid
    140011041032064] AH00045: child process 113426 still did not exit,
    sending a SIGTERM
    [Tue Nov 22 22:29:01.896130 2022] [core:error] [pid 111175:tid
    140011041032064] AH00046: child process 113420 still did not exit,
    sending a SIGKILL
    
    ... more of the same ...
    
    [Tue Nov 22 22:29:01.896389 2022] [core:error] [pid 111175:tid
    140011041032064] AH00046: child process 113426 still did not exit,
    sending a SIGKILL
    
    ... I assume this is where I said restart ...
    
    [Tue Nov 22 22:29:03.055559 2022] [mpm_event:notice] [pid 122709:tid
    139966049515392] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2
    mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations
    [Tue Nov 22 22:29:03.055753 2022] [core:notice] [pid 122709:tid
    139966049515392] AH00094: Command line: '/usr/sbin/apache2'
    
    ... and no errors since.
    
    Below is the server-status after restart and below that is my
    current mpm_event.conf
    
    Many thanks
    
    Mike
    
    Server Version: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 mod_wsgi/4.9.0
    Python/3.10

Server MPM: event

Server Built: 2022-09-30T04:09:50

---

Current Time: Tuesday, 22-Nov-2022 22:41:35 UTC

Restart Time: Tuesday, 22-Nov-2022 22:29:03 UTC

Parent Server Config. Generation: 1

Parent Server MPM Generation: 0

Server uptime: 12 minutes 32 seconds

Server load: 0.00 0.00 0.00

Total accesses: 14 - Total Traffic: 121 kB - Total Duration: 10274

CPU Usage: u10.91 s.38 cu0 cs0 - 1.5% CPU load

.0186 requests/sec - 164 B/second - 8.6 kB/request - 733.857 ms/request

1 requests currently being processed, 449 idle workers

Slot	PID	Stopping	Connections		Threads		Async connections
			total	accepting	busy	idle	writing	keep-alive	closing
0	122713	no	0	yes	0	50	0	0	0
1	122714	no	0	yes	0	50	0	0	0
2	122715	no	0	yes	0	50	0	0	0
3	122716	no	0	yes	0	50	0	0	0
4	122717	no	0	yes	0	50	0	0	0
5	123029	no	0	yes	0	50	0	0	0
6	123082	no	0	yes	0	50	0	0	0
7	123083	no	0	yes	1	49	0	0	0
8	123190	no	0	yes	0	50	0	0	0
Sum	9	0	0		1	449	0	0	0

    # event MPM
    # ServerLimit: Upper limit on configurable number of processes
    (default 16)
    # StartServers: initial number of server processes to start (default
    3)
    # MinSpareThreads: minimum number of worker threads which are kept
    spare (default 75)
    # MaxSpareThreads: maximum number of worker threads which are kept
    spare (default 250)
    # ThreadLimit: upper limit on the configurable number of threads per
    child process (default 64)
    # ThreadsPerChild: constant number of worker threads in each server
    process (default 25)
    # MaxRequestWorkers: maximum number of worker threads
    (ServerLimit*ThreadsPerChild)
    # MaxConnectionsPerChild: maximum number of requests a server
    process serves
    ##
    # After reading
https://www.liquidweb.com/kb/apache-performance-tuning-apache-mpm-modules/
    <IfModule mpm_event_module>
            ServerLimit                     16
            StartServers                    16
            MinSpareThreads                 400
            MaxSpareThreads                 250
            ThreadLimit                     64
            ThreadsPerChild                 50
            AsyncRequestWorkerFactor        2
            MaxRequestWorkers               800
            MaxConnectionsPerChild          0
    </IfModule>
    
    
    
-- 
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Just
ask and I'll send it to you. Your email software can handle signing.

Attachment: OpenPGP_signature
Description: OpenPGP digital signature