Re: Poor Load Balancer performance (Apache Users)

On Thu, May 27, 2021 at 6:22 AM Marc Serra <mserra@xxxxxxxxx> wrote:

Hi, I'm trying to understand how to load balancing works and testing it's benefits, and I'm in a trouble: in my tests (using ab command) I got better performance without the load balancer.

I try to describe my environment...

I created three VM on Virtualbox with the same hardware and software configuration and redirected ports using NAT:
CPU: 1
RAM: 1024MB
Network: NAT
Apache version: 2.4.41
Kernel: Linux node1 5.4.0-73-generic #82-Ubuntu SMP Wed Apr 14 17:39:42 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
OS: Ubuntu server 20.04.2

The machines:
central: only acts as a load balancer (not a balancer member), otherwise used as 'normal' apache server for results comparison. NAT from 8010 to 80
node1: when load balancer is configured in the central machine, acts as balancer member. NAT from 8011 to 80
node2: when load balancer is configured in the central machine, acts as balancer member. NAT from 8012 to 80

Here a list of apache loaded module.

spoiler!!! node1 and node2 have the same modules. central have the same modules as nod1 and node2 plus lbmethod_byrequests_module (shared)...

Apache loaded modules in central:
Loaded Modules:
core_module (static)
so_module (static)
watchdog_module (static)
http_module (static)
log_config_module (static)
logio_module (static)
version_module (static)
unixd_module (static)
access_compat_module (shared)
alias_module (shared)
auth_basic_module (shared)
authn_core_module (shared)
authn_file_module (shared)
authz_core_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
deflate_module (shared)
dir_module (shared)
env_module (shared)
filter_module (shared)
lbmethod_byrequests_module (shared)
mime_module (shared)
mpm_event_module (shared)
negotiation_module (shared)
proxy_module (shared)
proxy_balancer_module (shared)
proxy_http_module (shared)
reqtimeout_module (shared)
setenvif_module (shared)
slotmem_shm_module (shared)
status_module (shared)

Apache loaded modules in node1:
core_module (static)
so_module (static)
watchdog_module (static)
http_module (static)
log_config_module (static)
logio_module (static)
version_module (static)
unixd_module (static)
access_compat_module (shared)
alias_module (shared)
auth_basic_module (shared)
authn_core_module (shared)
authn_file_module (shared)
authz_core_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
deflate_module (shared)
dir_module (shared)
env_module (shared)
filter_module (shared)
mime_module (shared)
mpm_event_module (shared)
negotiation_module (shared)
proxy_module (shared)
proxy_balancer_module (shared)
proxy_http_module (shared)
reqtimeout_module (shared)
setenvif_module (shared)
slotmem_shm_module (shared)
status_module (shared)

Apache loaded modules in node2:
core_module (static)
so_module (static)
watchdog_module (static)
http_module (static)
log_config_module (static)
logio_module (static)
version_module (static)
unixd_module (static)
access_compat_module (shared)
alias_module (shared)
auth_basic_module (shared)
authn_core_module (shared)
authn_file_module (shared)
authz_core_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
deflate_module (shared)
dir_module (shared)
env_module (shared)
filter_module (shared)
mime_module (shared)
mpm_event_module (shared)
negotiation_module (shared)
proxy_module (shared)
proxy_balancer_module (shared)
proxy_http_module (shared)
reqtimeout_module (shared)
setenvif_module (shared)
slotmem_shm_module (shared)
status_module (shared)

Results when I run 'ab -k -n 10000 -c 1000 http://vm_host_ip_address:8010/' with no balancer...
Finished 10000 requests
Server Software: Apache/2.4.41
Server Hostname: 192.168.68.210
Server Port: 8010
Document Path: /
Document Length: 97098 bytes
Concurrency Level: 1000
Time taken for tests: 24.029 seconds
Complete requests: 10000
Failed requests: 5526
(Connect: 0, Receive: 0, Length: 5526, Exceptions: 0)
Keep-Alive requests: 8938
Total transferred: 870844342 bytes
HTML transferred: 868062523 bytes
Requests per second: 416.16 [#/sec] (mean)
Time per request: 2402.908 [ms] (mean)
Time per request: 2.403 [ms] (mean, across all concurrent requests)
Transfer rate: 35391.87 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 22 144.2 0 1039
Processing: 0 1087 2291.1 451 18504
Waiting: 0 1207 2615.1 426 18470
Total: 0 1109 2372.3 451 19514
Percentage of the requests served within a certain time (ms)
50% 451
66% 585
75% 678
80% 764
90% 1868
95% 7315
98% 10098
99% 11156
100% 19514 (longest request)
Finished 10000 requests

No changes on vhost configuration file...

<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Results when I run 'ab -k -n 10000 -c 1000 http://vm_host_ip_address:8010/' with balancer configured...
Finished 10000 requests
Server Software: Apache/2.4.41
Server Hostname: 192.168.68.210
Server Port: 8010
Document Path: /
Document Length: 97098 bytes
Concurrency Level: 1000
Time taken for tests: 15.527 seconds
Complete requests: 10000
Failed requests: 834
(Connect: 0, Receive: 0, Length: 834, Exceptions: 0)
Keep-Alive requests: 9166
Total transferred: 892851836 bytes
HTML transferred: 890000268 bytes
Requests per second: 644.04 [#/sec] (mean)
Time per request: 1552.701 [ms] (mean)
Time per request: 1.553 [ms] (mean, across all concurrent requests)
Transfer rate: 56155.42 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 22 146.6 0 1038
Processing: 0 657 1668.1 122 13453
Waiting: 0 752 1883.8 121 13453
Total: 0 679 1763.1 122 14476
Percentage of the requests served within a certain time (ms)
50% 122
66% 234
75% 330
80% 402
90% 1657
95% 4987
98% 7221
99% 8733
100% 14476 (longest request)
Finished 10000 requests

I replaced the apache vhost config file with...

<Proxy balancer://mycluster>
BalancerMember http://vm_host_ip_address:8011
BalancerMember http://vm_host_ip_address:8012
</Proxy>
ProxyPreserveHost On
ProxyPass / balancer://mycluster/
ProxyPassReverse / balancer://mycluster/

The index.html file is a 90KB plain text file for both node1 and node2

I'm missing something?

The problem is the way virtualbox handles this network configuration and in a real world the results would be different?

--
Marc Serra

Manxa 1876, S.L.
Ctra. Les Tries, 85. 17800 Olot (Girona)
Tel. 972 27 45 30 Fax 972 27 45 32
Manxa Industrial | Coneix més aquí
Manxa Ferros | Coneix més aquí
Manxa Ferreteria i Parament de la Llar | Coneix més aquí

El contingut d’aquest correu electrònic i els seus annexos és estrictament confidencial. En el cas que no siguis el destinatari i hagis rebut aquest missatge per error, preguem que ho comuniquis al remitent i procedeixis a la seva eliminació, sense difondre, emmagatzemar o copiar el seu contingut. Imprimeix aquest correu només si és necessari.
El contenido de este correo electrónico y sus anexos es estrictamente confidencial. En el caso de que no seas el destinatario y hayas recibido este mensaje por error, rogamos lo comuniques al remitente y procedas a su eliminación, sin difundir, almacenar o copiar su contenido. Imprimir este correo solo si es necesario.
The content of this email and its attachments is strictly confidential. If you are not the recipient and you have received this message by mistake, please notify the sender and proceed to its elimination, without spreading, storing or copying its content. Print this email only if necessary.
Le contenu de cet e-mail et de ses pièces jointes est strictement confidentiel. Dans le cas où vous n'êtes pas le destinataire et avez reçu ce message par erreur, veuillez en informer l'expéditeur et procéder à sa suppression, sans diffuser, stocker ou copier son contenu. Imprimez cet e-mail uniquement si nécessaire.