After talking in #fedora-noc, we would like to make phx2 app servers
talk to other app servers via proxy01, rather than (potentially) going
out to a server across the country for the benefit of load balancing.
The issue this solves is that because apps hosted on admin.fp.o talk to
each other using the same admin.fp.o roundrobin that users use, when an
application hits a random proxy and that proxy has gone down for some
reason, the application would block its thread waiting for a response.
Eventually this would happen on enough applications that we would get a
flood of alerts, and ultimately cause downtime.
By using (only) proxy01, it means re-adding a single point of failure,
but it seems likely that the case of "proxy01 is down so everything else
is down" would mean that phx2 was having a network issue anyway, meaning
we'd be in the same position (i.e., app servers unreachable). It seems
unlikely that proxy01 will just die at random, and if it does then that
is a whole new issue that we should address. So yes, the single point of
failure is bad, but it seems marginally better than what we have been
seeing lately.
I would like +1's to push this to puppet, which effectively reverts
1bac8c9a and 23ceebd5.
[codeblock@lockbox01 puppet]$ git show
commit f0445cffd64d3db980a4b689517fd4b95f6e7686
Author: Ricky Elrod <codeblock@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Date: Fri Dec 6 17:55:38 2013 +0000
Make phx2 boxen use proxy01 for admin.fp.o again
diff --git a/manifests/services/phx.pp b/manifests/services/phx.pp
index 72c24be..317bfc2 100644
--- a/manifests/services/phx.pp
+++ b/manifests/services/phx.pp
@@ -11,11 +11,10 @@ class phx {
}
case $environment {
'production' : {
-# host { 'admin.fedoraproject.org':
- #ip => '10.5.126.52',
- #ip => '66.35.62.166',
-# ensure => absent,
-# }
+ host { 'admin.fedoraproject.org':
+ ip => '10.5.126.52',
+ ensure => present,
+ }
host { 'cvs.fedoraproject.org':
ip => '10.5.125.151',
host_aliases => ['cvs']
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
