Send CentOS-announce mailing list submissions to centos-announce@xxxxxxxxxx To subscribe or unsubscribe via the World Wide Web, visit https://lists.centos.org/mailman/listinfo/centos-announce or, via email, send a message with subject or body 'help' to centos-announce-request@xxxxxxxxxx You can reach the person managing the list at centos-announce-owner@xxxxxxxxxx When replying, please edit your Subject line so it is more specific than "Re: Contents of CentOS-announce digest..." Today's Topics: 1. *correction* Notification of outage : mirrorlist.centos.org (ipv4) 2016-04-16 08:58 UTC (Karanbir Singh) ---------------------------------------------------------------------- Message: 1 Date: Sat, 16 Apr 2016 23:16:12 +0100 From: Karanbir Singh <kbsingh@xxxxxxxxxx> To: centos-announce@xxxxxxxxxx Subject: [CentOS-announce] *correction* Notification of outage : mirrorlist.centos.org (ipv4) 2016-04-16 08:58 UTC Message-ID: <5712B9AC.10600@xxxxxxxxxx> Content-Type: text/plain; charset=windows-1252 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Apologies for this, the correct date should be : 2016-04-16 Sat. The timestamps remain accurate. On 16/04/16 10:09, Karanbir Singh wrote: > Hi, > > At 05:00 UTC 2016-05-16 an automated component update caused the > mirrorlist service for CentOS to go down for all IPv4 based > services. The IPv6 service for mirrorlist.centos.org was > unaffected. > > By 08:11 UTC 2016-05-16 I rolled back the impacted components, > disabled the update mechanics, and restarted services. > > By 08:15 UTC 2016-05-16 Services were returning to normal. > > By 08:23 UTC 2016-05-16 we had multiple confirmations from around > the world that services were restored. > > --------------- system wide followup: > > - We will work to extend and add where needed, better testing > around each component involved in such roles. > > - I will work with Fabian and make sure that all our automated > component and system changes that impact a production service are > only run during regular working hours for the team. > > --------------- root cause: > > all the components involved in the downtime are backed up and we > will start looking at the root cause of why services went down, for > now the immediate focus was to restore services which was done by a > rollback. > > --------------- reporting issues: > > Note that for real time, time sensitive issues always drop into > #centos-devel on irc.freenode.net and let us know, along with > filing a bug report at bugs.centos.org - For any non time critical > issues, please report them at bugs.centos.org against the > 'Infrastructure' project, and we will aim to address them as soon > as possible. > - -- Karanbir Singh, Project Lead, The CentOS Project +44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS GnuPG Key : http://www.karan.org/publickey.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJXErmsAAoJEI3Oi2Mx7xbtJdgIAI1+4KTYCdI4O+kFlrJ5IlMy TwFGYt7O1WyGDfXC27VLehDxNP0lFiZOd1TMMSWy0/P64TYcBr5GLa4Vc7WLpLef ywgjvgNE9VY69z4D1Lkgdxlgnhg31tPtW8kA1Py24nKrEFMYYi7/2tUGZn43kE8D SRn0CywFizCFIScB0xI7uSJRE2PzFFwixIG4LaEdqppxcId6hsKb80hzpIP35pPU eXlagl5CMad2XD0aVqKrqoU5W9vS9swKv9rc6ASsUHB28W7KcT0mq0YsIojhS81L w8hiSsTpW3b62uBb55MIvkg1Y11H0qlkW306ZaZAXjTpRsvNaz82+qRmgcVVJvc= =Uvpc -----END PGP SIGNATURE----- ------------------------------ _______________________________________________ CentOS-announce mailing list CentOS-announce@xxxxxxxxxx https://lists.centos.org/mailman/listinfo/centos-announce End of CentOS-announce Digest, Vol 134, Issue 9 *********************************************** _______________________________________________ CentOS mailing list CentOS@xxxxxxxxxx https://lists.centos.org/mailman/listinfo/centos