Update Re: IETF Website Degradation

[Glen <glen@amsl.com>]

All -

We have determined that the degradation was caused by a DDoS attack against the www.ietf.org website.  The attack was a slowly-escalating attack, which began several hours ago, and increased in load over the afternoon.  The attack was directed at the Cloudflare servers, so we were not immediately impacted.

However, as time passed, the results of the attack started to spill over to the actual IETF webservers, with the result that our webservers started to slow.  We were alerted to this by our own monitoring systems, which is when we did an initial check, and I then sent the initial report out.

At this point, we have been unable to reach a human at Cloudflare, although we are continuing to try.  We have therefore put our Cloudflare account into "DDoS Mitigation Mode".

In this mode, users will see a brief interstitial page when browsing the IETF website.  This page allows Cloudflare to perform testing on each browser to determine whether the request is part of an attack or not.  You may see this page as you approach the IETF website.  It is nothing to be alarmed about, and is an expected side-effect of this protection mode.

It is unknown, at this point, why Cloudflare did not automatically detect, and block, the attack.

It is unknown, at this point, why the attack caused Cloudflare to start spilling requests over to us.

It is unknown, at this point, why we are unable to reach a human there. :-)

However, at this time, website service is restored, and, apart from the interstitial page on the IETF website, everything is running as expected.  We will continue to reach out to Cloudflare to address these remaining issues, and will get that check page deactivated as quickly as possible.

Thank you for your patience during that happily brief degradation.

Glen

Glen Barney

IT Director

AMS (IETF Secretariat)