Re: IETF Website Degradation

Glen <glen@xxxxxxxx> · Mon, 3 Aug 2015 15:33:41 -0700

Hi Adam -

Totally understood.

Alas, at the moment, without this mode, *nobody* could access the site.  :-)  So as a hopefully very temporary band-aid, I need to leave this on for now.

To all IETF participants and all those impacted by this, I'm really sorry for the inconvenience (and I am - it impacted me as well! ) and as soon as we can get this shut off safely we will do so.  I'm hoping this won't be long.  We will try turning it off several times this evening, and see if the attack has stopped; however, given Cloudflare's east coast location, it may be tomorrow morning (roughly 14 hours from now) before we can reach a human at Cloudflare to find out what's really going on.

Also, as a reminder, please note Adam's example:  I'm not on the IETF discussion list at the moment, so please email me directly if you need to reach me, as Adam did.

Thanks again for your patience!

Glen
Glen Barney
IT Director
AMS (IETF Secretariat)

On Mon, Aug 3, 2015 at 3:26 PM, Adam Roach <adam@xxxxxxxxxxx> wrote:

    Incidentally -- and I don't know if you
      turned this on or if Cloudflare did -- but the interstitial page
      that appears when accessing IETF pages is described by Cloudflare
      like this: "

      _javascript_ and cookies are required for the tests and recording
      the fact that the tests were correctly passed."

      I know some significant percentage of IETFers run with _javascript_
      turned off, so this may pose a hardship for them.

      /a

      On 8/3/15 17:03, Glen wrote:

                All -

                We are experiencing degradation on the websites served
                by the primary IETF server group, including the IETF
                main website, the IAB and IRTF sites, and the
                Datatracker.

              Page load times for all sites are extremely slow.

            We have engineers working on this now.

          The problem *seems* to be that Cloudflare is sending us
          hundreds of simultaneous connections from hundreds of
          different IP addresses across many netblocks.  We are
          contacting Cloudflare now to see why this is happening, and
          what can be done to mitigate it, and we are continuing to
          investigate options on our end to restore normal service.

        Other services and sites are not impacted, and one way or
          another we will have service restored shortly.  Thank you for
          your patience in this matter.

        Glen

        Glen Barney

        IT Director

        AMS (IETF Secretariat)