On Tue, Jul 28, 2020 at 9:50 AM Henrik Levkowetz <henrik@xxxxxxxxxxxxx> wrote:
On 2020-07-28 13:54, Richard Barnes wrote:
> On Tue, Jul 28, 2020 at 7:43 AM Henrik Levkowetz <henrik@xxxxxxxxxxxxx>
> wrote:
>
>> On 2020-07-28 13:17, Richard Barnes wrote:
>> > This morning I feel quite far from where we should be with MeetEcho,
>> since
>> > I am unable to access any of the sessions ("Unauthorized"). I've heard
>> out
>> > of band that others are having similar problems. It appears to be a
>> > Datatracker capacity issue compounded by MeetEcho's failure to do SSO
>> > properly (and thus 100x'ing DT load). Two bespoke tools conspiring to
>> trip
>> > us up.
>>
>> And a lot of FUD not based on reality.
>>
>> This morning, the maximum number of connections per second to the
>> datatracker
>> has been 12, and we are currently set up to handle 100 in parallel..
>> Average
>> response time including slow pages is ~0.8s.
>>
>
> The fact that this had to be manually adjusted at meeting time, apparently
> not having been scale-tested in advance, is exactly the problem here.
Again, you're guessing, and guessing wrong.
There was no adjustment of this at meeting time, and no need for it.
Sorry, that was how I understood you when you said, "has been 12, and we are currently set up to handle 100 in parallel".
If that's not the case, then what caused the outage?
>> There's been no 100x of load, and Meetecho is following the OpenID Connect
>> protocol perfectly.
>>
>
> To be explicit about what I mean: Normal SSO services cache login results
> so that they don't have to hit the OpenID endpoint every time. MeetEcho
> apparently does not, because it requires login for each session. So every
> time a session starts, MeetEcho is slamming the datatracker with login
> requests, when it could have sent none at all. I used "100x" because there
> are around 100 people in the MASQUE session this morning, but it's really
> much higher because the denominator should be basically zero.
I'm sorry? If you have 100 people who need to hit the datatracker 3 times
for 3 sessions, instead of 1 time, the factor is 3x, not 100x.
This is a silly point to fight over, but I'm talking about the load at meeting start time. If MeetEcho were properly caching logins, then there would be zero or maybe a handful of requests to the datatracker at the beginning of a session. Instead there are hundreds.
>> The point being that these bespoke tools have a cost, not just in units of
>> > dollars, but also in choice, reliability, etc. We should think hard
>> about
>> > what is so essential in our DNA that it merits all the costs.
>>
>> And base decisions and engineering on real data, instead of guesswork.
>>
>
> I'm not guessing, it's clear from the MeetEcho UI that they're doing SSO
> wrong.
It's perfectly clear that you're guessing at causes, and repeatedly guessing
wrong.
What am I misunderstanding here? Is MeetEcho actually caching SSO results and lying to users? :)
--Richard
Henrik
>
> --Richard
>
>
>
>>
>> Henrik
>>
>>
>> >
>> >
>> > On Tue, Jul 28, 2020 at 1:30 AM Carsten Bormann <cabo@xxxxxxx> wrote:
>> >
>> >> On 2020-07-27, at 20:28, Richard Barnes <rlb@xxxxxx> wrote:
>> >> >
>> >> >>> We should be minimizing our dependence on customized features.
>> >> >
>> >> >> why? so we can be dependent on webex? pfui!
>> >> >
>> >> > On the contrary, the more special features we have, the more locked in
>> >> we are to the tools with those special features, and the less
>> flexibility
>> >> we have to adapt with the times / budgets / needs of the community.
>> >>
>> >> I would agree with the sentiment on anything that is not essential to
>> our
>> >> DNA.
>> >> We probably should not use specialized accounting software.
>> >> But running our internet-drafts repository on Typo3 (*), while it also
>> >> would get rid of dependencies on customized features, is not what we
>> should
>> >> do.
>> >>
>> >> Web meetings that are replacing the traditional IETF week are essential
>> to
>> >> our DNA.
>> >> Different from running an insurance company or anything else that has
>> >> cookie cutter meetings, these really should be based on situated
>> software
>> >> [1].
>> >> Fortunately that can be built on a standard: The Web + WebRTC.
>> >> (Unfortunately, implementations of our own dogfood aren’t fully cooked,
>> >> but that is a problem with any web-meeting software. Also, A/V
>> >> fundamentally is hard, most people have little education in this space,
>> and
>> >> a lot of garbage hardware gets in the way.)
>> >>
>> >> I think we are quite close to where we should be with meetecho.
>> >> We are maybe not doing the development in a sufficiently agile way.
>> >> Of course many reasons can be cited why that is hard to do right now,
>> but
>> >> we should strive for agility.
>> >> It is also not clear that we own what we develop; RFC 873 applies.
>> >>
>> >> Grüße, Carsten
>> >>
>> >> (*) I’m sorry, not everyone here has used Typo3, so that sentence may
>> not
>> >> invoke the horror it should.
>> >> [1]: https://en.wikipedia.org/wiki/Situational_application
>> >> (The renaming doesn’t get the point. Well, I hope you do.)
>> >>
>> >>
>> >
>>
>>
>
