On 10/30/2010 11:31 AM, Joel M. Halpern wrote:
One of the positive effects
of our current system is taht because WG knows tha tthye have to clear all the
ADs, not just their own, they actually think about all these issues. And usually
manage to cope with them
A working group is diligent or it isn't. It gets a range of feedback and
responds constructively to it... or it doesn't.
The working group behaviors that I have seen that pay explicit attention to the
specific question of satisfying AD reviews have nothing to do with quality of
the work and more to do with guessing what will personally bother an AD. In
other words, it's about dealing with AD idiosyncrasy rather than with quality.
It is now common to get cross-area reviews and my own observation is that these
are a) typically quite reasonable and diligent, and b) dealt with constructive
by the working group.
ADs do sometimes come up with interesting and even important points, but AD
review is an extremely expensive and often frustrating mechanism that we already
have a vastly superior replacement for. Its timing is better and it distributes
the work far better.
The fact that an AD sometimes catches some important problem is typically taken
as proof that the AD review and Discuss mechanism is essential. This is highly
flawed logic, on two counts.
One is that it does not represent meaningful cost/benefit evaluation. The cost
is actually quite high in energy, delay and frustration, and the significant
benefit overall is quite low (if the wg has been diligent and has gotten
cross-area reviews.)
The other is that protocol specs have a statistical likelihood of bugs, even
with the AD review. We talk about AD review almost as if it ensures perfection,
but of course we know it does not.
Ultimately, we have to trust the real world to evaluate the safety and efficacy
of a protocol. That fact ought to give us permission to balance the cost and
benefit of the quality assurance efforts we require during specification
development and approval.
Yes, it would be very good to spot all of these things sooner. I have not yet
seen a proposal that actually works for doing so. But letting WGs or WGs + ADs
approve documents for general advancement is a step likely to lead to problems.
If all our WGs handed their ADs high quality documents that they had checked for
all these issues, then maybe we could look at this differently.
We do need quality assurance efforts. The basic idea that working group efforts
are subject to outside review prior to approval is a significant value-add by
the IETF, IMO. The question is how to provide sufficient review in a reasonable
way.
I believe that cross-area reviews largely satisfy that requirement. If
within-area reviews are also needed, the AD should commission them, not do them
directly.
d/
--
Dave Crocker
Brandenburg InternetWorking
bbiw.net
_______________________________________________
Ietf mailing list
Ietf@xxxxxxxx
https://www.ietf.org/mailman/listinfo/ietf