At 5:12 PM -0700 5/30/08, Joel M. Halpern wrote: > > >> These both sound like excellent reviews: you expressed your personal >> design preferences in the first instance but did not try to force it over >> the consensus of the working group, and pointed out a showstopper >> in the second. >> >> Now, show me in this draft how these two cases are distinguished, >> which is critical to getting a review done right? > >The problem I have is that I do not know how to write text in a draft >that distinguishes those two. The line between them is very tricky, and >possibly subjective. Here is where I think we may have some room for some fruitful discussion. A review that is raising a showstopper has a provable or disprovable statement in it. "This *will not work* in the following scenario" or even "This seems to have poor results in networks with high rates of non-congestive loss" creates the opportunity for the reviewer and working group to discuss the issue in terms that can be tested. The working group may argue (and, presumably, demonstrate) that either statement is false, but there is a way to move forward that is not just iteration of position. To put that task to the working group, a reviewer should be able to back their statement with a reasonably clear, worked set of reasons for the assertion. A working group can disprove the statement, indicate why an applicability statement is a better response ( demonstrating that the cases where it does work or does have reasonable results are useful enough and distinguishable enough that the document should go forward), or make relevant technical changes. But review comments that do not contain testable assertions end up being subjective. "I think ZNK ChillOut is better than ZNK BindMeTight, for the following reasons" may contain a good set of reasons, but it should never over-rule the consensus of a working group that has agreed on ZNK BindMeTight unless one of those reasons amounts to "it doesn't work". Comments of the type "I think Section 4 is not clear enough for an implementor to follow" are also subjective; they are very valuable and may serve as a guide to the WG/author team, but it is important for the reviewer to recognize that they may not result in change. >And part of the problem is to avoid turning it into a fight. If all >review comments get clear, reasonably timely responses, there is room >for the discussion without acrimony. I don't think that helps unless there is a clear set of expectations *on the part of reviewers* on what their responsibilities are in producing an actionable review and in accepting that some of their comments may result in no change. I believe that the right way to do that is to ground the description of the review process in a strong understanding of the document production and participation model of the IETF. If you don't, you end up with an unbalanced view in which the few hours of effort put in on a solicited review have a very much larger effect than the effort of the participants who produced the document and will use the protocol it describes. >Would it address your concern if the document said something like: > "Reviewers should be sensitive to the difference between > their personal opinions (and preferences) and issues > which will affect the correct operation or interoperation > of the documents under review" >? I don't think this is nearly enough. Ted >I have no problem with pointing out that there are two different >categories. I have real problems with trying to define a hard line >which distinguishes them. > >Yours, >Joel > >PS: While there are other differences in our views, they seem to be >questions on which reasonable folk may differ and we can let the >community sort out how to write the wording. >_______________________________________________ >IETF mailing list >IETF@xxxxxxxx >https://www.ietf.org/mailman/listinfo/ietf _______________________________________________ IETF mailing list IETF@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf