Responding to various people in one e-mail. To summarise, we have procedures that say what kinds of Discusses are appropriate, and personal engineering preferences are not. Legitimate issues should be raised, however, and in the case of most big issues, the right approach would be to send a document back to the working group for revision. But overall, my perception is that the ADs take their task very seriously, and try to do the right thing when performing their IESG reviews. I think they make a significant contribution to the quality of our documents. That being said, a couple of other things are also obvious. First, we have a very tail-heavy process. I get a lot of comments about that, and you all know it too. Moving more of the problem finding to earlier parts of the process is necessary (but also not easy). Secondly, I know from personal experience that feedback on what kinds of things get raised in the IESG review is very useful. Even if we are trying to do the right thing, we make mistake, and perhaps more importantly, our understanding of what kinds of discusses are useful keeps evolving. Please tell us when you think we are doing the wrong thing (be it not raising an issue or raising it in the wrong category). Finally, today too many big revisions are handled as a discussion between the authors and ADs. Big discussions should be done publicly in the working group, and in some cases the right action is for a document to be sent to the working group so that it can be reprocesses and returned. And then to the specific responses. Fred: > In my opinion, some individual ADs seem to, from their behavior, feel that they have not done their jobs unless they have raised a "discuss". The one that took the cake for me personally was a "discuss" raised by a particular AD (who shall remain nameless) that in essence wondered what he should raise a "discuss" about in my document. There were a couple of errors in that; he told me later that what he had done was opened the comment tool and typed that question, and subsequently accidentally hit the equivalent of "send" - the question wasn't intended to go out. But the question itself is telling: the issue was not "does the document meet the requirements of BCP 9", it was "what comment shall I raise"? And this is of course completely against our rules in the Discuss criteria document. But accidents do happen. If there were an intentional "I owe you" Discuss, that is obviously something that should go away, and authors and other ADs should complain about it. > Also, in my opinion, IESG review that raises a certain number of issues should not result in the document sitting in the IESG's queue for a few months while the authors go back and forth with the AD or the GEN-ART reviewer pounding the document into someone's idea of "shape" without working group involvement. I personally would prefer that simple matters get sorted out between the ADs and the author, but complex changes or additional content requested by the AD should result in the document being sent back to the working group. > I'm not saying it doesn't serve a purpose. I'm saying that I know of drafts that have been nearly rewritten during such back-and-forth, so what popped out was largely unrelated to what went in. In such cases, I think the document should have been returned to the working group with comments, not worked on privately. I agree 100% with this. Adrian: > Examples are useful because they give the IESG something to chew on. If you > don't call us when we do "bad stuff" we might never know. Indeed. For what it is worth, my own idea of what kinds of things I should as an AD complain about has evolved quite a bit. As a new AD I was overly eager, ready to defend the Internet against… bad documents. In some cases filing Discusses that in retrospect I probably should have filed as Comments. Over the years I've come to the realisation that most things are additional Comments for the authors to take into account, but that real brokenness should still cause a Discuss to be raised. But for such development to occur, you need to gain some perspective. Part of that process is getting feedback. > I believe that the clue is in the word "Discuss" and I hope that I (at least) am > willing to listen when the response is "we thought about it, it's not a big > problem." +1 Brian: > Seeing randomly selected drafts as a Gen-ART reviewer, I can > say that serious defects quite often survive WG review and > sometimes survive IETF Last Call review, so the final review > by the IESG does serve a purpose. This is one of the big issues. I think we all agree that our process is tail-heavy. More review and more problem discoveries occur at the end. This is trivial to see, but the crux is how that could be changed. Dave: > the tendency of ADs to inject spontaneous, late-stage personal engineering preferences, which might have been reasonable to add to the original working group discussion but realistically have no place in a final review And personal engineering preferences are against our rules for Discusses. But beauty is in the eye of the beholder, and so are bugs :-) Is a particular spec bug an legitimate, serious technical problem, otherwise somehow against agreed IETF consensus, or is it an engineering preference? > But what is tiring about this line of justification is its continuing failure to balance the analysis by looking at the costs and problems that come with it. Does it provide regular and sufficient benefit to justify its considerable costs? Agreed. Jari