There's something inherently wrong with trying to establish criteria for voting DISCUSS. My understanding was always that DISCUSS was supposed to be an indication that, at a minimum, the AD needs to understand the situation better before casting a yea or nay vote. The resolution of a DISCUSS might end up being a yes vote, a no objection vote, a request for clarification of the document. It should also be possible for an AD to say "now that I've understood better, I can't support this going forward" (for which ABSTAIN is not an appropriate label). It should never be wrong for an AD to vote DISCUSS, though it's reasonable to limit the amount of time during which a DISCUSS can stand for a particular document. It should also never be wrong for an AD to vote NO if the AD has initially voted DISCUSS, made a sincere effort to understand the issues, believes that he does so, and can state 2026 or other documented criteria for why the document is not suitable for approval. So my recommendations are: 1. Fix the broken IESG voting system before you try to establish more decision criteria. I'm serious. The system you have now is just too broken, and has been broken for a long time. It places too much pressure on ADs to cave in when a WG produces flawed output that isn't easily fixed. It is weighed too heavily in favor of approving a document - such that one AD can vote YES, another vote DISCUSS, no other ADs read the document and all vote NO OBJECTION, and the AD that votes DISCUSS will be expected to change that vote to ABSTAIN because nobody else felt like reading it. And the idea that an AD should ever, ever, be compelled to change his vote to an ABSTAIN is completely unacceptable. And the "votes" of ADs who haven't read the document should not count AT ALL. Here is a stab at better voting criteria: - The possible votes are: Yes, No, No Objection, Discuss, Abstain, Recuse - Yes means "I've read it, I believe it meets applicable criteria (2026 or whatever else is applicable), that there's community-wide consensus to support the document, and that all issues raised by reviewers (including directorate, last call comments, etc.) have been adequately addressed." - No means "I've read it, but one or more of the above criteria have failed to have been met." The criteria must be documented. - No Objection means "I've read it, and I think there are issues with it, but I don't think those issues are sufficient to block the document". The criteria should be documented, but the AD isn't compelled to do so. - Discuss means "I have read the document and see at least one potential issue which needs to be discussed." Discuss should always be raised before voting No. However, Discuss votes should time out and be replaced by Abstain (if not explicitly changed by the AD) after say 45 days. (however, nothing should ever stop an AD from changing his vote to No.) Note that the ADs, WG chairs, authors, etc. should attempt to resolve the issue offline (not during the telechat or other IESG meeting) but the discussion should be archived. - Abstain means "I did not read this", and/or "I don't consider myself competent to review this" - Recuse means "I have a conflict of interest with stating a position on this document" All Discuss votes must be changed (whether explicitly by the AD or by automatic timeout) before a ballot is finished. If there are two or more No votes, all ADs without a conflict of interest are expected to read the document, and the vote is taken again. In order to approve a document or standards action, there must be twice as many Yes + No Objection votes as No votes. 2. Don't overconstrain the use of DISCUSS. In particular, don't ever create a situation where a reviewer can't cite a problem with a document, regardless of whether that problem has previously been enumerated. It's fine to have guidelines for the benefit of new ADs but they should be nonbinding. You need to have other ways of dealing with the occasional stubborn AD who votes DISCUSS or whatever without a defensible reason. This applies to all IESG voting actions, not just moving from PS->DS/IS. 3. I take serious issue with the statement in the draft that IESG reviews are "reviews of last resort" and the implication that WG reviews are sufficient. In numerous situations this has not been the case. I'm all for having IESG place greater reliance on directorates (especially if this is formalized to some degree), so as to lessen IESG's workload to to get individual ADs out of the hot seat. But for IESG to presume by default that the WG has done due diligence is for IESG to shirk its responsibility. 4. When evaluating PS->DS/IS actions, IESG should avoid repeating the PS review. Instead it should look at what has changed between PS and DS/IS, and what has been learned as a result of implementation and deployment experience. The changes have to be minimal and "safe". No new features can be added, and there needs to be reason to have confidence (ideally backed by analysis and implementation experience) that any changes made will not impair interoperability between both PS and DS/IS versions of the protocol. Design flaws discovered since approval as PS are also fair game, and flaws for which the implications are better understood than at PS time should also be fair game. However it's not fair to revisit an issue that was raised at PS approval time, unless new information or understanding makes that issue of worse consequence than understood at that time. (e.g. say an issue was raised at PS time and it was decided it was too minor to hold up the document, but since then practical DoS attacks had been identified that exploited that problem...then it's fair to raise the issue. It's always fair to raise an issue about something that actually causes problems.) What's also not fair game is to "raise the bar" - to expect the document at DS to meet more stringent criteria than it was required to meet at the time of PS approval. IESG needs strong justification (e.g. community consensus) to raise the bar in this way for standards that are actually deployed. This includes but isn't limited to security issues. (Though it might be reasonable to make an exception to this if more than N years had lapsed since PS approval.) But IESG should not presume that there are no serious technical issues. It shouldn't presume one way or the other. And it shouldn't discourage ADs from raising DISCUSSes about technical issues at all. Keith On Aug 30, 2011, at 4:18 PM, Jari Arkko wrote: > During the discussion of the two maturity levels change, a question was brought up about DISCUSSes appropriate for documents that advance on the standards track. We discussed this in the IESG and I drafted some suggested guidelines. Feedback on these suggestions would be welcome. The intent is to publish an IESG statement to complement the already existing general-purpose DISCUSS criteria IESG statement (http://www.ietf.org/iesg/statement/discuss-criteria.html). > > Here are the suggested guidelines for documents that advance to IS: > > http://www.arkko.com/ietf/iesg/discuss-criteria-advancing.txt > > Comments appreciated. Please send comments either on this list or to the IESG (iesg@xxxxxxxx) in time before our next telechat (Thursday, September 8, 2011). > > Jari > > _______________________________________________ > Ietf mailing list > Ietf@xxxxxxxx > https://www.ietf.org/mailman/listinfo/ietf _______________________________________________ Ietf mailing list Ietf@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf