--On Tuesday, January 22, 2013 16:31 -0500 Thomas Narten <narten@xxxxxxxxxx> wrote: > FWIW, I share Joe's concerns. And Stephen's responses don't > really change my mind. > > This document seems to have a bit of missing the forest for the > trees. In the overall scheme of things, I don't believe the > draft will materially help, and is at best a distraction from > dealing with meaningful problems. > > The crux of the issue is that any attempt at "fast tracking" is > fundamentally about short-circuiting some aspect of our review > processes. We should only do that very carefully. In almost > all cases, individual judgement is needed as to when it is > appropriate to do that. ADs/WG chairs already have some > flexibility here. e.g., a WG can skip WG LC if they think its >... Hi. First of all, I am completely in agreement with Thomas and his analysis. Anything that can reasonably and appropriately be done by this sort of effort can be done by discretion without adoption of a formal procedure and adoption of a formal procedure creates additional and unnecessary risks. As co-author of the process experiment model, I also object to its use when it is not demonstrably necessary, for example to give extra status and force IETF-wide use where the actions suggested can be carried out as a matter of normal discretion (see Section 5 of the document). In this particular case, a better experiment would be to use a process like this for some otherwise-relevant drafts and in some areas only and compare the results. The "force IETF-wide use" part doesn't apply as long as its invocation is a matter of WG Chair and AD discretion and belief that the right conditions are met, so, other than consuming a rather large amount of community time since it was first proposed almost two months ago, I just don't see what it accomplishes other than adding to the collection of process documents and constraining the review process, including about issues that running code doesn't help to inform. The process experiment mechanism is, IMO, an entirely inappropriate way to simply make a statement that the IETF really, really, does believe in running code and interoperability... especially since one would hope that such a statement would not expire after a year. Another thing this draft accomplishes is to shift the responsibility for determining whether a document is "good enough" from an IESG interpretation of community consensus to the decision of a single AD as to whether or not changes are required (Section 3, item 5). While it would probably improve speed of processing, it is not, in general, a desirable change, especially if one believes in the first part of the Clark quote. It is probably just a nit in the grand scheme of things but, if the responsible AD doesn't have time to do a sufficient review to be confident in voting "yes" (Section 3, item 7) how can she possibly sign off on the belief than no changes are required (Section 3, item 5)? It seems to me that situation creates a situation in which an AD can initiate this process, take the word of the WG that everything is in order, and then not only avoid responsibility by abstaining but use the provisions of Section 3 item 8 to let the document go through (as long as some AD is willing to say "yes") without taking real responsibility. Especially because we don't have firm rules against a WG Chair and responsible AD coming from the same organization, I would hope that those who are concerned about IETF CoI and antitrust policies would take a careful look at that scenario because it would seem to avoid many of the protections that are inherent in wide community review and individual responsibility. Other than that, it is nothing more than a specific scenario consistent with Thomas's concern that this procedure could be used to short-circuit effective review. I'm not going to quote extensively from Thomas's other arguments or waste bits trying to summarize them, but let me add a few observations to Thomas's about the document and how it is being handled. > If it is really urgent to get a document done, it is far > better to take steps to make sure the editors are engaged and > responsive. That is more likely the real issue!!! And that no AD is going to screw around nit-picking or delaying things in other ways. Based on experience, I have absolutely no confidence that a prohibition against anything that isn't important (such as the one in this draft) will stop that behavior if, e.g., someone thinks the document is incomprehensible or ignores important issues. (And, as Thomas points out, it shouldn't). > Running code is valuable. Always has been, always will be. But > we need to resist the temptation of making "running code" more > equal than other criteria or putting it on a pedestal as some > sort of holy grail. Running code by itself is just a sound > bite. The importance of running code is what it tells us about > a protocol specification. Just the mere fact that there is >... Moreover, Dave Clark and sound bites notwithstanding, interoperability is lots more important to us than running code, partially because, while a lot of things might be learned by a good-faith implementation of a specification by someone not involved in its development, an implementation by an author or other active participant in the development of a document may actually tell us very little without third-party and interoperability testing ... including not telling us whether it is actually "running", much less conforming, code. The latter is a _huge_ can of worms if it is going to be a procedural gating factor in the way the document proposes. In that light, the document says "The spirit [...] is that early implementations and ongoing development and testing throughout the protocol work can significantly improve the quality of a protocol and of its specification." I agree with that and hope everyone else does too. But, other than general encouragement and observations that such testing might be appropriate (e.g., in Section 4 item 13) this proposal makes no provision for enabling or encouraging "ongoing development and testing throughout the protocol work". Instead, it concentrates only on an assertion that an implementation exists, that it is conformant enough to be informative, and that it "runs". I would have more sympathy with this proposal if its gating condition was serious interoperability testing among independent implementations, but that could still be done on a discretionary basis (as 2026 points out in Section 4.1.1). > But overall, I see this document at best as a no-op. However, > I fear that it can be used to short-circuit our review > processes in a way that doesn't help the IETF and that won't > really speed things up enough to justify the downsides. Yes. In addition, this document moves a number of informal IESG procedures -- including the DISCUSS criteria and even the Shepherd report requirements (now Informational in RFC 4858 plus a number of less formal "statements" and "templates" that have sometimes changed rapidly) -- into the status of formal procedural requirements despite the details of those procedures never having gotten community consensus about those specifications. Under normal circumstances, the relationship between this document and those specifications would require additional actions to prevent this document from going into permanent publication hold. [DCRIT] is called out as a normative reference without the discussion and alerts normally required; the shepherding procedures and templates are not called out at all. I'm also feeling a need to say something that people seem to be missing or avoiding. In my experience in the IETF, we almost never allow a individual submission document that is obviously very controversial to turn into a extended debate on the IETF list with the author trying to refute every comment. Sooner or later (usually sooner), the sponsoring AD says "hey, not ready for the IETF List, much less Last Call" and pushes it off into either a separate mailing list (to emerge only when there is real consensus) or asks the author to take steps to propose and organize a WG. So, Adrian, noting the ratio between discussion of this draft on the IETF list in the last few weeks and discussions of everything else, how long does professional courtesy to another IESG member (presumably in combination with your thinking this is a good idea) encourage you to allow the unintentional DoS attack on the IETF list to continue? regards, john