Russ and other IESG members, I have been trying to avoid getting embroiled in IETF process issues since the collapse of NEWTRK, but, after reviewing draft-housley-two-maturity-levels and the discussion on the IETF list, the situation leaves me little choice other than reengaging. Several of the comments that follow overlap comments others have made on the list, others are new (or updated versions of ones that predate the current proposals), but I think it is worthwhile to try to draw things together. This (unfortunately rather long) note addresses several topics that are almost, but not quite, separable. If people want to address specific ones of them, it may be useful to change titles and open up separate threads. They are: (1) Analysis of the problem as stated in draft-housley-two-maturity-levels-01 (2) Promoting more use of Draft Standard (under any name) by eliminating Full Standard. (3) Conflating protocol quality with document quality (4) Downward references (5) Abandonment of recommendation levels (6) The futility of using a small number of categories to describe a multidimensional situation: Status description versus status categories (7) The process questions surrounding proposal, posting, and processing of this document. ----------------------------- (1) Analysis of the problem as stated in draft-housley-two-maturity-levels-01 The draft indicates (Section 1) that: "The proposed changes are designed to simplify the process and reduce impediments to standards progression" and then "Since many documents are published as Proposed Standard and never advances (sic) to a higher maturity level, the initial publication receives much more scrutiny that is call (sic) for by RFC 2026 [1]." However, if the issue is that documents rarely progress beyond Proposed, then there is no evidence that tampering with the relationship between the current Draft and Full Standard levels will bring about any change in that relationship, i.e., increase the number of documents that move past Proposed. I note, as others have noted, that the current de facto requirements for Proposed are far in excess of what 2026 requires. If the problem is (as I believe it is at least in part) that initial publication receives far too much scrutiny and takes far too much time and energy, then formally no procedural changes at all are needed: the IESG simply needs to stop doing that and instead and apply only the level of scrutiny that RFC 2026 requires, and no more. Such a change would result in more rapid publication of protocol documents as Proposed Standards (perhaps reducing the fraction of the Internet that actually runs on I-Ds) so the community can evaluate whether reducing the community exhaustion from getting things to Proposed results in more documents moving to the next level (whether that is one next level or one of two. It appears to me that there are portions of the IETF community who are willing to advance documents to Draft. Most of that subcommunity is willing to put in the work to continue to advance documents to Full Standard and sees value in doing so. Other portions of the community see little value in moving documents past Proposed. It doesn't seem to me that this proposal will benefit either sub-community -- those who won't advance past Proposed under either system and those who are willing to advance to Draft and Full-- so I'm not quite sure who would benefit. Those who have been anxious to advance documents within the existing model have often met significant resistance from IESG members. It is somewhat disingeneous for the IESG (or even some of its members) to resist efforts to move documents to Draft or Full Standard, to place requirements on approval of Proposed Standards that go well beyond those of 2026, and then to complain about how few documents advance to Draft or Full Standard and use those complaints as the justification for abolishing the Full Standard level. On the other hand, the analysis in the document ignores the observation that, independent of the interoperability demonstration, moving a specification up in maturity levels often results in significant improvement in document quality and clarity. Again, it might represent an even larger improvement if the IESG more closely adhered to what many of us believe is the intent described in RFC 2026 and earlier documents -- to get a specification published for examination and implementation purposes as soon as all known technical defects have been identified or eliminated but without investing the resources to polish the text editorially. See (3) below for more discussion on that subject. Conclusion: Whatever this proposal may or may not accomplish, there is no evidence that it will solve the problems identified in its Introduction. (2) Promoting more use of Draft Standard (under any name) by eliminating Full Standard and renaming. YMMD, but I believe that the fraction of the community who now believe that having something at Proposed (1st step) only is a bar to implementation is fairly small. If it weren't, we'd be in bad trouble as a consequences of the very active use of a number of protocols that have never advanced beyond Proposed. It is possible --although I have some doubts-- that rebranding would help here. But, to the extent to which the need is to rebrand, that does not, in itself, justify changing the maturity level model. If one simply wanted to rebrand to reflect present-day reality, we would change Proposed/ Draft/ Standard to Standard/ Standard with Gold Star/ Standard with Gold Star and Network Cable Wreath. Those titles are somewhat in jest, but the point is that one wants to make the first level "Standard" and then start talking about endorsements of higher specification quality or assurance (see #6 below). Our real terminology/ branding problem is that "Proposed Standard" has periodically been interpreted -- sometimes by people or organizations with other agendas -- as "substandard and really not suitable for deployment". In fairness to them, that was exactly our intention in the early 1990s. That intention was very much part of the "upgrade or die" language in 2026, but has never been used. Leaving "Proposed Standard" as part of our vocabulary doesn't solve that problem. Finally, along the same lines, I disagree that STD numbers "offer little value". If properly used, they offer considerable value in providing a stable identifier, at least unless more sophisticated arrangements are adopted (see #6). The problem is that we are largely running on Proposed Standards and STD numbers are assigned only to Full Standards. The NEWTRK WG discussed solving that narrow problem in early 2006 by simply assigning protocol-specific identifiers to documents when they reached the first standards-track level. Like most other NEWTRK proposals, it was never formally considered by the community. It would make much more sense to follow that advice than to discard a facility that should be a useful one. See https://datatracker.ietf.org/doc/draft-ietf-newtrk-docid/ for more information and discussion. I also note that BCP numbers are sometimes used to cluster groups of documents just as STD numbers are. In both cases, there may be issues with idea and mechanism for clustering, but, if one can extrapolate from the BCP experience, the problem with STD numbers is that they are not assigned early enough, not that they exist and are inherently confusing, as your draft suggests. Conclusion: there is probably more than adequate justification to do some rebranding and/or to assign STD numbers (or equivalent) at entry into the Standards track, but the new naming categories proposed will lose information without really improving anything. (3) Conflating protocol quality with document quality Advancing documents according to the current model actually provides two separate (and separable) functions. One is certification of some minimal proof of interoperability (Draft) and utility (Full). The other is the specifications almost always undergo significant improvements in the quality of their documentation and description as people are required to re-examine and re-edit them in the process of advancing between grades. I suggest that the IESG is not taking enough advantage of that distinction and the advantages of the review process and may have lost sight of it entirely. First, we would be much better off in terms of the perceived speed with which we can do work if we stopped doing editorial nit-picking of first-level (Proposed Standard) documents, concentrating on making them only good enough to be clear to a well-intentioned reader. Such nit-picking and fine-detail editorial work for first-level documents is problematic whether it is done in the WG or the IESG and almost as problematic if done by professional RFC Editor staff. If nothing else, it is one of the factors that induces exhaustion with documents and acts as a deterrent to advancing them. Either way, we should have a high document-quality expectation at the second (and, if it exists, third) maturity levels. Pragmatically, that expectation is generally inconsistent with either "adopt the first written version of the spec initially at the highest level" or waving of the minimum time in grade rules. For the latter, some time away from the document and opportunity for the community to try to use it are likely prerequisites to having a useful editing/ document quality improvement effort. I would favor giving the IESG the ability, after Last Call and demonstration of community consensus to waive those time requirements in exceptional circumstances. But abolishing them seems to me to be the wrong approach. Conclusion: Improving document quality of mature specifications should be, and remain, an important goal. An excessively high threshold for Proposed Standards and reducing the number levels and qualifications (including elimination of minimum times in grade) for advancement may frustrate, rather than further, that goal. (4) Downward references I don't see the evidence for "major cause of stagnation in the advancement of documents" unless one relaxes the rule to permit normative references to I-Ds (which I do not favor). If the IESG believes that a document is being stuck unreasonably (or is likely to get stuck), it has the ability to invoke the RFC 4897, which merely requires asking the community (in a Last Call that can be combined with the Last Call on the document if desired but that can also be applied later) and noting the downref in the document. That is not an onerous procedure, yet the IESG has never chosen to use it. If we really want to treat standards at the second maturity level (or third, if it remains) as better and more completely-specified than those at lower levels then the downref restriction should remain, with the community able to make exceptions when the lower-level document(s) are considered sufficiently mature and well-specified. If, by contrast, additional maturity levels are endorsements indicating that almost-orthogonal quality levels have reached (as in the "gold star" recommendation of #2 above), then the downref issue may become irrelevant. Conclusion: This proposed change solves a problem that has not been demonstrated to exist, even to the extent of demonstrating that the elimination of current mechanisms for permitting downward references would save the IESG significant time. (5) Abandonment of recommendation levels Section 3.3 of RFC 2026 specifies a set of "requirement levels" for technical specifications (usually supplied in an applicability statement). We have largely stopped using those requirement levels, but the conflation of publication of the document in the standards track (and some decisions to not publish in the standards track) with recommendations that the specification actually be implemented and used has been another source of confusion including the publication of specifications for some very stable and extensively deployed protocols and other functions as Informational or Experimental, rather than Standards Track ("if you are going to do this, this is the interoperable way to do so") with "not recommended" ("but you probably shouldn't do it at all"). Discussion of documents that seem to require being misclassified is another impediment to rapid and efficient progress. It is disappointing that the IESG did not decide to address this issue. Conclusion: Whatever its other strengths and weaknesses, the proposal is patchwork that ignores a major component of the situation and should not be approved until and unless that issue is addressed. Suggestions on the IETF list, including, if I recall, some from IESG members, that the proposal should be approved in Maastricht are obviously inconsistent with the nature of the proposal and its relationship to actual cause and effect analysis. (6) The futility of using a small number of categories to describe a multidimensional situation: Status description versus status categories. It seems to me that much of the root problem is that we are trying to use a small number of labels -- Proposed, Draft, and Full in the current arrangement --to describe some rather complex categories. We've seen extensive deployment and use of Proposed Standards, some of which are rather badly documented. We've seen specifications widely deployed but based on a single reference implementation and therefore not meeting the interoperability tests for Draft. We've seen specifications held up because of quibbles about interoperability testing in spite of the fact that every participant in the IETF is using products based on those protocols on an everyday basis. And we have seen a huge amount of confusion about the relationships among various documents -- confusion that assignment of STD numbers doesn't solve, but at which they might do better if we assigned them earlier in the process (see #2 above). While it was widely misunderstood (and sometimes misrepresented), the original intention behind the "ISD" proposal was to gradually eliminate excessive reliance on maturity level categories, requirements categories, and STD numbers and the inherent rigidity of each in favor of textual descriptions of what we know, what has been examined, and what the relationships are. In retrospect, some of the requirements and mechanisms were probably applied/ required at the wrong times and too generally: there are probably situations in which nothing is needed, situations in which Doug's "cover sheet" proposal or something very much like it would be sufficient, and situations in which all or part of a full description is more appropriate. It is also worth noting in this context that the errata mechanism, while better than nothing, is not sufficient to solve any of the problems with the standards track... if only because even an "approved" erratum isn't an authoritative, community consensus document but merely a prediction of the consensus the community might reach if asked. An extension of the ISD model also provides a way for the community to say, by the usual consensus mechanisms, that a provision of a protocol spec is in error or ambiguous and that it should be read in some particular way (or replaced with particular text). For discussion purposes, I'm going to try to get an updated version of the ISD proposal --one that includes some of the thinking referred to above-- posted before the cutoff. (7) The process questions surrounding proposal, posting, and processing of this document. Independent of the content of this document, the procedure used to develop it, discuss it in the community, and presumably to reach conclusions about it seems to be to be seriously questionable. It is identified as derived from an IESG discusson. It was written by the IETF Chair and General Area Director, who has previously taken the position that process change proposals were unwelcome unless they met narrow categories, categories that were not widely exposed to the community much less the result of community consensus. It has been scheduled for plenary time at IETF 78 (by the IETF Chair and/or the IESG) and announced with timing that makes preparation of well-developed alternate proposals difficult. I note that the IESG discussions that led to this proposal apparently do not even appear in published minutes. Even if the IESG review is ultimately "sponsored" by some other AD, the fact that this proposal affects the way the IESG does business and is derived from IESG discussions gives it the appearance of an IESG proposal, scheduled for discussion by the IESG (or an IESG member) under conditions set by the IESG or that member, and then subject to consensus review and approval by the same IESG. This does not appear to me to be how we want to do things; certainly it calls the openness and transparency of the IETF into question unless we argue that IESG members are granted powers to make for the community and independent of any requirement for fairly-determined community consensus by virtue of appointment to their positions. We ought to be able to figure out a better way. Perhaps the development and advocacy for this document even illustrates that a different way to deal with process change proposals should be the next change that should be made to 2026. regards, john _______________________________________________ Ietf mailing list Ietf@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf