This has been an interesting discussion, but somewhat one-sided. Authors, editors, and WG Chairs and participants tend to believe that they produce perfect output, but that is demonstrably not always true. Although the IESG supposedly provides a quality control, it's clearly not working. Rather than vague assertions, I'll mention a couple of specifics: RFC 4021 and draft-fax-esmtp-conneg. RFC 4021 has been published as Standards Track. In accordance with RFC 2026, it contains a section referencing RFC 2119 and the latter's keywords which indicate levels of requirements. However, none of those keywords appear anywhere in the document but the section stating that 2119 in fact defines such keywords! Moreover, the document does not impose any protocol or format requirements and does not lend itself to any implementations (its purpose is to document items being included in an IANA registry); the Standards Track is the wrong category for such a document -- there is no way that it can advance to Draft Standard. This is a spectacular and surprising instance of complete failure of the authors, the IESG, and the RFC Editor to control quality. It is surprising and spectacular because the document structure is so simple and its purpose evident. There is no way that the IESG and RFC Editor should have categorized that RFC as Standards Track. There are many issues with the draft mentioned above; despite having ABNF expertise among the authors, there are at least a half-dozen specific problems with the ABNF. The document lacks discussion of interaction with directly-related protocols (ESMTP submission, e.g.), has no BCP 90 registration forms for message header fields defined, no discussion of internationalization issues, and an inadequate IANA considerations section. The draft has been through at least one Last Call (version -09 and possibly also version -02), and has been approved as a Proposed Standard (-13) despite the known technical omissions, the vagueness which precludes the specification from being well-understood, etc. This is another failure of quality control; the document does not meet the minimal requirements for Proposed Standard (RFC 2026 section 4.1.1). There are clear requirements for the Standards Track; apparently the authors, WGs, the IESG and the RFC Editor aren't taking the necessary time to review documents for compliance with those requirements. I have separately (NEWTRK mailing list) suggested that those requirements should be distilled into a checklist, and have further suggested that some tolls might be capable of highlighting specific problems such as the RFC 4021/2119 issue. Clearly, I'm bemoaning the lack of quality of recent work; others have complained about timeliness. Still others have mentioned relevance and comprehensibility. There have been some allusions to cost (in the form of increased manpower for review). There is a well-known engineering issue: o Quality o Timeliness o Cost Pick any two. Part of the problem with time/quality tradeoff is that many drafts are in such a poor editorial state that it impedes review. One possible way to expedite the technical review would be to ensure that documents are edited for readability and comprehensibility prior to IESG review and Last Call. That may involve cost (somebody has to provide editorial advice). Or the IESG could simply push back documents of poor editorial quality (ideally with a list of resources -- currently scattered among RFCs, presentations, semi- official documents (ID-guidelines, ID-Checklist, ID-nits), etc.). RFC 3774 has been mentioned in this discussion; that document and RFC 3844 discuss some problems and potential solutions, but do not address the generally poor quality of documents entering the review stage. Some of the issues identified have been partially addressed (e.g. RFC 3935 provides the mission statement mentioned as lacking in section 2.1 of RFC 3774 -- but the IETF web site makes no mention of the IETF mission and has no direct link to the mission statement). 3774 also bemoans a perceived lack of assessment based on practical experimentation, ignoring the fact that most RFC categories specifically provide for such practical experimentation (i.e. the Standards Track plus Experimental (and Historic for failed experiments)). 3774 also makes the extraordinary claim that the quality bar has been raised. As mentioned above, a part of the quality problem can be attributed to lack of organization of relevant material in a single place. The NEWTRK "ISD" effort might be able to help -- but as noted above, many of the relevant information is in documents that are not RFCs, and which may have no clear official status, and moreover, which may change frequently. However, there is a danger that the benefit of consolidating pointers to related information may be diluted by incorporation of unrelated agendas (specific document format issues), and the danger of causing a further deterioration in quality (by weakening the iterative improvement process embodied in progression along the Standards Track). And it's not at all clear that making information more readily accessible will improve the quality of documents; the examples of quality control failures given above cannot be attributed to ignorance of relevant requirements. Some relevant information could be more widely disseminated with clear benefit; for example, some boilerplate pointing to the RFC Editor's errata page could be included in every published RFC, informing readers of the mechanism for documenting relevant updates. One problem is that the IESG routinely sabotages development along the Standards Track by disbanding WGs as soon as a PS is produced, leaving nobody to do the work necessary for advancement to Draft. Charters should probably explicitly provide for WG activity leading at least to Draft Standard (or to Historic if the necessary two independent implementations fail to develop within a reasonable time). _______________________________________________ Ietf@xxxxxxxx https://www1.ietf.org/mailman/listinfo/ietf