Hi John,
At 17:38 04-07-10, John C Klensin wrote:
(1) Analysis of the problem as stated in
draft-housley-two-maturity-levels-01
The draft indicates (Section 1) that:
"The proposed changes are designed to simplify the process
and reduce impediments to standards progression"
and then
"Since many documents are published as Proposed Standard
and never advances (sic) to a higher maturity level, the
initial publication receives much more scrutiny that is
call (sic) for by RFC 2026 [1]."
There is also:
"In practice the annual review of Proposed Standard documents after
two years has not taken place."
I understand that the ADs have a lot of work. Why it took 14 years
for someone to say that the annual review is not being done is a mystery.
However, if the issue is that documents rarely progress beyond
Proposed, then there is no evidence that tampering with the
relationship between the current Draft and Full Standard levels
will bring about any change in that relationship, i.e.,
increase the number of documents that move past Proposed. I
note, as others have noted, that the current de facto
requirements for Proposed are far in excess of what 2026
requires. If the problem is (as I believe it is at least in
part) that initial publication receives far too much scrutiny
and takes far too much time and energy, then formally no
procedural changes at all are needed: the IESG simply needs to
stop doing that and instead and apply only the level of
scrutiny that RFC 2026 requires, and no more. Such a change
would result in more rapid publication of protocol documents as
Proposed Standards (perhaps reducing the fraction of the
Internet that actually runs on I-Ds) so the community can
evaluate whether reducing the community exhaustion from getting
things to Proposed results in more documents moving to the next
level (whether that is one next level or one of two.
Agreed.
It appears to me that there are portions of the IETF community
who are willing to advance documents to Draft. Most of that
Yes, until they are asked whether it is worth the bother.
Here are some quotes from the IETF 76 plenary:
"Loa Andersson mentioned a working group that has been around for 13
years with one RFC that is at Draft Standard. We don't have a process
problem, we have a mindset problem. There is too much work required
to get a document to Full Standard."
'Ralph Droms has chaired a working group for 20 years with two documents
at Draft Standard and the rest at Proposed. When considering progression
on the maturity ladder, Ralph was once asked by an AD "is that really
where you want to spend your time?"'
"Larry Masinter asked do the documents accurately reflect what is
implemented and deployed? Lars Eggert responded why doesn't the
IESG get this question? What do you want the IESG to do? Larry
answered that he wanted a clearer process for defining a document for
Full Standard. Pasi Eronen reminded folks that the reason we are
getting these questions is because we badly screwed up the handling of
first draft from the YAM working group. We have a better idea going
forward."
Those who have been anxious to advance documents within the
existing model have often met significant resistance from IESG
members. It is somewhat disingeneous for the IESG (or even
some of its members) to resist efforts to move documents to
Draft or Full Standard, to place requirements on approval of
Proposed Standards that go well beyond those of 2026, and then
to complain about how few documents advance to Draft or Full
Standard and use those complaints as the justification for
abolishing the Full Standard level.
Based on publicly available information, I would say that the answers
lies in the IESG evaluation of
draft-ietf-yam-5321bis-smtp-pre-evaluation-04. The draft is a path
that precludes any appeals or recalls. It kills two birds with one stone.
On the other hand, the analysis in the document ignores the
observation that, independent of the interoperability
demonstration, moving a specification up in maturity levels
often results in significant improvement in document quality
and clarity. Again, it might represent an even larger
improvement if the IESG more closely adhered to what many of us
believe is the intent described in RFC 2026 and earlier
documents -- to get a specification published for examination
and implementation purposes as soon as all known technical
defects have been identified or eliminated but without
investing the resources to polish the text editorially. See
(3) below for more discussion on that subject.
The IESG does a lot of work polishing drafts because of the quality
of some of the drafts that gets to IESG evaluation. The work that
the IETF Community should be doing is being pushed up to the
IESG. The end-result is that the intent in RFC 2026 has fallen into disuse.
Conclusion: Whatever this proposal may or may not accomplish,
there is no evidence that it will solve the problems identified
in its Introduction.
You mean like "since we are not using this mechanism, let's get rid
of it instead of thinking why the mechanism is there".
(2) Promoting more use of Draft Standard (under any name) by
eliminating Full Standard and renaming.
YMMD, but I believe that the fraction of the community who now
believe that having something at Proposed (1st step) only is a
bar to implementation is fairly small. If it weren't, we'd be
in bad trouble as a consequences of the very active use of a
number of protocols that have never advanced beyond Proposed.
It is possible --although I have some doubts-- that rebranding
would help here. But, to the extent to which the need is to
rebrand, that does not, in itself, justify changing the
maturity level model.
I doubt that re-branding can change a mindset.
If one simply wanted to rebrand to reflect present-day reality,
we would change Proposed/ Draft/ Standard to Standard/ Standard
with Gold Star/ Standard with Gold Star and Network Cable
Wreath. Those titles are somewhat in jest, but the point is
that one wants to make the first level "Standard" and then
start talking about endorsements of higher specification
quality or assurance (see #6 below).
Yes.
Our real terminology/ branding problem is that "Proposed
Standard" has periodically been interpreted -- sometimes by
people or organizations with other agendas -- as "substandard
and really not suitable for deployment". In fairness to them,
that was exactly our intention in the early 1990s. That
intention was very much part of the "upgrade or die" language
in 2026, but has never been used. Leaving "Proposed Standard"
as part of our vocabulary doesn't solve that problem.
One alternative is to remove "Proposed Standard" from the vocabulary.
more information and discussion. I also note that BCP numbers
are sometimes used to cluster groups of documents just as STD
numbers are. In both cases, there may be issues with idea and
Yes, and that seems to have been forgotten.
(3) Conflating protocol quality with document quality
Advancing documents according to the current model actually
provides two separate (and separable) functions. One is
certification of some minimal proof of interoperability (Draft)
and utility (Full). The other is the specifications almost
always undergo significant improvements in the quality of their
documentation and description as people are required to
re-examine and re-edit them in the process of advancing between
grades. I suggest that the IESG is not taking enough advantage
If we expect clear, concise, and easily understood documentation, we
should re-examine the documents and re-edit them if necessary while
not destabilizing the protocol.
Either way, we should have a high document-quality expectation
at the second (and, if it exists, third) maturity levels.
Yes.
(4) Downward references
I don't see the evidence for "major cause of stagnation in the
advancement of documents" unless one relaxes the rule to permit
normative references to I-Ds (which I do not favor). If the
IESG believes that a document is being stuck unreasonably (or
is likely to get stuck), it has the ability to invoke the RFC
4897, which merely requires asking the community (in a Last
Call that can be combined with the Last Call on the document if
desired but that can also be applied later) and noting the
downref in the document. That is not an onerous procedure, yet
the IESG has never chosen to use it. If we really want to
The problem is that the IETF has been publishing BCPs that are never
implemented because they got forgotten along the line.
treat standards at the second maturity level (or third, if it
remains) as better and more completely-specified than those at
lower levels then the downref restriction should remain, with
the community able to make exceptions when the lower-level
document(s) are considered sufficiently mature and
well-specified. If, by contrast, additional maturity levels
are endorsements indicating that almost-orthogonal quality
levels have reached (as in the "gold star" recommendation of #2
above), then the downref issue may become irrelevant.
Conclusion: This proposed change solves a problem that has not
been demonstrated to exist, even to the extent of demonstrating
that the elimination of current mechanisms for permitting
downward references would save the IESG significant time.
It does not matter if there are Downward References for "Proposed
Standard". If the RFC referred to is mature enough, the Downward
Reference should not be an issue. It's not like the rules say that
"downward references are not permitted".
Conclusion: Whatever its other strengths and weaknesses, the
proposal is patchwork that ignores a major component of the
situation and should not be approved until and unless that
issue is addressed. Suggestions on the IETF list, including,
if I recall, some from IESG members, that the proposal should
be approved in Maastricht are obviously inconsistent with the
nature of the proposal and its relationship to actual cause and
effect analysis.
It is political expedient to resort to patchwork to avoid opening the
larger debate. As we take a narrow view of the issues, we lose the
bigger picture.
(7) The process questions surrounding proposal, posting, and
processing of this document.
Independent of the content of this document, the procedure used
to develop it, discuss it in the community, and presumably to
reach conclusions about it seems to be to be seriously
questionable. It is identified as derived from an IESG
discusson. It was written by the IETF Chair and General Area
Director, who has previously taken the position that process
change proposals were unwelcome unless they met narrow
categories, categories that were not widely exposed to the
community much less the result of community consensus. It has
been scheduled for plenary time at IETF 78 (by the IETF Chair
and/or the IESG) and announced with timing that makes
preparation of well-developed alternate proposals difficult. I
note that the IESG discussions that led to this proposal
apparently do not even appear in published minutes. Even if
the IESG review is ultimately "sponsored" by some other AD, the
fact that this proposal affects the way the IESG does business
and is derived from IESG discussions gives it the appearance of
an IESG proposal, scheduled for discussion by the IESG (or an
IESG member) under conditions set by the IESG or that member,
and then subject to consensus review and approval by the same
IESG. This does not appear to me to be how we want to do
things; certainly it calls the openness and transparency of the
IETF into question unless we argue that IESG members are
granted powers to make for the community and independent of any
requirement for fairly-determined community consensus by virtue
of appointment to their positions.
The author of the draft is the current IETF Chair. I have some
reservations about the IETF Chair driving such a proposal through the
process. Although the IETF Chair is also an IETF participant, it can
be perceived as problematic when the person writes a non-technical
proposal that has to be evaluated by the IESG.
This matter was discussed at the IESG retreat and the discussions
have not be published. I am not asking for them to be
published. But there should be a record in the IESG minutes if the
"Internet Engineering Steering Group (IESG) discussed the possible
ways of reducing impediments to standards progression".
I agree that draft gives the appearance of an IESG proposal. Nobody
would be foolish enough to write an alternative proposal. If there
were such a person, he or she would find it difficult to get an AD to
sponsor the proposal. According to AD Sponsorship guidelines, such a
proposal would have to be sponsored by the General Area Director
which happens to be the IETF Chair.
This is the second time in two months that we have seen such
"procedures" being used. Although there is nothing wrong with the
motives, i.e. to solve a problem, the way it is being done is plainly wrong.
Regards,
-sm
_______________________________________________
Ietf mailing list
Ietf@xxxxxxxx
https://www.ietf.org/mailman/listinfo/ietf