Geoff and Marshall,
I gather there has been a good deal of selective conversation
in the halls about this document this week. While I think I
agree with most of the symptoms and failures that drive your
analysis, my conclusions are somewhat different. Since I had
the opportunity to observe the behavior of the IESG as an AD in
the mid-90s, sat in on most of their calls and retreats for two
years more recently, and have spent the last months as an
outsider and sometime victim of the problems you identify, but
am not now part of the IESG or IAB, I've got some perspective
on these issues and how things play out in practice. I'm quite
unhappy with the IESG and the way things have been proceeding
(or not) lately, but I find myself reacting to some of your
observations and suggestions with the belief that they will fix
some problems while aggravating others and that others don't
address the right issues.
This note is rather long: I think it is important to look at
specific and real problems and issues. If those were simple,
I'm sure the IESG would have solved them long ago (I certainly
haven't talked with anyone on the IESG who thinks the current
situation is a good one).
As a preview of what follows, my observations have led me to
conclude that:
* We should always be careful what we wish for. To a
certain extent, we have insisted, even if only implicitly or
consequently, that the IESG behave in certain ways. In many
cases, they have tried to do so. We have then become upset
about the implications or side-effects of their conforming
to our wishes. We can make changes that do not change the
formal process, but should be quite careful that revised
expectations do not yield an outcome as unattractive as what
we have seen in the last few years.
* Some of the problems result from fundamental conflicts of
roles and responsibilities within the IESG. Unless we
change those conflicts, neither a new "PACT" nor new faces
are likely to result in significant long-term improvement.
And changing the conflict models probably requires changes
in the basic standards process.
* Some of the pathologies many of us have observed in
dealing with the IESG are the direct consequence of
otherwise- reasonable people trying to cope with overload,
and coping very badly. We must be careful that changed
expectations of the IESG do not increase the workload; if we
do increase the workload, it is reasonable to predict that
we will see different, perhaps worse, pathologies (or just
more of the same ones). And I believe that one or two of
your suggestions would have just that effect.
Disclaimer: Some of the ideas that follow have emerged from
conversations with several others over the last six months or a
year. I'll take responsibility for the bad ideas, but may not
deserve much credit for the good ones.
I. Issues with PACT (draft-huston-ietf-pact-00)
1. While I agree that the four qualities you identify are
important, I am particularly interested in the tension between
timeliness and competency. Every incompetent (or incomplete,
or inadequate, or irrelevant) protocol the IETF produces
damages our general credibility as well as just being useless
in itself. As a community, we also should understand that
there is no way that any body that depends on informed and
thoughtful consensus is ever going to be fast enough to get
ahead of (or even keep up with) a handful of aggressive
entrepreneurs who are determined to be first to market with
_something_, no matter what ends they leave loose. I believe
that this issue, and the appropriate balance between the two
goals, needs careful consideration by the community more or
less independent of other issues. It appears to me that many
of your specific behavioral proposals are strongly biased
toward speed, potentially at the cost of competence. If we
move in that direction, it should be after making an explicit
decision to do so.
We might even think about whether "we need to do something very
quickly to respond to marketplace pressures, even if it can't
get adequate development and review" should normally translate
into "no WG" rather than "hasty standard".
(2) We may be in agreement here, but it seems to me that, if
one imposes fixed, "short term" models on WGs, conditions based
on "time to approval" must involve meaningfully enforceable,
and realistic, requirements on the IESG for quick turnaround of
documents, questions, etc.
(3) In talking about operations, I believe that the IESG
actually has been given _three_ difficult tasks, the two you
identify (technical oversight and process management) and
standards approval and certification. I suggest that, in a
small --but historically very time-consuming-- group of cases,
those first two may actually be in conflict with the third.
See "suggestions for structural changes", below.
(4) Your proposed greater emphasis on area focus, which appears
to influence your proposed voting and several process steps, is
reasonable as long as the work of a given WG falls neatly
within a given area and is assigned to that area. In practice,
that has often not been the case: WGs have ended up in one area
rather than another because one AD has been particularly
interested or another has found it particular abhorrent, or
because of specific dangers of interactions with otherwise-
unrelated work, or as an arbitrary decision when the nature of
particular work could fall within any of several areas. And,
as long as we are serious about the importance of security,
scalability, and operability of Internet protocols approved by
the IETF, it is hard for me to reconcile those goals with a
vote-balancing procedure that makes it too easy for a WG in one
area to override objections of "insecure", "won't scale", or
"cannot be provisioned or operated on the public Internet
without damaging others" from other areas.
I would hope that we don't get increased area focus at the
price of lowering the competency of our evaluation process for
work with implications in several areas. Indeed, I believe
that, especially for the large fraction of applications work
that interactions operationally with other areas of IETF
expertise, we could figure out ways to increase, rather than
decrease, the amount of careful cross-area review.
More broadly, I have recently reformulated my view of what
the IESG should be doing as a standards-ratifier. I think this
view is consistent with the 2026 model, but hasn't been stated
this way. I would now say that individual ADs are responsible
for determining (in conjunction with WG Chairs) that WG
consensus has been reached, but that the IESG as a whole is
responsible for judging whether the WG's consensus output is
consistent with an overall competent IETF (and really
"internet technical community") consensus about reasonableness,
technical competence, scalability, interaction with other
protocols, and so on. If that model is reasonable, then
turning ADs into even more powerful monarchs over their area-
fiefdoms, and given them more leverage and advocacy
responsibility on getting products from those areas approved,
will, I believe, tend to reinforce some of today's bad
behavior, rather than decreasing it.
If our choices are only between bad behavior that is carried
out in secret and bad behavior that is public and well-
documented, I prefer the second and assume that all of us do.
But I would rather concentrate thinking and resources on ways
to eliminate or drastically reduce bad behavior and the
incentives for it. Otherwise, we end up too dependent on
deliberately-ponderous recall procedures or may have to wait
most of two years to deal with someone who has responded badly
to the particular type of stresses the IESG represents.
(4) There is a class of WG for which the "bounded outcome"
model will, fairly clearly, fail. And, unfortunately, such WGs
seem to be on the increase. It has become common to have a
situation in which a group of people with narrowly-focused
interests come together and insist, quite loudly and
persistently, that they want to do a particular piece of work
within the IETF. Such groups are often approved by the IESG:
whether to give them a chance, or because turning them down is
too painful, or because the work might actually be useful.
But, unless we can devise rules that prevent such groups from
being chartered, or that kill them immediately if they cannot
involve a broad spectrum of the IETF community in their work
(and involve them actively), then presuming that their output
represents community approval, is extremely dangerous to the
goal of producing only IETF protocols that are competent on the
public Internet. My observations of such groups is that it is
often difficult or impossible to get them to focus on even the
applicability (or security or scaling) boundaries of their
work; it is difficult to hold the time delays that occur when
the IESG identifies and tries to remedy those problems in order
to produce a competent, or competently-bounded and documented,
protocol as an IESG failure because of excessive processing
time. And rewarding IESG inability to turn such documents
around quickly with approval of the work as standards (which is
not what you proposed, but might be the effect) is not the path
to having all IETF standards be competent.
The question of whether potential WGs for which there is only
narrow, but intense and articulate, interest should be
chartered at all is another one that I believe the community
should debate carefully and openly enough to give the IESG
clear guidance: the answer is not obvious, and the IESG should
not, IMO, be setting this sort of strategic policy for the IETF
in internal and largely-secret discussions. At a minimum, the
IESG should propose a policy for public review, rather than
making invisible internal policies.
(5) Some of the provisions of your proposal are built around
the notion of the IESG "rejecting" a WG document. In practice,
that almost never happens, at least partially because of some
of the "conflict of roles" issues discussed elsewhere. I
personally believe that it should happen more often, but it
doesn't with the current procedures and personalities (or any
set of the latter in recent memory). Instead, the IESG
(implicitly or explicitly) tasks the relevant AD to go into a
negotiation process with the WG (or at least with its Chair
and/or Editors). That negotiation process is somewhat dubious
under the current procedures (it lends itself to important and
substantive decisions being made in secret and without adequate
opportunity for competent WG review and interaction) but,
unless we figure out a way to get rid of it, it would be the
end of some of your time limits as outlined.
II. Suggestions for Structural Changes
Some of these suggestions require changes to our basic
standards-producing procedure, some do not. And some might be
supplemental or complementary to some of your suggestions. I
believe that at least one of them is a seriously bad idea, but
one that we need to revisit periodically to make sure we
[still] understand and agree with the tradeoffs involved. I
haven't differentiated among them -- my goal is not to make a
counterproposal to yours but to open up the discussion to a
broader range of options and alternatives.
(1) Workload.
The current workload of the IESG is incredible, very nearly or
more than what most people would consider a full-time job if
all of our expectations are to be met. That workload, the huge
responsibilities we have placed on the roles, and the many
complaints about their priorities and what doesn't get done on
the schedule we would like, create huge levels of stress on any
IESG member who cares (which, in a typical year, is all of
them). The stress, in turn, tends to bring out the worst in
many people: someone who could manage things very well at a
lower stress level is likely to start taking shortcuts -- in
review processes, in openness about what is happening, in
explanations of why things are done or about technical
objections, in procedures, etc. -- when the stress and time
pressures get high enough. If we don't like that, we need to
change the problem (and probably the job description): more
rules and guidelines may make things worse rather than better.
(2) The conflicts of roles
The IESG ended up with responsibility for authorizing/
chartering WGs, managing the WGs and process, providing
technical supervision, and approving their outputs onto the
standards track after the Kobe meltdown and the subsequent
Boston Tea Party. That result was somewhat reactive and, while
it was probably the right solution at the time, it may have had
some unintended consequences that we should, a decade later,
review. We had best not expect the IESG to do that review for
us: they haven't been able (or willing) to focus on it during
that decade, they haven't been able to define their role in a
Charter that the community could review, and, if anything, the
number of "policies" they create internally and hand down from
on high seems to be on the increase. But this is also
symptomatic of what I see as a conflict of roles: it is
unreasonable to expect the IESG to both manage the standards
process -- and do all of those other nice things -- and to
establish procedures that the rest of us will be happy with.
_We_ need to make the decisions about what we want from the
IESG, not have them guess, or, worse, devise procedures that
meet their convenience but not our needs.
But the problems of conflicts are more general. As suggested
above, a good deal of the interactions between WGs and ADs,
especially late in the process and more especially when a WG is
perceived of as being in trouble, involves the AD getting
intimately involved in the WG. We've seen ADs writing sections
of documents, mediating between various factions in the WG,
acting as go-between between the WG and other activities. and
so on. Resources permitting, I think this is all to the good.
But expecting that same AD to fairly and effectively walk into
the IESG and make standards decisions that represent the whole
community just ignores human nature and raises the stress level
excessively. At some stage, I think we need to revisit and
carefully reevaluate whether having all of these functions in
the same body is an optimal idea. Note that shifting either
the day-to-day WG technical and process management ("adult
supervision") role, or the standards approval one, away from
the IESG would significantly reduce the IESG workload.
Having ADs as liaisons (or other representatives) to or from
other bodies creates another area of procedural conflict. If
we want openness about what is being said to those
organizations on IETF's behalf, it is certainly best to
establish boundaries so that people aren't talking [only] to
themselves about the relationship between those external roles
and the AD one.
(3) AD as advocate before the IESG
There is another element of the "ADs get too involved with
their WGs to properly evaluate their work for standardization"
model. Especially with the narrowly-focused groups described
above, there is a reasonable chance that, at the point that a
WG decides it has completed its work, the AD is disappointed
and frustrated with the quality of that work. Reasonableness
requires that the WG have an opportunity to have its output
considered by the IESG, but that now requires that the AD act
as advocate for the work. That is an unreasonable demand, and
the consequences stress the AD, represent the WG poorly, or
both. For this and other reasons, it might be reasonable to
shift the presentation and advocacy burden to the WG Chair,
holding that person responsible for preparation of the protocol
summaries and draft action notices (some ADs have required this
in the past, but it is not the norm) and then, on a scheduled
basis, presenting the WG's work to the IESG, responding to
questions, etc. This would almost certainly require the IESG
to get more organized about its agenda -- e.g., it would make
the "I haven't had time to read this" deferrals even more
destructive than they are today, but might increase some
aspects of IESG workload and stress -- but might be worth some
evaluation.
(4) Things the IESG doesn't need to do is
In the absence of an IESG Charter, RFC 2026 and 2028 provide
our only real guidance to what the IESG is supposed to be
doing. Almost all of those things have to do with advancing
standards. But the IESG has, for an assortment of usually-good
reasons, taken on a series of other tasks, some of which become
higher priority than standards processing and WG management. If
we want more productivity on standards documents out of the
IESG, we need to reduce that supplemental task list, ideally to
zero, by moving the tasks elsewhere. For example:
(a) The IESG role in evaluating individual-contribution drafts
proposed as informational or experimental RFCs has evolved from
"none", to "reviewing documents to be sure that they don't
end-run WG activities or interfere with other work", to that
review to that review plus reviewing documents for general
competence, to taking it upon itself to engage in rewriting and
revision negotiations with authors. If we want them to
maximize the efficiency and timeliness of standards processing,
we need to figure out how to deal with these other tasks in a
way that reduces their role, responsibilities, and the time
they spend on it. Or we can decide that we like it this way,
but then we had best not complain when one type of document or
the other is delayed.
(b) A decade ago, it was routine to hand registration
responsibility associated with a given protocol off to the IANA
with only the vaguest instructions and guidelines. The IANA
dealt with it (typically well, but better with some things than
with others), and came back and asked for guidance when needed.
When the IANA decided it needed an advisory committee around a
particular topic or registration area, it organized one. We
are now working with an IANA with a considerably different
internal structure and skill level, which inevitably shifts
responsibility for structuring registrations and evaluating
their appropriateness back to the IETF. That is probably A
Good Thing, and increasingly-specific "IANA Considerations"
sections of RFCs are useful for both the relevant standards and
the IETF. But many of those sections call for experts to
review registration requests, and IESG has taken many of those
expert roles onto itself or onto particular ADs. Perhaps that
is a bad idea: it is additional work that the IESG is not
required to do (and may not even be required to manage) and we
might have a clearer test of a standard (or other document) and
its instructions and specifications about registration if the
person taking the lead in approving the document was not the
same as the person who would interpret it.
(c) The RFC Editor has traditionally been a key part of our
standardization, publication review, and sometimes technical
editing processes as well as performing the function of a copy
editor and publisher. As mentioned above, the IESG has
determined that it needs to be deeply involved in those
processes, especially at the level of in-depth technical review
of individual contributions. We should consider whether to
separate WG-created informational and experimental documents
(and other documents the IESG determines are directly related
to standards-track materials) from other types of materials,
and really handle the other materials in another way, with
minimal IESG review for non-conflict / non-end-run and absence
of disasterous interactons. If the RFC Editor needs a
supplemental editorial board, we should look into ways of
creating one, not have the IESG assume that role by default.
(4) The IETF Chair, roles, and overload
Just as the role of the IESG is undefined by any formal
document that has been approved by the community, the role of
the IETF Chair, and our expectations of it, is badly
underdefined. That has resulted in a series of "everything not
assigned elsewhere that, in the Chair's opinion, needs to be
done is the Chair's responsibility and under his or her
authority". So the IETF Chair is now an AD of an area who is
responsible for several working groups, IESG Chair, public
representative of the IETF, meeting planner, secretariat
super-director, occasionally king, and probably a few other
things. If we don't like it, we had better fix the role
definition, not [just] the person involved.
(5) Staffing the process
One way of alleviating IESG load is to add serious staff to the
process. This would bring us all of the advantages, and all of
the risks, of a professional standards secretariat. The risks
include having the secretariat, rather than the volunteers,
take over the process generally and the queue into the IESG
specifically. But the model would be that we hire a few
people, probably students between masters and doctoral work, on
short-term (a year or two) appointments and assign them to a
few areas areas each. Their job would be to read I-Ds on an
ongoing basis and alert the ADs when something needed special
attention or intervention. When a document appeared ready for
IESG processing, they would check it carefully against IESG
criteria and internal work, bounce it back to the WG if there
were obvious deficiences, and, otherwise, prepare a case for
advancement for the IESG. We may continue to not want that
type of situation or character of professional staff, but it
would certainly change the complexion of the IESG job and its
workload.
Similarly, some considerable IESG time goes into documents are
submitted that are really not clear enough, from an editorial
standpoint, for review. Having IESG members doing copy-editing
cannot be a good use of their scarce time. Given resources, we
could employ technical editors who could flag the
editorially-inadequate documents and work with the WG to get
them fixed; fixed before the IESG got involved in a detailed
evaluation. If nothing else, documents that are well-written
tend to be easier and faster to read, and that too would save
the IESG time.
(6) The form of pushback - objections going to the WG vs a
private negotiation process.
Finally, there is typically a huge difference between
-- pushing a flawed document back to a WG with a clear
indication and explanation about what points need to be
examined and fixed and
-- working in secret with document editors or WG chairs to
refine a document, then passing it to the WG as a finished
product.
We have been doing too much of the latter; we need to do much
more of the former. And, again, this would, in the long term,
save IESG time since having ADs rewriting documents is rarely
an efficient use of their time.
I hope we can all work together to bring about real change and
solutions, not just new rules that, given the same pressures
and roles, will result in deteriorating behavior -- whether
similar or different to the deteriorating behavior we have
today.
regards,
john