--On Friday, 29 July, 2005 11:43 +0200 Brian E Carpenter
<brc@xxxxxxxxxxxxxx> wrote:
> Of course, my second sentence is written backwards. Duh.
>
> I *really* think that combining them as in the -00 draft,
> rather
> than separating them out, somewhat confuses the issue.
Let me explain why I did it, and, in the process, quibble with
your characterization. That doesn't mean the two should not be
split up -- I just want to explain my reasoning.
>...
>> My take is that there are two main aspects to John's proposal.
>> I think that separating them out, rather than combining them
>> as in the -00 draft, somewhat confuses the issue.
>>
>> 1. Defined criteria for appointee performance, to be
>> objectively evaluated towards the end of their terms.
>>
>> I think this is an excellent idea - just like annual
>> evaluations that most of us have in our day jobs, this would
>> be useful for everybody, including new ADs, incumbents who
>> are reappointed, and incumbents who are not reappointed and
>> want to know why.
Yes, but...
>> As I already noted, we can't switch this on from one day to
>> the next - the evaluation criteria have to be set up first,
>> with the evaluations applied later. And the criteria must be
>> fair with respect to the job description and the workload.
I carefully didn't say "defined criteria". I think that, for
the IETF, while "defined criteria" are certainly a desirable
target in principle, they lie on the road to madness. In your
day job, you presumably have a more or less specific job
description. There are clear, and presumably fairly efficient,
ways to change that job description when that becomes necessary.
And evaluation criteria typically track the job description.
With the IETF, things change and we expect the various entities
involved to adapt. The intent behind the question I asked you
offlist about the authority for the IESG's "back-stop" role
wasn't about what RFC 2026 says, but about how it has been
interpreted to give individual ADs the authority to block
anything, or even hold it for individual review. For me, and I
would suggest for the community at the time the introductory
text in section 6 of 2026, was written, the key phrases in that
paragraph are at the beginning of the last sentence:
"...experienced collective judgment of the IESG" and "concerning
the technical quality". To me, those phrases put significant
constraints on actions (or blocking non-actions)
of individual ADs or on judgment based on personal taste and not
experience. They also put significant constraints on IESG
_editorial_ nit-picking about issues that do not significantly
impact technical quality. From that perspective, things have
gotten somewhat out of balance. The "Discuss" draft is, in that
regard, very helpful in starting to improve the balance,
although it is probably not the only piece of the puzzle.
But the point is that the fact that the IESG can interpret that
text differently than I do, or even differently from the way a
large fraction of the community does (were that the case), is,
as far as I am concerned, A Good Thing. Circumstances change
and our ability to make adjustments, nearly in real time, to
reflect those changing circumstances, is vital. I believe that
insisting that the IESG discuss every possible change and
adaptation with the community at length before making it would
be, well, silly... and perhaps fatally silly. I believe that
the IESG does need to be much better at informing the community
about decisions and changes made (that has improved
significantly over the last year or so, but still isn't where,
IMO, it should be) and to provide realistic opportunities for
feedback, course corrections, and rebalancing from the community
(areas in which we are, again just IMO, doing less well).
But the flexibility needs to be there. To some extent, what we
tell the IESG and the nomcom is "you understand the general
objectives, please apply the criteria needed to make things
work". The nomcom is not expected to pick people out, or
evaluate them, based on some fixed criteria. They are expected
to figure out what the community needs at a given time and make
decisions accordingly. Their questions to the community, IMO,
should focus at least as much on "what does the community or
this area need" as on "is this person suitable" (at least
judging from questions asked, they do pretty well at that).
So, from my point of view, thanks, but I deliberately didn't
propose the development of specific criteria that have to get
locked in place, be predictable, and require process to
institute and change. I do not consider, e.g., "a one-term AD
should be returned for a second term unless that person has
really not worked out" to be a specific performance criterion.
Even though it is fairly specific guidance to the nomcom as to
how to proceed, the interpretation of "really not worked out" is
up to them... and should be.
None of that reduces the value "for everybody, including new
ADs, incumbents who are reappointed, and incumbents who are not
reappointed and want to know why".
And, as a side benefit, if we don't go off into a process of
developing a plan about how to establish objective criteria and
evaluate them, then the amount of work that is required to get
this going is _very_ significantly reduced.
>> 2. Strong guidance to the NomCom about how to strike the
>> correct balance between continuity and turnover.
>>
>> I think giving such guidance to NomCom is a good idea on
>> balance. We have running code proof that the weak guidance in
>> RFC 3777 leads to some NomComs going for a lot of turnover
>> and other NomComs going for almost none. I remain against
>> rigid term limits, but guidance that says, basically "you
>> need a good reason *not* to re-appoint a 2 year AD, and you
>> need a good reason to re-appoint a 4+ year AD" seems better
>> than what we have today.
>>
>> The advantage of separating the criteria/evaluation part from
>> the actual nominating/appointing part is that this will avoid
>> contaminating the evaluation with concerns about turnover. It
>> could even be done by a separate team, perhaps.
>>
>> Another advantage is that point 2 is completely compatible
>> with RFC 3777 as written. Whereas point 1 needs quite some
>> work.
I agree with all of that, but that brings me to Rich's note and
the reason these two "aspects" were written together.
>> Richard Draves wrote:
>>> [I'm sorry to be joining this discussion late.]
>>>
>>> I see several different goals in John's draft:
>>> 1. Setting guidelines for length of service.
>>> 2. Early notification to incumbents.
>>> 3. Reducing the nomcom's workload.
Those might be the goals, somehow, but the second and third were
not intended except as accidents. I would have stated those
goals as (reversing the order for reading convenience):
3. Not increasing the nomcom's overall workload so significantly
that the increment becomes a problem. While reduction of
workload is always A Good Thing, my objective in terms of Nomcom
workload was to suggest that the two-stage process would not add
significant work and would permit better focus on the efforts on
evaluations and decisions that mean something.
2. Early notification is, again, a nice side-benefit, but was
not a goal. The goal was to eliminate or dramatically reduce
three problems... with the understanding from your note that we
disagree about whether they are problems:
(i) Recruiting people to run against incumbents who will almost
certainly be returned anyway is bad strategy. Jeff Schiller has
pointed out one of the problems. Another is that people (or
their organizations) gradually get discouraged and won't make
themselves
available year after year. Things were a little different in
the old days, when the assumption was that the nomcom would know
most of the plausible candidates and vice versa. But today,
with burdensome and privacy-intrusive questionnaires,
double-interviewing, and so on -- each desirable under the
circumstances of nomcoms today-- just being a potential
candidate (incumbent or not) involves a non-trivial time
commitment. Even if there were no long-term effects, it is not
desirable for the community to have people who could be doing
technical work diverted into candidate preparation if selection
of someone else is a foregone conclusion.
(ii) Realistically, the model for evaluating incumbents and that
for evaluating other candidates should be different. The
incumbents are, presumably, a known quality and what one sees is
a much better predictor of what one is going to to get than any
amount of questionnaires or interviews with the incumbent.
Conversely, asking other IESG members, WG chairs in the area,
etc., about performance and behavior of an incumbent involves
looking for real answers about real behavior, not speculation
about how someone might perform as an AD. As just one example,
today, fairness just about requires that an incumbent fill out
the same questionnaires and answer the same questions, as a
possible candidate no one has heard of. I've come to the
conclusion that is wrong: the situations are different, we
should treat them differently, and we should be loading as
little additional work on, e.g., overextended IESG members, as
possible... especially additional work that arises only in the
interest of being fair.
(iii) A separate review process, with strong guidelines about
retention and rotation and a focus on incumbent evaluation,
would significantly reduce the sense of being rejected by the
community among incumbents. That sense is, in my opinion, a
significant contributor to our tendency to lose those who have
been involuntarily retired from the IETF, at least for some
years. We cannot, IMO, afford to lose even one person that way
who has had the skills and experience to work even moderately
well within the IESG and who has acquired that experience.
While the numbers are lower, I would also assume that it would
encourage some incumbents to make themselves available for
another term when, otherwise, they might just drop out to avoid
going through the semi-public candidacy process.
So the proposal is that the nomcom evaluate incumbents and,
using more explicit (and therefore consistent) guidelines about
the balance between rotation and continuity than we have had
before, decide whether to return them. Then that gets
announced, and new candidates are sought only for open positions.
>>> I think giving the nomcom more guidance about appropriate
>>> length of service is a fine thing. (And I tend to agree that
>>> service beyond three terms should be unusual.)
FWIW, the proposal would make service beyond _two_ terms
unusual. Sorry, but, in Internet time, typical tenure in day
jobs, etc., six years is a _very_ long time. The proposal
argues that rotating IESG members back into the community after
four years is a positive benefit; much of that benefit is lost
if we consider six years the norm and time beyond that
"unusual". The proposal, for the first time as far as I can
remember in anything that has gotten as far as an I-D, takes the
position that rotation is at least as important as continuity,
in most cases more so.
>>> Having a
>>> generally-recognized length of service would help generate
>>> candidates to replace a well-regarded AD who has served
>>> several terms.
Knowing, for certain, whether than AD is going to serve another
term or not would, IMO, do a much better job of this.
>>> I think announcing early decisions regarding incumbents is
>>> not good. I think there's great value in looking at the
>>> slate of candidates as a whole. I don't like the idea of
>>> making a decision about an incumbent without seeing the pool
>>> of potential replacements.
This is where we disagree. I think beauty contests that involve
reasonably-well-regarded incumbents in competition with other
candidates are bad for the IETF, bad for the incumbents, and bad
for the alternate candidates. This is not about a sporting
event, it is about a selection mechanism for someone who can
(and will) do the job for a while, in which incumbents are known
quantities and others are not, and in which, if there is
insufficient leadership depth in an area to find decent
candidates, or even to replace an AD who has turned out to have
significant unattractive behaviors, that is a problem the IETF
and, specifically, the IESG, should be looking at seriously, not
leaving it to the nomcom to try to make a least-bad choice. I
actually consider it a positive situation for the nomcom to need
to make a "do we want two more years of this" decision about an
incumbent before figuring out whether better options are
available.
>>> The goal of simplifying life for the nomcom is laudable, but
>>> in fact if an AD incumbent is doing a good job, there are
>>> generally very few credible & willing alternatives for the
>>> job so it doesn't take many cycles to sort out the
>>> situation.
But you say about that you don't want to evaluate incumbents
without knowing what is in the pool. If you are taking a small
number of credible and willing volunteers for the pool as an
indication that the incumbent is going a good job, I suggest
that is an even stronger reason for evaluating and deciding upon
the incumbent first. If you are not, but instead not
considering the incumbent until there is a pool to compare him
or her with... From my point of view, both versions lead to a
strong case for handling consideration of the incumbent
separately. And, again, simplifying the life of the nomcom was
not a goal -- better quality decision-making, better retention
of former IESG members in the IETF, better quality and more
diverse candidate pools when they are needed, and effective
rotation were.
You may, of course, still disagree and probably do.
regards,
john
_______________________________________________
Ietf@xxxxxxxx
https://www1.ietf.org/mailman/listinfo/ietf