>>>>> "Harald" == Harald Alvestrand <harald@xxxxxxxxxxxxx> writes:
>>
>> The primary reason you want to encourage meta-experiments is that a
>> lot of the hypotheses you want to test involve delegation. For
>> example I want to test the hypothesis that the right way to solve the
>> mailing list mess is to delegate it to the IESG. I could delegate it
>> to the IESG as part of an experiment along with a initial procedure.
>> But if I do that I'm testing a different hypothesis: is delegating
>> something to the IESG with the whole IETF designing the initial
>> conditions a way to solve the problem. As you might imagine that's a
>> different hypothesis.
Harald> I do not believe that is a correct statement.
Harald> An RFC 3933 experiment by its very nature involves three different
Harald> sub-experiments:
Harald> - Whether you can get the IETF to agree to running this experiment
Harald> - Whether the bodies that administer the experiment can perform the
Harald> experiment so that we have something to measure
Harald> - Whether the experiment produces useful results
I agree with this characterization.
I don't see how this characterization is in conflict with mine.
Harald> I think what you are proposing will lead to failure of the *first*
Harald> experiment, which means that you won't get to the other two.
Harald> If you say "give the IESG the power to produce procedures, we're not
Harald> going to give you even a hint as to what these procedures might be",
Harald> this is what I would characterize as selling a pig in a poke, and it's
Harald> perfectly reasonable for the IETF community to ring up a "no sale" on
Harald> the consensus register.
The IETF certainly may do that. You clearly believe the IETF should
do that. I believe the IETF should not do that.
Ultimately Brian will have to make a consensus call.
I think it is reasonable to delegate comuing up with procedures to the
IESG. I think that if we're going to delegate that then the
procedures should not be discussed in this draft.
If the community wants more oversight, here are some things I think would be reasonable to ask for:
* Add criteria the IESG will use for evaluating procedures or for
creating procedures.
* Requiring proponents of this experiment to submit candidate procedures that demonstrate procedures are possible
* Adding explicit review requirements for procedures that the IESG proposes.
* Establishing principles the IESG should use
If you believe that one of these options would help, I invite you to
propose text or direction.
You could also propose an experiment to run with specific procedures.
I would not object to that experiment but I would also not be
interested in contributing to it.
Harald> The theory that there is a different hypothesis being tested if the
Harald> IESG sets the initial procedures than if the initial procedures are
Harald> part of the experiment also seems doubtful to me; in both cases what I
Harald> think will happen is that the IESG will start off with procedures
Harald> designed by Sam Hartman, and the big difference is that the community
Harald> will have seen the initial procedures before deciding to run the
Harald> experiment.
Harald> The whole IETF has never designed anything and never will; all design
Harald> is done by specific people, and (hopefully) reviewed by some
Harald> large-enough fraction of the IETF. That's how EVERYTHING in the IETF
Harald> works.
>> Since I'm on the IESG I'm actually in a
>> reasonably good position to negotiate an initial procedure that the
>> IESG will be happy with and that would be similar to a procedure the
>> IESG would come up with on its own. However we want 3933
>> experiments--even experiments delegating things to the IESG--to be
>> documents that anyone can write. So we should require that authors of
>> 3933 experiments demonstrate stakeholder buy-in for experiments but
>> not require that they take actions as if they were the stakeholders.
>>
Harald> I do not believe this theory either.
So, the main point of the paragraph here is that I don't believe only
IESG members should be able to propose experiments that change things
for the IESG. I hope we can agree on that.
Harald> RFC 3933 talks about the IESG and the community. The "stakeholder"
Harald> concept is of your introduction, and I believe it serves no benefit.
Harald> An RFC 3933 proposal author HAS to do three things:
Harald> - Write up what he's proposing in enough detail that it can be evaluated
Harald> - Convince the IESG that the experiment has enough merit to warrant a
Harald> Last Call
Harald> - Generate enough review and comment in the community that the IESG
Harald> can confidently say that there's community consensus to run the
Harald> experiment
I proposed the concept of stakeholder buy-in as an attempt to solve
the first criteria. I think that the IESG should almost always last
call experiments when the proponets can demonstrate it is possible to
do the work and when those effected are interested in doing the work.
I think that using this criteria will help resolve long-standing
questions of where to focus process reform energy and of what people
need to do in order to move their proposals forward. I think that the
IESG should focus on process reform where the stakeholders are
interested in the work over process reform where the stakeholders are
not interested.
I do agree that when the community really desires some reform and the
stakeholders are not interested, we need some resolution that allows
the reform to proceed possibly with resignations. I think the bar in
that case needs to be much higher.
But my concept of stakeholders is more about establishing a criteria
for forward progress than about blocking things.
Harald> Certainly, a community feedback that says "I'm one of the people
Harald> required to do work under this experiment, and I'm not going to do it"
Harald> is a strong statement that needs consideration when assessing whether
Harald> there's IETF consensus to run the experiment. But sometimes that's
Harald> just part of the "rough" in rough consensus, even when deciding to run
Harald> the experiment will cause someone to resign his role. And sometimes
Harald> it's not.
>> The second reason that you want to allow meta-experiments is that we
>> want to encourage RFC 3933 as the first step in process change.
>> Process change often results in BCPs. You want the 3933 experiment to
>> be reasonably similar to a BCP so that when appropriate you can easily
>> convert a successful experiment into a BCP. You would probably
>> replace any evaluation criteria with results of the evaluation,
>> replace the sunset clause with something else. However you want the
>> operative language to remain the same. A significant result of the
>> mailing list discussion is the concern that our BCPs are too specific
>> and encode operational details. If you require that meta-experiments
>> are not allowed you strongly push us in the direction of
>> overly-specific BCPs. I think that would be a very bad idea.
>>
Harald> Here I agree, but disagree that this document does an appropriate job of it.
Harald> I think the separation of "principle" and "process" is a good thing -
Harald> the principle of this document being (I think) that the IESG sets
Harald> mailing list management procedures, and the process being the process
Harald> by which such procedures are set, and the third level being the
Harald> procedures themselves.
Harald> However, just stating the principle alone at an abstract level is not
Harald> enough to either start or evaluate the experiment.
I disagree. If you'd provided more detail on why you don't think
that this document is sufficient, I could respond to it. However I
think it might be useful to see what the consensus position is before
we spend a lot of time on this issue.
--Sam
_______________________________________________
Ietf@xxxxxxxx
https://www1.ietf.org/mailman/listinfo/ietf