Dave, On 2011-07-29 13:53, Dave CROCKER wrote: > > On 7/28/2011 7:22 PM, Brian E Carpenter wrote: >> Dave, we are shouting past each other so I will not repeat myself >> on all points. However, > > Brian, > > I did not ask you to repeat anything -- and don't want you to. > > Rather, I asked you to move beyond cliche's and personal anecdotes into > consideration of tradeoffs. That is, I pointedly asked you /not/ to > repeat yourself. > > Please engage in the substance of such balanced analysis comparing > benefits against costs. It's not as if that's an unusual approach for > evaluating expensive activities of questioned value... May argument is that if we move the cost (that of final review prior to committing the text to the RFC Editor), we do not remove the problem (of the final review being exhaustive and time consuming), unless we also lower the implied standard of quality. To my mind it's implicit in RFC 3935 that the present quality standard is not lowered - especially, that in addition to verifying that a specification is valid in itself, we verify that it does not damage other aspects of the Internet. That makes final review by a cross-area team essential, and today that team is the IESG. I wouldn't mind moving the responsibility to some other cross-area team, but it seems like a zero-sum game to me. > >>> Have you seen a pattern of having a Discuss cite the criterion that >>> justifies it? I haven't. It might be interesting to attempt an audit >>> of Discusses, against the criteria... >> >> It might, and at the time when the IESG had a large backlog of unresolved >> DISCUSSes and the current criteria were being developed (I'm talking >> about 2005/2006), the IESG did indeed end up looking at all the old >> DISCUSSes against the criteria, and held dedicated conference calls > > I do not recall seeing this analysis made public. It was a dynamic process, but I wouldn't be surprised if Allison or Bill Fenner or somebody did post data at some stage; but that's ancient history now. The point behind my > suggestion was to prmit transparent consideration of the use of Discuss. > So whatever was done, it was not transparent to the community. > > Further, my suggestion was for /current/ patterns, not in the past. And all the data is in the tracker these days; it's very easy to find the DISCUSSes against any draft. >> to talk through many of those DISCUSSes and in many cases persuade the AD >> concerned to drop them, or rewrite them in an actionable format. In my >> recollection, Allison Mankin was the leader of the charge on this. >> >> But these are all judgment calls, so I don't think there can be an >> objective audit. > > I don't recall requiring it to be "objective". In fact, audits are > often subjective. That's ok as long as: > > a) the auditor gives thought to using reasonable criteria > > b) the criteria are made public > > c) they are applied consistently That is the judgment part. > > Let's try to refrain from throwing up artificial barriers, as an excuse > not to hold the process accountable. > > >> There could be an audit of how many DISCUSSes take >> more than N months to clear, or something like that. There were tools >> around for that some years ago, but I don't know if they exist for the >> modern version of the tracker. > > That's nice, but not all that useful. In contrast, looking at the > substance of Discusses against the criteria that are supposed to be used > for justifying them is directly relevant. > > (Note that you replaced a focus on core substance and clear import, with > something superficial and with a very ambiguous semantic. In > particular, longer-vs-shorter holding times have no obvious implication > about the /appropriateness/ of the Discusses.) I think that a long holding time clearly implies one of two causes: 1. A fumble by the shepherd; in fact it was to avoid this sort of problem that the shepherding process was carefully defined. Again, credit to Allison. 2. A failed dialogue between the DISCUSSing AD and the authors and/or WG. I would assert that most of those are due to non-actionable DISCUSSes. There was a lot of analysis of real cases of serious delay behind the section on "DISCUSS non-criteria" in http://www.ietf.org/iesg/statement/discuss-criteria.html The fact that we're having this conversation several years later tells me that human nature hasn't changed much recently. ADs *do* need to look at those criteria and it's absolutely in order for the community to complain when they don't. >>> Herein lies the real problem: As with many process and structure >>> discussions in the IETF, folk often see only a simplistic, binary choice >>> between whatever they prefer, versus something akin to chaos. >>> >>> The world is more nuanced than that, and the choices more rich. >>> >>> Here is a small counter-example to your sole alternatives of status quo >>> or rubber stamp: >>> >>> Imagine a process which requires a range of reviews and requires >>> ADs to take note of the reviews and the community support-vs-objection. >>> >>> Imagine that the job of the ADs is to assess these and to block >>> proposals that have had major, unresolved problems uncovered or that >>> lack support, and to approve ones that have support and lack known, >>> major deficiencies as documented by the reviews. >> >> The only difference between that and what I see happening today is that >> the ADs actually verify the reviews by looking at the drafts themselves. > > You are factually wrong. AD's do their own reviews. ADs formulate > their own lists of issues and requirements. AD's assert them as the > basis for a Discuss. I didn't mean to imply that they (all) don't. But the criteria for a comment being a DISCUSS are on public record. > They often do pay attention to other reviews -- sometimes quite > mechanically, rather than from an informed basis -- but the exemplar I > put forward is a fundamentally different model of AD behavior and > responsibility. I can't guess how you can misunderstand the difference. I don't think we differ on the responsibility that most ADs feel when ballotting on a draft. I'm sure there is a significant variety of "personal algorithms" by which ADs operate. >> And why do they do that? Because they aren't going to take the >> responsibility for approving a document that they haven't read. Nobody >> would, I hope. > > Therein lies a core problem with the model: It hinges on personal > investment by the AD, ... by whoever is making the final determination ... > and a lack of trust in the community process, > using the excuse that the process is not perfect, as if the AD's own > evaluation process is... Nobody, surely, is claiming perfection? What I am claiming is that knowing one has final responsibility makes a person strive for something closer to perfection - which means noticing defects in a draft that nobody else has noticed. Enough on this thread from me. Regards Brian _______________________________________________ Ietf mailing list Ietf@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf