Hi again John, Substantive response this time... On 01/14/2013 01:05 PM, John Leslie wrote: > The IESG <iesg-secretary@xxxxxxxx> wrote: >> >> The IESG has received a request from an individual submitter to consider >> the following document: >> - 'A Fast-Track way to RFC with Running Code' >> <draft-farrell-ft-03.txt> as Experimental RFC >> >> The IESG plans to make a decision in the next few weeks, and solicits >> final comments on this action. > > I'm pretty darn uncomfortable _ever_ picking a fight with any > sitting AD, But I feel obligated to say this seems like a terrible > idea to me. > > As a background, I'm a long-time believer in "rough consensus" for > Proposed Standard and "running code" for advancement along the > standards track. I do not believe the two mix well. I don't get why they don't mix well to be honest. And I don't get it from your mail (so far). But I think you're maybe interpreting the draft differently from how its intended so let's see if we can figure it out. I'd also point out that this experiment is optional. And the intent is that if the experiment were a success that the fast track process would always be optional - to be used or not as WG chairs choose, with the co-operation of authors/editors and ADs, pretty much any of whom could basically force the draft to be processed in the normal manner. Some of your concerns read as if the goal were that all PS RFCs would be processed this way. That's not the goal and that's stated in the draft. > A standard needs to be as simple as possible (but no simpler); > running code needs to be complex. > > Stephen Farrell wants to "speed up" our process by adding an > incentive for "having" running code. Specifically: > ] > ] This memo describes a way to fast-track some final stage reviews for > ] a working group draft when the working group management and area > ] directors believe that the existence of one or more implementations > ] serve as a practical review of the engineering choices made by the > ] working group. > > Actually, now that I read that carefully, it doesn't even need to be > "running" code. :^( Well I reckon its pretty clear when you read the whole thing, but text tweaks are welcome. > ] This experiment needs an implementation that matches the specification > ] contained in the draft. > > (Verifying that has proven to be surprisingly hard. One guesses > that the "match" may not be rigidly enforced...) > > Understand, IMHO there is much to be gained by testing the > interoperability of independent implementations of a specification; > but that is a long-winded process, hardly anything which could be > done during the "fast-track" Stephen is proposing. It may be a long process. I think that'll depend on the subject matter. Could well be that this isn't suitable for things where sensibly checking the implementation matches the draft is too complex. And maybe that's a good thing (if it encourages us to write simpler specifications for example). > ] It is up to the WG chairs and responsible AD to satisfy themselves > ] on this point within the normal bounds of trust > > "Normal bounds of trust" isn't particularly compatible with our > paradigm of transparency of process... > > I don't even think it means the same thing to the rather limited > set of people named here. Yeah you may be right. Can't recall who suggested that phrase (maybe it was me, but it doesn't sound like me:-) but it could be improved on I guess. We do also trust WG chairs and ADs for lots of stuff already, so would it be better if it said, e.g.: This experiment needs an implementation that matches the specification contained in the draft. It is up to the WG chairs and responsible AD to satisfy themselves on this point. There is no implication here as to the licensing of the implementation whether open- or closed-source. ... > ] and without any implication about the licensing related to an > ] open- or closed-source implementation. > > Any mention of licensing seems out of place here. But it only says that licensing can be anything. I do think that's worth saying but maybe could be better split out as above. > > ] Note that the existence of an implementation is not sufficient to > ] ensure that the draft will automatically pass working group or IETF > ] last call, nor that it will receive IESG approval. > > Nor should it be, of course... > > Getting to the devil-in-the-details: > ] > ] 1. Any Working Group Last Call (WGLC) or Area Director (AD) review > ] ... will run in parallel with the two-week IETF Last Call > > ] 2. The IETF last call announcement message must say that the draft > ] in question is following the fast-track process... > > ] 3. The draft itself should note (e.g. in the introduction) that it > ] has been fast-track processed and reference this RFC. > > ] 4. Only comments that would be "DISCUSS-worthy" according to the > ] IESG Discuss Criteria [DCRIT] need be handled during last call. > ] Other comments can be handled or not, at the authors/editors > ] discretion. > > This is actually normal for IETF-wide LastCall (though it leads > to raised tempers when seemingly valid comments are ignored). > > It is not normal for WG LastCall; nor is it even compatible with > "rough consensus" -- by which we mean that any new questions raised > have been considered to the satisfaction of a substantial majority > of the participants. Well WGLC isn't part of 2026, and others have argued that that means that there's no need for this to even be a process-experiment. That seems to be at odds with your position. My take on this is that yes, this is a proposal for an experiment that changes what we do now with WGLC, and its one I think we ought try. I think we just disagree on whether that's a good thing to try or not. > > ] 5. The responsible AD for the WG concerned makes the decision as to > ] whether changes are required as a result of review comments, and > ] determines whether or not those have been completed. If > ] significant change or extended discussion is required, or if > ] changes are not complete within two weeks after the end of fast- > ] track last call, then the draft is returned to the WG by the > ] responsible AD and the document is withdrawn from the experiment. > > This is not an easy read. :^( > > First, it centralizes the decision in one person (whereas normal > WGLC process has two or more WG Chairs judging consensus of the group). I disagree. All the WG chairs must already want to use the fast-track experiment so they presumably are already ok with the draft at the start of IETF LC. So the only thing the responsible AD is judging is whether the IETF LC comments are such that going ahead to IESG review is ok, which what happens now. > Second, it centralizes the decision in one person whether changes > in the document properly address any WG or IETF-wide comments. This > kind-of matches current practice in the IESG for Area Director > comments; but does not match the theory -- and we could on very short > notice regress from the level of trust which allows this to happen > now. I don't get what you mean there about the theory. > Third, the two-week drop-dead time limit is unworkable. Not just > IMHO; in practice it hardly ever happens that everything resolves > itself that quickly. You might be right. But I think if some WG chairs and authors/editors ask for fast-track then they also need to be willing to work quickly to handle IETF LC comments, and they ought only do this where they expect it to work and are willing to try make it happen. So I'd argue to keep this timer. (The duration of the timer I agree might be wrong.) > AD DISCUSS comments tend to bunch up in the > hours before the telechat, and frequently need to be clarified > _after_ the telechat. I don't know what you mean about AD DISCUSSes - IESG review isn't changed in that respect and the draft does not require authors to handle those any differently from today. > Fourth, it's not really clear what happens after "the document is > withdrawn from the experiment." It would appear at first blush that > it must now start a separate WGLC, followed by a new IETF LastCall. > I would never consent to this process for a document _I_ cared about. You are correct, the intent is that the draft goes back to the WG and is then processed as normal. > ] 6. As soon as the responsible AD has confirmed that the authors/ > ] editors have made any changes required as a result of fast-track > ] last-call, then the document should enter IESG review and be > ] placed on the next IESG telechat agenda that is more than one > ] week in the future. > > Oh... Stephen wasn't talking about the IESG process at all! > > I'm even more confused now. I'm confused that you're confused! > I guess this means that one person (the responsible AD) takes a > look at any and all comments during the "combined" LastCall and > responsible-AD-review and makes a unilateral decision of what > changes the authors/editors should make (on short notice); then > things open for IESG-member DISCUSS comments. Right. That's the idea. But note the WG chairs kick off the process. > This would seem to preclude the current practice of putting > drafts on the IESG agenda _before_ the end of IETF LastCall. I'm > only getting more confused as to how this speeds up our process. It may be that the set of drafts for which this is useful and the set that are reasonable to put on a telechat agenda before the end of IETF LC are disjoint. But its an interesting, though I'd say minor, point that might come up during the experiment if we try it. > ] 7. Given the reduced time associated with fast-track processing, the > ] responsible AD may not have time to carry out sufficient review > ] prior to IESG evaluation to be confident in balloting "Yes" for > ] the document. Draft progression during and after IESG review is > ] otherwise unaffected, so a "Yes" ballot is needed from some AD. > > I _do_ understand that part, thought it's a bit obscure to those > who haven't been IESG members: no document gets approved by the > IESG without at least one "Yes" ballot-position; and Stephen wants > to be able to decline to ballot "Yes" on a document he (as > responsible-AD) has almost unilaterally re-written. (This would > be a really-bad idea, IMHO.) No, that's not the intent, nor I think would it be what'd happen; "almost unilaterally re-written" would not be appropriate I entirely agree, but I don't see any AD doing that after IETF LC now, and I can't see it happening with this experiment. Today, the convention is that the responsible AD ballots yes to kick off the process of IESG review. This is just saying that that AD might start IESG review with a different position (I would guess a no-objection, though there are exceptions to everything;-) because that AD hasn't had time to fully evaluate the draft. That AD might e.g. have asked someone else to check the implementation (allowed by this draft) but might want to do a fuller review during IESG evaluation before presumably balloting yes. Requiring the yes ballot up front is just a bit too demanding with the fast track timers. > ] 8. If at any point in the fast-track process the responsbile AD does > ] not act in a timely fashion then the IESG Secretary should ensure > ] that the draft enters IESG evaluation and is scheduled for the > ] the soonest IESG telechat that is more than one week after the > ] end of the two-week post-IETF LC period. That is, if the > ] responsible AD does not act, then the default action applies > ] which is that the draft enters IESG evaluation and is placed on > ] the earliest IESG telechat. > > (This is too complicated for me to try to explain in this email.) Maybe you'll come back to it? > ==== > > There's a lot more I could review, but I won't in this post. > > Were I an AD responsible to ballot on this, I'd ballot "abstain". > "Abstain" means "There's no point trying to satisfy me," and is > the way an IESG member essentially votes "No." > > I don't believe "running code" is even particularly helpful in > run-of-the-mill Proposed Standards. Right. I guess that's where we disagree. > I've seen it _seriously_ harm > the documents in cases where the "running code" has shipped and > there is reticence to change it: no matter how many weaknesses are > pointed out, you can't get WG consensus to change the spec. I agree that happens. I don't think this'll change how often. But if it made that happen more often then yes, we should probably declare the experiment over and not make this part of the formal process. I could argue that this might reduce that problem, on the basis that the draft and code are more likely to be the same, but I've no empirical basis on which to base that argument. (Same as you wouldn't if you argued that this experiment will make it happen more often.) > In cases where "running code" has proven helpful, it's my > honest opinion that the spec has become way too complicated (and > thus the "running code" has proven which parts are unworkable). Hmm. I've had somewhat the opposite experience, so I guess YMMV. I've not gone to dig up examples though but in DKIM I think we ended up with a simpler spec. due to some folks doing good work with running code before and during the WG's processing of the drafts. > I don't believe it's possible to "fast-track" "runnng code"; > nor do I believe it would be desirable were it possible. Code > needs to deal with complex cases that, for the most part, don't > need to be standardized in the first place. But code which > doesn't deal with these is useless in the marketplace. That may be right, but we're not talking about fast-tracking the code. > IMHO, we designed a separate "Proposed Standard" step to get > a specification published quickly without delving into the many > questions that arise when writing actual code. That's fair. I think this experiment partly reflects one way in which the standards process theory and IETF practice have diverged over the years. The bar for PS is mostly now much higher than was the original intent. I think that in some cases, this experiment might lower that bar a little. > There are plenty > of Standards-Development Organizations that start from "code". > We don't need to become one of them. We also have issues with RFCs for which there is no code and never will be;-) But more to the point, I think that in a lot of cases where the IETF has done a good job, there has been running code before the WG even started, so I disagree that starting from code is bad. Rubber stamping stuff whether based on code or not would be bad, but isn't part of this experiment. Cheers, S. > > -- > John Leslie <john@xxxxxxx> > >