Hi Ted, Thanks that change looks good to me. I'll whack it in thanks. I do still like the word "reward" though so I'll tag that on too:-) Cheers, S. On 12/05/2012 07:36 PM, Ted Hardie wrote: > Hi Stephen, > > Some further comments in-line. > > On Wed, Dec 5, 2012 at 9:58 AM, Stephen Farrell > <stephen.farrell@xxxxxxxxx>wrote: > >> >> Hi Ted, >> >> On 12/05/2012 05:22 PM, Ted Hardie wrote: >>> Reading through Stephen's draft and the discussion to date, I think there >>> is some confusion/disagreement about what it is having an implementation >>> at this stage signals. >>> >>> One way to break up the work of the IETF is: >>> >>> Engineering--making decisions about the trade-offs related to >>> different approaches to solving a problem. >>> >>> Specification--producing text that describes how to inter-operate with >>> others. >>> >>> Standardization--describing the applicability of a specification or >>> its suitability as the basis of other work >>> >>> (Since we reflect all of these in the same document production, it's >>> really muddier than this, but bear with me) >>> >>> My experience is that people implementing during the working group >>> discussion phase generate really useful data about the engineering; >>> they can tell you the real impact of different trade-offs, so that >>> this isn't based on general experience. But it's not such a great >>> signal about the specification itself, since the spec is designed to >>> be usable by folks who were not part of the working group process. >> >> Good point. >> >>> Stephen's draft says: >>> >>> Note also that this experiment just needs an implementation that >>> makes it possible for the WG chairs and responsible AD to verify (to >>> the extent they chose) that the implementation matches the draft. >>> >>> and later: >>> >>> An implementation of the draft (ideally open-source) is required >>> for fast-track last-call. If there is no implementation or if the >>> implementation is unavailable or does not implement the draft >>> sufficiently closely then the document needs to be returned to the >>> WG. This only requires one implementation, not two and the WG >>> chairs and responsible AD decide themselves how much validation is >>> required for this. >>> >>> Given the "sufficiently closely" and the timing of production, I >>> assume that the signal we're looking for here is confirmation of the >>> engineering choices. I think that's fine (though I'm not sure this >>> needs formal experiment status). >> >> I hadn't thought about this in terms of signals, but its an >> interesting way to look at it. >> >> BTW, others have also said (and I agree) that lots of the things >> suggested in the draft don't need to be a formal experiment, >> but I think doing it that way has merit in any case. >> >>> But I believe we need to be really >>> careful that it isn't mistaken for signal about the specification's >>> quality. It can happen that a working group has "lore" about what to >>> do that gets folded into the implementations done by those >>> participating, but which never quite makes it into the spec "because >>> everyone knows it". An implementation written during the working >>> group process is potentially subject to this effect. >>> >>> An interoperating implementation written to the spec by a non-working >>> group participant would be great signal about the specification quality, >>> but it is not likely to be available at the stage of the process this >>> draft >>> targets. >> >> That's very true. >> >> I guess I'd be a bit reluctant to try to add text to the >> draft about this though, since it might be a bit of a rathole >> to try discuss generic good/bad aspects of implementations >> and specs. I could see that discussion running and running >> and getting nowhere;-) >> >> > I don't think it needs to be a rat-hole. Let me suggest the following > text: > > OLD: > > Sometimes, it can take a long time to get a Proposed Standard > produced in the IETF. This memo proposes an optional way to speed up > the parts of the process that happen after a WG has done its job by > building a "reward" for having an implementation (ideally open- > source) available into IETF processes. > > > NEW: > > Implementations developed during the production of an Internet-draft > can indicate that a working group has had the opportunity to get > early confirmation of its engineering choices. This memo proposes > an optional way to parallel process some final stage reviews when > the working group management and area directors believe that the > implementation can itself serve as a practical review of the engineering > choices. > > Does that make sense? > > > >> But maybe it'd be a good idea to maintain a wiki as the >> experiment runs where such things could be captured that >> could feed into later evaluation of how things went. >> >> If that sounds useful, I'm willing to start that (maybe >> once the draft's gotten past IETF LC) and add a bit of text >> to the draft pointing at the wiki page. I'm also happy to >> do some maintenance on that as things progress, if they do. >> >> If you think some other changes would help instead or >> as well, I'll gladly take text of course, or suggestions >> for how to (re-)structure the text. >> >> > I think it might generally useful to talk about this in terms of > parallel processing review, rather than issuing a reward, but > that may be simply a a style preference. > > regards, > > Ted Hardie > > >> Cheers, >> S. >> >>> Again, not objecting to the experiment; I just want to be clear about >>> what signal we believe we're getting from the implementation. >>> >>> Just my two cents, >>> >>> Ted >>> >> >