Hi Olafur,
On 01/14/2013 04:39 PM, Olafur Gudmundsson wrote:
> On 11/01/2013 10:14, The IESG wrote:
>>
>> The IESG has received a request from an individual submitter to
>> consider the following document: - 'A Fast-Track way to RFC with
>> Running Code' <draft-farrell-ft-03.txt> as Experimental RFC
>>
>> The IESG plans to make a decision in the next few weeks, and solicits
>> final comments on this action. Please send substantive comments to
>> the ietf@xxxxxxxx mailing lists by 2013-02-08. Exceptionally,
>> comments may be sent to iesg@xxxxxxxx instead. In either case, please
>> retain the beginning of the Subject line to allow automated sorting.
>>
>
> I have experience in process like this, as my WG DNSEXT has required
> multiple implementations and inter-op testing before advancing before
> advancing documents that make significant changes to the DNS protocol.
> Having done this I'm confident that the resulting specifications and
> code was much better.
>
> I support this experiment but offer the following comments.
Phew, so I'm not entirely crazy, good to know:-)
> Comment #1: The important part of running code is to assess
> clarity of the specification, thus implementation by editors of the
> document should not count as one-of-two required implementations
> Implementations by editors co-workers are ok IFF the the editors keep
> track of communications that lead to changes in code or draft.
First this draft only requires one implementation and not two as
your WG did. Second, I don't know what you mean by "keep track
of communications" - can you explain? (Or send a pointer to
one you used in the WG?)
That's also sort of like the point Stefan W. raised. And he
suggested:
"If the source code has been developed
independently of the authoring of the draft (and ideally by non
WG participants), it is likely that the implementation and the
draft match, and that pitfalls unaware developers may find have
been found and dealt with. If, on the other hand, draft
author(s) and implementation developer(s) overlap, then it is
sensible to scrutinize the draft more closely, both with
respect to its match with the implementation and for
assumptions that author/developer may have taken for granted
which warrant documentation in the draft."
Are you saying something like that would help?
Personally, I think there are thousands of nuggets of advice
that we could possibly add, but I'm not convinced that we
should - I think we'd be better off trying to keep this draft
shorter and simpler and then if the experiment succeeds we can
incorporate those things that experience has shown are worth
including.
> Comment #2: It is important that participants all realize that point of
> the exercise is not to point figurers at bugs. Rather the goal is to
> improve the specifications and make ALL the implementations as compliant
> and bug free as possible.
Agreed. Not sure what text would change though.
Same comment about "nuggets" as above:-)
> Comment #3 (Section 4 point #6)
> Test cases used for interoperability are critical. These test
> cases MUST be public. Evaluations of test cases generated by the
> implementors and/or other working group participants are critical as
> that is a great indicator of the quality and thoroughness of the tests.
> IMHO public test cases render the point of open vs. closed source
> irrelevant.
I think that's arguable, but a reasonable argument. Again though,
I don't think we're at the point where a MUST ought go into this
draft and that'd be better done when the experiment's done.
Or do you have a suggestion for text? (I'm not at all sure how
I'd write that up now.)
> Comment #4: The IETF-LC and WGLC statements SHOULD contain references to
> the testing performed and the implementations that participated.
I could buy that as a SHOULD or "ought." I've noted that
in the working version as a change to maybe make. [1]
Cheers,
S.
>
> Olafur
>
>