On 15 Feb 2015 15:34, "Michael Richardson" <mcr+ietf@xxxxxxxxxxxx> wrote:
>
>
> Dave Cridland <dave@xxxxxxxxxxxx> wrote:
> > 1) AUTH48 is interesting as a metric; it's clearly an indicator of a
> > document actually getting published, and one that doesn't get delayed
> > outside the author's control unlike publication. Moreover, anyone going
> > through AUTH48 has had to deal with an IESG telechat on their document;
> > that's extremely useful experience in leadership selection. Possibly
> > "last call" and "telechat" are also good enough - and interesting in as
> > much as a document that "fails" last call is arguably as important a
> > contribution as one that goes on to publication.
>
> So, instead of AUTH48, you'd pick an earlier state like IESG Review.
> (see https://datatracker.ietf.org/doc/help/state/draft-iesg/, but
> actually I think write-up and the diagram are old and missing some states...)
>
> That (perhaps correctly) skips things that go through the Independent
> Submission Stream.
>
> I agree that including the authors into the state above solves the problem of
> a document which is so good that it needs no review and no tickets.
>
That all sounds good. And yes, I think an independent stream submission is reasonable to skip. I did consider suggesting that only IETF consensus documents should count, but that is probably too narrow.
> > 2) I think the document uploader concept is good, but limiting the
> > documents to only those which get scheduled time in a WG session isn't
> > so good; some WGs don't meet often, or even at all, and a document
> > that's so well crafted that it doesn't need discussion in a face to
> > face meeting is a really good document in my opinion. Clearly we don't
> > want arbitrary documents either. Any WG document would seem a "good"
> > contribution, and I suspect any document with a shepherd assigned
> > should be safe, too, since that effectively implies an expectation to
> > publish.
>
> The problem with shephard assigned is that it could be very late in the
> process, and a document which *does* get a lot of discussion and revision,
> represents a lot of contribution of effort. It is not unreasonable to me
> that a single document occupies the entire IETF "life" of 3-4 persons.
>
Yes, true. As I said, I think measuring contribution is a very hard metric to find, and there's an implicit suggestion lurking here that we're after long-term, quality contributions - I'm not sure I disagree with that incidentally.
But in general terms, any document that has any of those characteristics - it's WG document, one that's getting WG "time" by some measurement, or one that has a shepherd assigned - seems likely to be good for a document uploader metric.
It might be interesting to have the tools adapted so the uploader can note which of the authors have actively contributed this time around, too, to avoid the situation where two co-authors have to share uploading. Similarly, it might be useful for WG chairs to be able to tag a document as having been discussed within their WG.
> > 3) This works for WGs that use a ticketing system and have
> > meetings. Not sure what percentage that actually is.
>
> A number of WG chairs don't like the current ticketing system, but increasing
> I think that WGs are going to use some ticket system.
>
And, thinking about it further, we probably want to promote the use of a ticket system anyway. In addition, I'm more comfortable with measurable metrics.
Again, it might be useful for the ticket system to be able to record whether the ticket resulted in useful discussion, and note who was involved. Again, this is at the whim of somebody, but I really don't think excluding whims is entirely a positive thing.
Dave.