In addition to choosing the right way to look at the distribution of total action times, I strongly recommend breaking down the transactions into component parts and looking at the details. The exchange I had with Sam Hartman was a good example. On 5/23. Sam wrote:
> I'd think two months would be doing good for IESG processing. That > includes AD review, IETf last call, telechat delays and a bit of slop > for interaction with the authors. However that's time to travel > through the queue, not actually time spent on that document. > > Because of the delays involved with telechats and last call, getting a > WG document done in less than a month of wall time or an individual > document done in less than 1.5 months is very unlikely.
What catches my attention -- and brings shudders as I remember my time as an AD -- is that he paints a picture that looks like
T[IESG] = T[AD] + T[last call] + T[telechat delay] + T[authors]
I see two problems with this formulation:
(1) T[AD] is in fact composed of several separate things, and the differences between them are almost certainly critical in understanding process delays. In particular, AD time is spent in several different ways:
(a) Initial document review (done by all ADs) (b) AD followups (done by the sheparding AD) (c) AD rereview and discussion to clear discussses (done by all ADs)
There are timeouts associated with (a) - once a document is on the IESG's agenda, ADs are required to have their review done by the next telechat. They can get an extension, but only once. This sohuld therefore end up being similar to telechat timinig.
There are, however, no timeouts associated with (b) and (c). And more to the point, the pressures on sheparding AD are quite different than those on the other ADs to act in a timely fashion.
My experience has been that the distribution for (b) is bimodal (or
trimodal if you count the case where the document just passes without
any discusses). In some cases the sheparding AD only has to pass on the discuss comments to the authors, done. In others, however,
the AD has to spend a bunch of time in effect figuring out what
amounts to a compromise. This can take a lot of time.
In my experience (c) is the truly problematic case. In my experience it is both highly variable and the average time taken seems to be much longer than it should be. Additionally, as sheparding AD I often found it was only possible to clear discusses by explicitly bringing the document up on a subsequent telechat, so there's a periodicity here as well.
I have in the past advocated putting a timeout on (c), and I continue to think this is a really good idea.
(2) When I was an AD I went to a good deal of trouble to overlap the various phases of the process whenever possible in order to save time. Nothing prevents ADs from entering comments early. Nothing prevents documents from being revised during last call. I don't think I was terribly successful in this, but much of my time as AD was spent without the tools we have now. I therefore think the additive nature of your formula has to be characterized as a worst case, and it needs to be noted that substantial gains can be had by overlapping terms.
Now each of these terms has quite different characteristics. The last call and telechat delays add up to a few weeks, to the overall time is at least a couple of months. But those terms probably have moderately low variance. On the other hand, the T[AD] and T[authors] terms have very low minimums but extremely high variance, and it's within those terms that the real issues emerge.
Agreed.
I suspect if you break those terms down even further, interesting and useful dynamics will emerge.
Exactly so. Breaking them down further is IMO essential to getting anything useful out of the analysis.
Ned
_______________________________________________ Ietf@xxxxxxxx https://www1.ietf.org/mailman/listinfo/ietf