On Fri, Jul 05, 2019 at 01:02:18PM -0400, Ted Lemon wrote: > On Jul 5, 2019, at 12:31 PM, Nico Williams <nico@xxxxxxxxxxxxxxxx> wrote: > > At IETF we have commitment reviews, naturally, but we only have > > inception reviews for WGs and their initial work items, but no inception > > reviews for subsequent work items, and no interim reviews. > > Isn’t a call for adoption an inception review? If not, it should be. I was referring to review from *outside* the WG. Calls for adoption are typically intra-WG only. Reviews by a shepherd/AD, directorates, the IESG, and IETF (IETF LC) are commitment reviews. The WG spinup process is an inception review for the WG and its initial work item slate. We have no formal external-to-WGs interim review process. I have seen ADs ask for interim reviews by experts outside a WG, so we do have an informal interim review process. And maybe that should be good enough provided we engaged it *often* -- currently we don't. > We actually kind of suck at commitment reviews, in a lot of cases: the > work is done, after all, and everybody who thought it was a good idea > at adoption time is tired of looking at it. So it’s not uncommon > (and I’ve seen it on documents I’ve finished) that the document is > done and polished, and then there’s no response at all to the WGLC. I'd say this is actually us being bad at interim reviews. Our commitment review process is very good, but very heavy-duty and thus slow and painful. By reserving all that pain for the very end of the process -precisely when the WG participants are out of steam- the end result sucks. The problem isn't that we're bad at commitment reviews, rather the opposite combined with the lack of interim reviews. All the _surprises_ come at the end, when authors/WGs least want them. > [...] > > The problem the IETF has is that unlike at your former job, there is > no way to enforce this process other than by killing documents, and > that often only impacts the people who are working on them and the > people who would benefit from publication, and not all the people who > thought it was a good idea at adoption time, but have wandered off > during the course of the work. Ok, so let's fix that. My take is that a) directorates and shepherds/ADs should be expected to provide interim reviews as requested, b) WG chairs should request them when the ether the chairs believe, or the WG has consensus that, "now is a good time for interim review. Interim reviews should also be announced on the ietf@xxxxxxxx list, but review feedback should be on WG list. This invites interim reviews from the wider community as well. Now, I don't know how to incentivize the wider community to provide interim reviews. Reviewing I-Ds is really time-consuming and energy sapping for me personally -- the biggest problem is making the time and finding a way to get it funded when it's a lot of time. I suspect that the directorates are short on cycles too. I get that what I'm proposing may not be feasible at this time. Nico --
