John,
On 10/10/24 5:22 PM, John C Klensin wrote:
Jean,
Per Brian, moving this to the IETF list and adjusting the subject
line. And pruning considerable text that I think was included in
Brian's note and my response...
--On Thursday, October 10, 2024 14:54 -0500 Jean Mahoney
<jmahoney@xxxxxxxx> wrote:
(With my Gen-ART Secretary hat on)
John,
On 10/10/24 2:06 PM, John C Klensin wrote:
--On Thursday, October 10, 2024 13:23 -0500 Robert Sparks
<rjsparks@xxxxxxxxxxx> wrote:
...
At least in principle, there is a difference between (i) Last Call
as a community discussion mechanism whose effect is to inform the
IESG about community consensus and (ii) Last Call as a mechanism
to feed information, opinions, and other advice into the IESG so
the ADs can determine what they think is the right decision for
the Internet. If those directorate/area reviews are given
privileged status -- input into the telechats that ordinary IETF
participants don't get, more flexibility about deadlines, etc.
[JM] WRT to Gen-ART reviews, the reviewer should submit the review
before the Last Call.
Unclear. Do you mean "before the Last Call starts and is therefore
only a review for discussion within the area" or "before the Last
Call ends".
[JM] Before Last Call ends.
If the former, I think that is a great idea -- it might
even inform relevant ADs as to whether to initiate the Last Call. I
don't think that, in practice, that has been happening very often
(certainly for draft-emailcore-rfc5321bis there has been no
discussion on the ART list since well before publication was
requested and, of course, no Gen-ART review posted at all so far.
If the latter, that document constitutes a counterexample and, again,
no posted review yet.
[JM] draft-ietf-emailcore-rfc5321bis is a long document (114 pages), and
its Last Call deadline was the default two weeks after the announcement
of Last Call. You can talk with the AD about extending the LC. This may
help with receiving more reviews.
The telechat review that Robert mentioned is
when the Gen-ART reviewer follows up on their LC review (using the
same mailing lists that were used for the LC review) to say whether
their comments have/haven't been addressed.
But that requires that there be an earlier, public, review
identifying those comments (inconsistent with "assigned at IETF Last
Call" [1]). In a way, it would constitute a supplement to the
portion of the Shepherd's report that identifies outstanding issues.
And, if it were what the IESG and community intended, the area
reviews should probably be due, not during the Last Call window but a
few days later so the reviewers can consider all Last Call comments
and whether they were addressed.
If the reviews are assigned only when, or after, IETF Last Call
starts,
[JM] Yes, this is the case.
then there presumably need to be two postings from the
reviewer during the Last Call window -- the initial review with any
issues identified and a second one, providing answers to the
"addressed/not addressed" topics.
[JM] If the review highlights issues beyond nits, then it could prompt a
discussion thread with the authors (note that Gen-ART reviews are sent
to draft.all@xxxxxxxx and the draft's WG mailing list if applicable in
addition to the gen-art mailing list). These discussions can extend
beyond the LC deadline.
My entirely subjective
impression is that almost never happens, at least in public and on
the Last Call mailing list.
I am currently not
assigning explicit telechat reviews because usually the reviewer
will follow up on their own.
Even, to come back to Brian's comment, less public.
[JM] The reviewer follows up on the lists to which they sent the LC
review -- gen-art with draft.all@xxxxxxxx and any relevant WG mailing
list CCed, so the followups are public.
-- then the "treat this like any
other review" boilerplate of most of those reviews becomes a joke
or worse. It would be somewhat different if those really were
directorate or area reviews -- reviews that were written (or
finalized) only after specific discussion about the document within
that area or directorate and that represented consensus in that
group. But they often are not -- they are more often the opinions
of an individual who comes up in rotation or draws a short straw.
[JM] I assign a Gen-ART review to the next reviewer in rotation.
Please see [1] for details about the review team.
Nothing there surprises me, but, unless the reviewer reads the
document, prepares a draft review, and posts it to an Area mailing
list (probably not just the review team list) for comment, it isn't
really an Area review but a review from an individual who is assumed
to have some of the perspective of the area. Maybe that is happening
in the General Area (or at least Gen-ART),
[JM] Gen-ART reviews are from individuals who are reviewing documents
from a general perspective. They consider the document's clarity,
protocol architecture, normative language, normative references, and
IANA Considerations when reviewing the document.
but I have not seen
symptoms of any multistage review of that type in any of the Areas I
watch more closely.
[JM] Directorates can have different processes. Links to those processes
can be found on their Datatracker pages [2].
In the same context, the problem with sharing draft reviews only with
the Area review team
[JM] Gen-ART reviewers don't create draft reviews for internal team
discussion. A Gen-ART reviewer posts their review simultaneously to
gen-art [3], draft.all@xxxxxxxx, and any relevant WG mailing list. The
discussions are between the reviewer and the authors, and also with
other WG participants and/or the AD, depending on the review.
Best regards,
Jean
is that it means, unless special arrangements
are made, one has to have the time to do reviews in order to see what
reviews are going out in the Area's name. For those who cannot spend
unlimited time on the IETF or who have to make tradeoffs between
general (or other area) work and their specific technical tasks, that
is a hard problem -- indeed, making sure that documents are broadly
reviewed from many perspectives, are what Last Calls are supposed to
be about and the approach you describe might actually frustrate that.
best,
john
[1] https://datatracker.ietf.org/group/genart/about/
[2] https://datatracker.ietf.org/dir/
[3] https://mailarchive.ietf.org/arch/browse/gen-art/
