Hi Peter,
thank you for your thorough review and detailed comments. Please find my notes below under the GIM>> tag. I've attached the diff highlighting all the updates.
Regards,
Greg
On Tue, Oct 10, 2023 at 5:42 AM Peter Yee via Datatracker <noreply@xxxxxxxx> wrote:
Reviewer: Peter Yee
Review result: Ready with Issues
I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair. Please treat these comments just
like any other last call comments.
For more information, please see the FAQ at
<https://wiki.ietf.org/en/group/gen/GenArtFAQ>.
Document: draft-ietf-ippm-pam-06
Reviewer: Peter Yee
Review Date: 2023-10-09
IETF LC End Date: 2023-10-20
IESG Telechat date: Not scheduled for a telechat
Summary: This specification offers a scheme for metrics that are used in
determining if what you’re getting on a network is what you contracted for. I
have some issues with the draft and the usual nits. [Ready with issues]
Major issues: None
Minor issues:
General: For something that's Standards Track, I find the heavy use of "can",
"could", and “might” leaving me wondering if this specification should be
Informational instead. It all seems a bit loose. You could do this or that, if
you wanted… I get that the specific metrics are defined here, but even so,
there's not a lot of meat on the bone.
GIM>> As the result of the discussion with our AD Martin Duke, the -07 version of the document was set onto the Informational track. In your opinion, the wording you've pointed out could be appropriate for an informational document?
Page 4, 1st full paragraph, 2nd sentence: I've not followed this work, so
pardon me if I'm repeating a previous argument. However, I would find terms
like "compliance", "compliant", and "non-compliant" (maybe "incompliant")
preferable over "availability", "available", and "unavailable". Sure, you get
PCM for an initialism instead of a nice acronym like PAM, but "available" also
has a common meaning that makes it less desirable, in my opinion. Compliance is
already used in the previous paragraph and seems like it could be substituted
without a lot of drama. Of course, this will also affect section 3.3, should
you choose the change the terms.
GIM>> We, the authors, over the course of developing the document, discussed the terminology among ourselves,e.g., the use of "conformance" and "compliance", and with the WG. Current terminology, as I understand it, is consistent and reflects the consensus of the WG.
Page 8, 2nd paragraph, 3rd sentence: I don’t find availability based on
successive intervals all that good a way of making that determination. A
service with an unavailability threshold set would be deemed "available" if it
had a pattern of "unavailability threshold" - 1 interval violations followed by
one VFI, rinse and repeat. Might you not want something like a percentage of
VIs over a larger window of time instead? I realize this a couched in
permissive language, but then the following text runs with the idea that
successive intervals is the way to go. This also goes back to my general issue
above about the looseness of this specification.
GIM>> Thank you for the question. Violated Interval Ratio is one of the PAM metrics that could be defined as noted in the last paragraph of Section 3.3. That and other metrics can be used to improve the understanding of the state of a service regarding the defined SLO thresholds.
Nits/editorial comments:
Page 2, center header: I'm not sure how “PAM for Multi-SLO” applies as a
compression of the title. “Multi-SLO” never appears in any other context than
the header of the document.
GIM>> The intention of the shortened title was to highlight that PAM can be particularly useful for services governed by more than one SLO, multiple SLOs, as in the full title of the document - "Precision Availability Metrics for Services Governed by Service Level Objectives (SLOs)". If you find the short title not reflecting the intent of the document, could you kindly suggest an alternative?
Page 3, 1st full paragraph, 4th sentence: I’d change “set of thereof” to “set
thereof”.
GIM>> Agreed
Page 4, 1st full paragraph, 3rd sentence: delete “, and which” along with
“therefore” and if that parses better. Otherwise, the sentence feels like a
phrase.
GIM>> That text has been edited in the current version -07:
OLD TEXT in -06:
"Precision" refers to the fact that services whose end-to-end service
levels are governed by SLOs, and which must therefore be precisely
delivered according to the associated quality and performance
requirements.
NEW TEXT in -07:
"Precision" refers to services whose end-to-end service levels are
governed by SLOs and must be delivered precisely according to the
associated quality and performance requirements.
Would you find the current version clearer and acceptable?
Page 4, 2nd full paragraph, 1st sentence: change “is” to “are”.
GIM>> Agreed. Also s/a separate topic/separate topics/ Right?
Page 5, section 3.1, 1st paragraph, 3rd sentence: “second” is given of an
example of a different granularity, but that’s the granularity given in the
second sentence, so it doesn’t make a good example of a different granularity.
Decamillisecond is fine.
GIM>> Removed "second or". Is that acceptable?
Page 5, section 3.1, 1st paragraph, 5th sentence: remove a spurious white space
from “(VFI )”.
GIM>> Done.
Page 5, section 3.1, 1st paragraph after the bullet items, 1st sentence: insert
“of” between “monitoring” and “performance”.
GIM>> Done.
Page 6, 2nd paragraph, 2nd sentence: change “to distinguish” to
“distinguishing”.
GIM>> Agreed.
Page 6, definitions for “VPC” and “SVPC”: How is a severely VPC different from
a VPC?
GIM>> Although, the applicability of these two metrics is the same, they reflect aspects of different severity.
It’s also not clear to me from the text that the performance parameters
for VI/SVI apply to individual packets, so how is this count generated? Only
from parameters that can be applied on a per-packet basis (so not things like
jitter)?
GIM>> Indeed, some listed metrics in this list provided as examples of what could be used. Our intention is to define new metrics in the future, in a separate standard document. Do you see that as a reasonable approach for the informational document or would suggest another path forward?
Page 7, 6th and 7th bullet items: I’d change “measurement interval” to
“measurement session” (session was used previously) or “period”. Otherwise, the
term “interval” becomes unnecessarily overloaded.
GIM>> I think that using "interval" in these sentences is intentional to reference VI, SVI, and VFI. Could you kindly suggest how we can make it clearer?
Id-nits says:
== There is 1 instance of lines with non-ascii characters in the document.
I’m not sure where that character is and I admit to not searching for it.
GIM>> It appears that the warning is triggered by 'ø' in the following in Acknowledgment:
I hope that this is tolerable.The authors greatly appreciate review and comments by Bjørn Ivar
<<< text/html; charset="US-ASCII"; name="draft-ietf-ippm-pam-07.diff.html": Unrecognized >>>
-- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
