Hi Christian,
Thank you so much for the review. Please see below.
On Mon, Oct 18, 2021 at 10:02 AM Christian Amsüss via Datatracker <noreply@xxxxxxxx> wrote:
Reviewer: Christian Amsüss
Review result: Ready with Issues
## Summary for the IoT Directorate
This document extends the Application-Layer Traffic Optimization (ALTO)
protocol, by which applications that need to a peer from a selection can ask
their uplink for preferences. It adds more fine-grained vocabulary to pick the
peer with best expected bandwidth, or best latency.
As for conventions around the Internet of Things, this makes no choices -- if
follows the path set out by ALTO, and adds terms and considerations for metrics
established outside of ALTO.
The mechanisms (more of ALTO than this particular document) can be applicable
to IoT applications when unconstrained devices relay data into cloud systems --
then they might make better choices in presence of decentralized backends. If
this document gains traction in deployments, these systems will need deeper
application knowledge pertaining to the application's requirements (selecting
the low-latency vs. the high-bandwidth option).
## High-level
The document can be followed well after brief familiarization with ALTO; it is
in a good over-all shape.
2.2 Performance Metric Statistics:
* The metric-identifier definition seems disconnected with the rest of the
terminology. I assume from context that this is a way for building a
CostMetric value. If this assumption is wrong, some of the later points are
moot.
(By the way, what language is the metric-identifier definition? It's not
ABNF, and I don't find any other formal language in thenormative references.)
Good comment. The document gives the high-level grammar (1 line) at the beginning of Sec. 2.2.
It looks that your suggestion is the write out the complete grammar upfront:
It looks that your suggestion is the write out the complete grammar upfront:
<metric-identifier> ::= <metric-base-identifier> [ '-' <stat> ]
=>
<metric-identifier> ::= <metric-base-identifier> [ '-' <stat> ]
<metric-base-identifier> ::= `delay-ow` | `delay-rt` | `delay-variation`
| `hop-count` | `lossrate` | `tput` | `bw-residual` | `bw-maxres`
<stat> ::= `min` | `max` | `median` | `mean` | `stddev` | `stdvar`
| `p<percent>`<percent> := ASICC number between 0 and 100, inclusive.
Is the above what you have in mind?
* p0 and p100 are aliases to min and max IIUC. Is there a preferred form, and
why is the other form allowed? (Same for p50 and median.)
p<percent> is a uniform notation. Singling out 0, 50, 100 creates complexity, although there is the more common
min/max/median. How about we add a sentence at the end of the paragraph:
min/max/median. How about we add a sentence at the end of the paragraph:
"... Note that some systems use quantile, which is in the range [0, 1]. This document uses percentile to make the identifier easier to read."
=>
"... Note that some systems use quantile, which is in the range [0, 1]. This document uses percentile to make the identifier easier to read. When there is a more common form for a given percentile, it is RECOMMENDED that the common form being used; that is, instead of p0, use min; instead of p50, use median; instead of p100, use max".
* Allowing decimals into the cost metric identifier introduces a dot which is
reserved as per RFC7285 Section 10.6.
Yes. Correct. From the grammar above, we made sure that we would not introduce a dot. We can add a sentence to point out that when choosing the base, we should follow this. Should we do this?
* comparing to 7 IANA Considerations: The registered identifeirs are only the
metric-base-identifiers.
Registering all combined values from metric-base-identifier x stat is not
practical (especially with the numeric percentiles); however, the 'priv:'
prefix indicates that some structure was intended in RFC7285.
A way out could be to formalize this structure and register the
metric-base-identifiers for use with and without a stat parameter following
the colon (instead of the dash).
So the suggestion is that ":" as the consistent internal structure separator. This is quite reasonable.
* Section 2.2 (Performance Metric Statistics) allows adding -stdvar to
metric-base-identifiers; delay-variation-mean is probably similar to
delay-ow-stdvar. It's only similar (and not identical) due to the offset from
minimum detailed in 3.3.3; anyhow, pointing out that out in the "Note that in
statistics" paragraph, e.g. as in "... networking convention. Due to this, it
is expressed as a dedicated metric and not just as delay-ow-stddev".
OK. Sounds good. We will add the note.
A few words on which statistic can be used with which metric could also help
with bw-maxres. (What does bw-maxres-p50 mean, is it meaningful at all?)
Mathematically, any percentile of a set of a single value is the value itself. But it is indeed a good idea to clarify it. We will add a sentence at the end of 2.2
=> Note that although one can use generic statistics (i.e., any percentile in [0, 100]) and multiple specifications may give the same value, it helps to choose the more intuitive and robust definition. For example, when the set is expected to be a single value. The max operator is more robust and hence recommended.
=> Note that although one can use generic statistics (i.e., any percentile in [0, 100]) and multiple specifications may give the same value, it helps to choose the more intuitive and robust definition. For example, when the set is expected to be a single value. The max operator is more robust and hence recommended.
3 Packet Performance Metrics:
* Cost-Context Specification Considerations: There is a lot of duplication
here, especially around SLA and the topic of "link" descriptions. (For the
technically interesting parts, the later sections already refer back to
3.1.4).
Agree. We will take a look and try to compress a bit, say by referring to previous sections.
6 Security Considerations:
* On the topic of confidentiality, can these metrics be used with the "ordinal"
cost mode? A client might just want to pick the peer that has the "best"
round-trip time, and could thus avoid the need to obtain the confidential
metrics.
Yes. They can. Then the mapping (to the ordinal value space) should use these metrics. It is a good idea to point this out. We plan to add this sentence in the introduction and security:
Intro (right above Table 1) => Note that these metrics can be used to support both numerical and ordinal use cases. When it is the ordinal mode, the mapping to the ordinal value space should use these metrics.
Intro (right above Table 1) => Note that these metrics can be used to support both numerical and ordinal use cases. When it is the ordinal mode, the mapping to the ordinal value space should use these metrics.
We will add a sentence in Security, on the benefit as you mentioned.
## Nits
* Table 1: The current XML RFC format can do tables that render to HTML, this
would also make it easier to follow the links.
For tables and figures, also more accessible labeling is available there.
Good idea. We will use the format.
* Section 3 Packet Performance Metrics: The pkt.* notation appears very ad-hoc,
and is not used anywhere outside the paragraph. If these are referring to
external definitions, please consider referencing that definition; otherwise:
do these labels add value?
See your point. We will revise the pkt. notation to refer to metrics.
* "smaller than milliseconds": smaller than one millisecond?
OK. Will fix.
* "Content-Length: TBA": Does this add value to the examples?
We will add the final exact value when publishing, as it is part of HTTP header.
* Example in 3.1.3: Does the example make sense in terms of addresses? Querying
metrics between an IPv4 and an IPv6 address seems strange. (And is this
primarily used in a field where IPv4 is still dominant?)
It is an example to illustrate both ipv4 and ipv6. If we do ipv4->ip4 and ipv6-ipv6, we will
need two examples, and it takes more space.
need two examples, and it takes more space.
In particular, the hop count example has an unreasonably high number of hops
between 192.0.2.2 and 192.168.2.89...
Good catch. We will make it smaller :-)
* "the -<percentile> component": "any 'stat' component"?
Not sure what this is? Any more specific locator?
## Slightly off topic
(ie. this is out my expertise and would be checked later by someone else, but
it may be helpful to revisit it right away)
* IANA considerations: Creating a new regisry usually comes with more than
"reqeusts the creation of" and some values;
https://datatracker.ietf.org/doc/html/rfc8126#section-2.2 in particular asks
for a registration policy.
(Given the document says "MUST be one of three", there may not be a need to
set up a registry in the first place).
Very good comment! We have made the changes. The newer version fixed this issue.
Please see -18 which is just loaded today.
Thank you so much!!
Richard
-- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
