Hi Christian,
We made two changes to address your comments. Please see below.
> > * Allowing decimals into the cost metric identifier introduces a dot which
> > is reserved as per RFC7285 Section 10.6.
>
> Yes. Correct. From the grammar above, we made sure that we would not
> introduce a dot. We can add a sentence to point out that when choosing the
> base, we should follow this. Should we do this?
I think it'd be easiest (with the grammar or without) to to introduce
the number as a nonnegative integer. (That'd allow dropping mentions of
the minus and exp components).
The idea of making the number following p as a non-negative integer 0..100 is quite interesting.
If we limit the precision to only 2 digits, we are all set. One issue is that we may need higher
precision such as 99.9%. One possibility then is to consider delay-rt:p<y> as the 0.<y>
quantile. For example, delay-rt:p999 is the 99.9%, delay-rt:p01 is the 0.1%. One issue of
this design, however, is that we cannot represent 100% precisely.
If we limit the precision to only 2 digits, we are all set. One issue is that we may need higher
precision such as 99.9%. One possibility then is to consider delay-rt:p<y> as the 0.<y>
quantile. For example, delay-rt:p999 is the 99.9%, delay-rt:p01 is the 0.1%. One issue of
this design, however, is that we cannot represent 100% precisely.
As a result, this is the current version to address your comments:
NEW:
percentile, with letter 'p' followed by a number:
gives the percentile specified by the number following the letter
'p'. The number MUST be a non-negative JSON integer in the range
[0, 100] (i.e., greater than or equal to 0 and less than or equal
to 100), followed by an optional decimal part, if a higher precision
is needed. The decimal part should start with the '.' separator
(U+002E), and followed by a sequence of one or more ASCII numbers
between '0' and '9'. The total length of the cost metric string
MUST not exceed 32, as required by [RFC7285]. Assume this number
is y and consider the samples coming from a random variable X.
Then the metric returns x, such that the probability of X is less
than or equal to x, i.e., Prob(X <= x), = y/100. For example,
delay-ow:p99 gives the 99% percentile of observed one-way delay;
delay-ow:p99.9 gives the 99.9% percentile. Note that some systems
use quantile, which is in the range [0, 1]. When there is a more
common form for a given percentile, it is RECOMMENDED that the
common form being used; that is, instead of p0, use min; instead
of p50, use median; instead of p100, use max.
percentile, with letter 'p' followed by a number:
gives the percentile specified by the number following the letter
'p'. The number MUST be a non-negative JSON integer in the range
[0, 100] (i.e., greater than or equal to 0 and less than or equal
to 100), followed by an optional decimal part, if a higher precision
is needed. The decimal part should start with the '.' separator
(U+002E), and followed by a sequence of one or more ASCII numbers
between '0' and '9'. The total length of the cost metric string
MUST not exceed 32, as required by [RFC7285]. Assume this number
is y and consider the samples coming from a random variable X.
Then the metric returns x, such that the probability of X is less
than or equal to x, i.e., Prob(X <= x), = y/100. For example,
delay-ow:p99 gives the 99% percentile of observed one-way delay;
delay-ow:p99.9 gives the 99.9% percentile. Note that some systems
use quantile, which is in the range [0, 1]. When there is a more
common form for a given percentile, it is RECOMMENDED that the
common form being used; that is, instead of p0, use min; instead
of p50, use median; instead of p100, use max.
> > A few words on which statistic can be used with which metric could
> > also help with bw-maxres. (What does bw-maxres-p50 mean, is it
> > meaningful at all?)
>
> Mathematically, any percentile of a set of a single value is the value
> itself. But it is indeed a good idea to clarify it. We will add a sentence
> at the end of 2.2
>
> => Note that although one can use generic statistics (i.e., any percentile
> in [0, 100]) and multiple specifications may give the same value, it helps
> to choose the more intuitive and robust definition. For example, when the
> set is expected to be a single value. The max operator is more robust and
> hence recommended.
That sounds more confusing to me (the max operator ... so I should use
bw-maxres-max?); maybe (if correct) something like
| Note that unlike the other metrics, maxres is a single value and not
| sampled over time. Thus, it MUST NOT be specialized with a stat
| indicator, but is to be used in the base form.
After some thinking, we think the best way to address your comments is to introduce
a new statistical operator, and this is a nice fix. Specifically, here is the proposal:
NEW:
2.2. Performance Metric Statistics
The measurement of a performance metric often yields a set of samples
from an observation distribution ([Prometheus]), instead of a single
value. A statistical operator is applied to the samples to obtain a
value to be reported to the client. Multiple statistical operators
(e.g., min, median, max) are being used.
Hence, this document extends the general US-ASCII alphanumeric cost
metric strings, formally specified as the CostMetric type (defined in
Section 10.6 of [RFC7285]; see above in the CostType definition), as
follows:
A cost metric string consists of a base metric identifier (or base
identifier for short) string, followed by an optional statistical
operator string, connected by the ASCII character colon (':',
U+003A), if the statistical operator string exists.
a new statistical operator, and this is a nice fix. Specifically, here is the proposal:
NEW:
2.2. Performance Metric Statistics
The measurement of a performance metric often yields a set of samples
from an observation distribution ([Prometheus]), instead of a single
value. A statistical operator is applied to the samples to obtain a
value to be reported to the client. Multiple statistical operators
(e.g., min, median, max) are being used.
Hence, this document extends the general US-ASCII alphanumeric cost
metric strings, formally specified as the CostMetric type (defined in
Section 10.6 of [RFC7285]; see above in the CostType definition), as
follows:
A cost metric string consists of a base metric identifier (or base
identifier for short) string, followed by an optional statistical
operator string, connected by the ASCII character colon (':',
U+003A), if the statistical operator string exists.
Examples of cost metric strings then include "delay-ow", "delay-
ow:min", "delay-ow:p99", where "delay-ow" is the base metric
identifier string; "min" and "p99" are example statistical operator
strings.
ow:min", "delay-ow:p99", where "delay-ow" is the base metric
identifier string; "min" and "p99" are example statistical operator
strings.
The statistical operator string MUST be one of the following:
cur:
the instantaneous observation value of the metric from the most
recent sample (i.e., the current value).
percentile, with letter 'p' followed by a number:
...
If a cost metric string does not have the optional statical operator
string, the statistical operator SHOULD be interpreted as the default
statical operator in the definition of the base metric. If the
definition of the base metric does not provide a definition for the
default statistical operator, the metric MUST be considered as the
median value.
string, the statistical operator SHOULD be interpreted as the default
statical operator in the definition of the base metric. If the
definition of the base metric does not provide a definition for the
default statistical operator, the metric MUST be considered as the
median value.
"
Then, for the bw-maxres, we have the following:
"4.3.3. Intended Semantics and Use
Intended Semantics: To specify spatial and temporal maximum
reservable bandwidth from the specified source to the specified
destination. The value corresponds to the maximum bandwidth that can
be reserved (motivated from [RFC3630] Section 2.5.7). The spatial
aggregation unit is specified in the query context (e.g., PID to PID,
or endpoint to endpoint).
The default statical operator for maximum reservable bandwidth is the
current instantaneous sample; that is, the default is assumed to be
"cur".
"4.3.3. Intended Semantics and Use
Intended Semantics: To specify spatial and temporal maximum
reservable bandwidth from the specified source to the specified
destination. The value corresponds to the maximum bandwidth that can
be reserved (motivated from [RFC3630] Section 2.5.7). The spatial
aggregation unit is specified in the query context (e.g., PID to PID,
or endpoint to endpoint).
The default statical operator for maximum reservable bandwidth is the
current instantaneous sample; that is, the default is assumed to be
"cur".
Is the above better?
Thank you so much!
Richard
-- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
