Dear Jürgen,
Sorry for the late reply, as our lead author is off without a stable connection..
Please see below.
On Wed, Feb 19, 2020 at 4:38 AM Schönwälder, Jürgen <J.Schoenwaelder@xxxxxxxxxxxxxxxxxxxx> wrote:
Dear authors,
I will comment inline...
On Tue, Feb 18, 2020 at 06:39:04PM -0500, Y. Richard Yang wrote:
> Dear Jürgen,
>
> Thank you so much for the careful reviews. The lead author of the document
> is Sabine and the response below is formulated by her and then discussed
> among the authors. Since she is off this week, so let me send the response
> on her behalf to make progress.
>
> Her response for an item will start with ---- [[SR]] and end with ----
>
> Please see below.
>
> On Tue, Feb 11, 2020 at 4:49 AM Jürgen Schönwälder via Datatracker <
> noreply@xxxxxxxx> wrote:
>
> > Reviewer: Jürgen Schönwälder
> > Review result: Has Issues
> >
> > I am not an ALTO expert so keep this in mind in case some of my
> > comments have little meaning for ALTO experts.
> >
> > - If multiple clients retrieve information about costs in the future
> > and take independent scheduling decisions based on that information,
> > it might turn out that a "cheap" time turns into "expensive" time,
> > i.e., the result may not be an improvement of QoE but rather the
> > opposite. How is this dealt with? I see this issue touched on in
> > some places and the suggestion seems to be the advice to implement
> > SSE. But should ALTO servers make promises long into the future if
> > clients do not implement SSE?
> >
> > I am concerned that announcements of "cheap" times can easily turn
> > into instabilities if many clients start to opt for the announced
> > "cheap" times and I think this deserves to be discussed explicitly
> > in the Operational Considerations section.
> >
> >
> Good comment! We will do some reorganization to more explicitly,
> systematically discuss this issue. In particular, we plan to make explicit
> the following considerations in the Operations Considerations section
> (Section 8):
>
> PROPOSE TEXT:
> It is important that both the operator of the network and the operator of
> the applications consider both the feedback aspect and the prediction-based
> (uncertainty) aspect of using the cost calendar. Consider the cost calendar
> as a traffic-aware map service (e.g., Google map). Using the service
> without considering its own effect, a large fleet can turn a not-congested
> road into a congested one; a large number of individual cars each choosing
> a road with light traffic ("cheap link") can also result in congestion or
> result in a less optimal global outcome (e.g., the Braess's Paradox
> [citation]); an accident may happen and hence cause unexpected delays.
>
> Although individual network operators and application operators can choose
> their own approaches to address the aforementioned issues, this document
> recommends the following considerations. First, a typical approach to
> reducing instability and handling uncertainty is to ensure timely update of
> information. Hence, the "ALTO Incremental Updates Using Server-Sent Events
> (SSE)" Service [draft-ietf-alto-incr-update-sse] SHOULD be used to update
> the calendar faster if supported by both the server and the client."
> Second, Although there are theoretical analysis [RoughgardenThesis] and
> Internet based evaluations [SIGCOMM2013SelfishRouting] showing that
> uncoordinated behaviors may not result in substantial sub-optimal
> results, when a network operator updates the cost calendar, and when an
> application reacts to the update, they should still consider the potential
> feedback effects.
Thanks. I was looking for something like this.
Great. We will add the text.
> > - p6: "time zone (in UTC)" ?? do you mean "time zone offset relative
> > to UTC"? After reading, I think this is misleading. Perhaps you
> > wanted to say "start time (including time zone)"
> >
> > ----- [[SR]]
> As the time zone in the provided start time is always UTC+0, would the
> following update be OK?:
>
> - remove text item "time zone (in UTC)"
> - update the "calendar-start-date" description in the next paragraph as
> follows:
>
> OLD
> "calendar-start-date": specifying when the Calendar starts, that is to
> which date the first value of the Cost Calendar is applicable.
>
> NEW
> "calendar-start-date": specifying when the Calendar starts, that is to
> which date the first value of the Cost Calendar is
> applicable. The reference time zone for the provided time values is UTC.
> -----
Is this not 'calendar-start-time' later on? Or are calendar-start-date
and calendar-start-time two different things?
Good point. I believe that they are the same. Let us unify.
If times are always in UTC, then why not say:
The calendar-start-(date|time) is always expressed in UTC.
This way the reader does not have to think about what a reference time
zone might be.
Sounds good. Simpler is always better.
> > - p7: broken sentence
> >
> > The extensions in this document and encode requests and responses
> > using JSON [RFC8259].
> >
> > ----- [[SR]]
> yes, thanks: "and" should be removed. New text will be:
> "The extensions in this document encode requests and responses using JSON
> [RFC8259]."
> -----
OK
> > - p7: editorial nit
> >
> > OLD
> >
> > this document extends: the IRD, the ALTO
> >
> > NEW
> >
> > this document extends the IRD and the ALTO
> >
>
> ----- [[SR]] thanks, and we will update.
OK
> > - I am a bit concerned about changing the type of json fields, i.e.,
> > from a number to an array of numbers. The ALTO specifications may
> > allow this but is there a certain risk of implementations not being
> > smart in handling this correctly? Were alternatives considered that
> > do not change the type of existing json fields? Given the explicit
> > text why this is OK from an ALTO perspective, it might be that the
> > WG did iterate on this (sorry if I raise a closed issue again).
> >
> >
> ----- [[SR]]
> Indeed the ALTO WG did iterate on this. In the base protocol RFC7285
> section 11.2.3.6, an ALTO cost is specified as a generic JSONValue, to
> allow extensions. Implementations of 7285 SHOULD assume it is a JSONNumber,
> and the implementor of this extension should consider the issue. We will
> add a sentence to alert the implementor.
> -----
OK
> > - p9: can a time-interval-size be negative? can it be zero?
> >
> > ---- [[SR]]
> it can neither be negative nor equal to zero. Would the following addition
> be OK?
> OLD
> A "time-interval-size" value contains a JSONNumber.
> NEW
> A "time-interval-size" value contains a positive integer JSONNumber (i..e,
> greater than zero integer).
This is a change from double precision floating point number to an
integer. Was that the intention of this change?
This is my typo edit and you caught it. Really careful reader!! It cannot be integer because the unit is second and we want to have values lower than second such as ms (0.001) values.
> - p9: what does 'at least equal to 1' mean? Do you mean 'a positive
> > integer number of values'?
> >
> >
> ----- [[SR]] yes, Would the following update be OK?
> OLD
> the integer number of values of the Cost Calendar array, at least equal to
> 1.
> NEW
> the positive integer number of values of the Cost Calendar array, at least
> equal to 1.
OK
> > - p14: I am confused about the time zone aspect. The time zone pops up
> > in section 3.2 but it is a bit unclear what is meant there (see
> > above). On page 14, there is a statement about a 'reference time
> > zone' (what is this?) and then there is text about HTTP header
> > fields, and I am lost how that format relates to the rest of the
> > document. Was the idea to say that calendar-start-time uses the HTTP
> > header time format? Then say this where calendar-start-time is
> > defined. And why do you use the HTTP format and not RFC 3339 format?
> > Perhaps this data format for date and time is popular in ALTO and
> > hence it makes sense, I guess I do not know enough about it. Anyway,
> > the time zone seems to be part of the calendar-start-time - if so
> > this was not clear while reading top-down. And if there is free
> > choice for the data and time format, I would find RFC 3339 probably
> > a good robust alternative.
> >
> >
> ----- [[SR]]
> Section 3.2 is part of an overview and was meant to define "what" the items
> describe rather than how they are encoded. So the format specification as
> an HTTP header field format was left to normative section 5 in page 14..
> How about we add a sentence in section 3.2, in addition to the update
> proposed above, as follows:
> OLD
> "calendar-start-date": specifying when the Calendar starts, that is to
> which date the first value of the Cost Calendar is applicable.
> NEW
> "calendar-start-date": specifying when the Calendar starts, that is to
> which date the first value of the Cost Calendar is applicable. The
> reference time zone for the provided time values is UTC.
>
> This document specifies the use of the HTTP header time field format
> specified in [RFC7231].
>
> Indeed, this data format for date and time is popular in ALTO.
> -----
I am fine if section 3.2 is silent about the timezone as long as the
definition in section 5.1.2 is clear about the timezone. (And under
the assumption that calendar-start-time is the same as
calendar-start-date).
> > - p17: Why is this SHOULD needed? Because you want to ensure that
> > there is always a value that can be used 'right now'? What about
> > values that are stale, i.e., calendar-start-time is way in the past?
> >
> > [...] The value provided for the "calendar-
> > start-time" attribute SHOULD NOT be later than the request date.
> >
> >
> ----- [[SR]] Yes, this is the idea. If the IRD indicates it provides a
> Calendar for a given cost type of a given resource, it should provide a
> value that the Client can use when it requests one. The Client can figure
> out when to request the next Calendar, if needed, by multiplying the
> Calendar attributes "time-interval-size" and "number-of-intervals".
>
> We plan to add this comment on page 18, before the comments on attribute
> "repeated". How does this sound?
>
> -----
OK
> > - p17: I am confused by the description of "repeated". Is the intent
> > to say that the calendar value array simply repeats N times? Am I
> > right in assuming that if "repeated" is not present, it defaults to
> > 1?
>
> ----- [[SR]]
> yes, if member "repeated" is not present, it means that the calendar value
> array is only valid for 1 time.
> Would the re-phrasing below for paragraph 1 be clearer?
>
> OLD
> [...] The number N includes the provided iteration
>
> NEW
> [...] The number N includes the iteration provided in the returned response.
> ----
OK
> > Or does "repeated" always have to be present? (Similarly for
> > number-of-intervals, can it be absent and then it defaults to 1?
> >
>
> ----- [[SR]] the member "number-of-intervals" is not optional and is
> defined in section 4.1 as an in integer at least equal to 1
> -----
OK
> > - In which sense is draft-ietf-alto-incr-update-sse normative given
> > that the current text says 'can be used' but does not mandate its
> > use?
>
>
> ---- [[SR]]
> this draft is cited in section 3 that is actually non-normative, in which
> it was thus decided to avoid normative terms.
> How about we start section 3 with the following text, like in RFC8189 which
> is also an ALTO extension?
> "The following is a non-normative overview of the multi-cost ALTO
> extensions defined in this document. It assumes the reader is familiar with
> the ALTO protocol [RFC7285]."
> -----
I do not think that all references in a normative section
automatically need to be normative. The question is typically "Can I
implement the specification without using a certain reference?" Given
that SSE is not mandatory to implement, it seemed to me that the
reference to SSE is non-normative.
OK. Good guidance.
> > In which sense is RFC 8259 normative? Is it required to make
> > ALTO clients and servers switch to RFC 8259? The Operational
> > Considerations section says "RECOMMENDED" to switch. But how does an
> > implementation work that does both RFC7285 (w/o UTF-8) and RFC8259
> > (with UTF-8)? I think a clearer message that ALTO clients and
> > servers implementing this extension must comply to RFC 8259 is
> > desirable.
> >
> >
> ----- [[SR]]
> RFC7285 reference RFC7159 and encodes JSON text in UTF-8, UTF-16,UTF-32,
> UTF-8 is default encoding format according to section 8.1 of RFC7159.
> Since RFC7159 has been replaced by RFC8259 after the publication of
> RFC7285, and since RFC7285 sets mandatory encoding to UTF-8, we will
> change RECOMMENDED to MUST, as you suggested. How does this sound?
>
> -----
This seems more consistent (but ultimately ALTO implementors must
decide whether requiring UTF-8 only is an issue for them).
OK.
> > - Since you write "SHOULD at least IEEE 754 double-precision floating
> > point", does this not make the IEEE 754 reference a normative
> > reference?
>
>
> ----- [[SR]] The text follows RFC7285, section 11.3.2.3, in which IEEE 754
> is an informative reference and cited with a SHOULD.
> ----
On second thought, I wonder whether you really want to mandate how a
value is 'stored'. Perhaps you wanted to require 'SHOULD use at least
support IEEE 754 double-precision floating point [IEEE.754.2008]
precision'.
Note that several lines up you suggested to change to integer, but I
am not sure you wanted that - please double check that you get this
right. Anyway, since you refer to properties of a concrete formats
defined in another document, it seems to me the reference is actually
normative.
<soap>
In general, I find the double precision requirement a bit irritating
given the large range of numbers doubles can represent (roughly from
2.2x10^-308 to 1.8x10^308). And yet, many practically useful numbers
such as 0.1 can't be represented exactly as floating point numbers,
i.e., there is always rounding involved, making comparisons rather
complicated.
Often people want to have doubles in the hope that this maximizes the
chances of not getting hit by rounding issues but the sad truth is that
floating point numbers remain dangerous.
</soap>
Totally agree. It is interesting that you use the 0.1 second example. I assume that you have the classical Patriot 0.1 second rounding bug in mind (https://www.viva64.com/en/b/0445/), which is quite close to our setting, where 0.1 is their interval size, although I do not foresee that ALTO cost calendar will be used in such a setting :-)
I believe that the initial design used only integers, with a unit (s, ms, ns, ...) to avoid the floating-point issue (instead of 0.1 sec, it is 100 ms), and then the feedback and WG decision is to use generic floating points to have a more uniform, single unit (second) design. One proposal, which I am not sure others like, is to use a smaller unit, say ms, and then most likely we will deal with only integers (in the range of [-(2**53)+1, (2**53)-1]) and can still handle smaller units should the setting arise. The downside is that there might be more zeros if the interval is longer (60 sec -> 60000 ms). What do you think?
Another possibility is to keep the current design but add some text to discuss the precision issue. A Proposal is the following:
"time-interval-size":
* is the duration of an ALTO Calendar time interval in seconds.
A "time-interval-size" value contains a JSONNumber. ALTO
Servers SHOULD use at least IEEE 754 double-precision floating
point [IEEE.754.2008] to store this value. Example values are:
300 , 7200, meaning that each Calendar value applies on a time
interval that lasts 5 minutes and 2 hours, respectively.
* is the duration of an ALTO Calendar time interval in seconds.
A "time-interval-size" value contains a JSONNumber. ALTO
Servers SHOULD use at least IEEE 754 double-precision floating
point [IEEE.754.2008] to store this value. Example values are:
300 , 7200, meaning that each Calendar value applies on a time
interval that lasts 5 minutes and 2 hours, respectively.
=>
"time-interval-size":
* is the duration of an ALTO Calendar time interval in seconds.
A "time-interval-size" value contains a JSONNumber. Example values
are: 300 , 7200, meaning that each Calendar value applies on a time
* is the duration of an ALTO Calendar time interval in seconds.
A "time-interval-size" value contains a JSONNumber. Example values
are: 300 , 7200, meaning that each Calendar value applies on a time
interval that lasts 5 minutes and 2 hours, respectively. Since the
interval size can be smaller than 1 second (e.g., 100 ms), the value
specified can also be a floating point (e.g., 0.1). Both ALTO clients and
servers should be aware of potential issues caused by the precision
issues caused by using floating point numbers. For example,
interval size can be smaller than 1 second (e.g., 100 ms), the value
specified can also be a floating point (e.g., 0.1). Both ALTO clients and
servers should be aware of potential issues caused by the precision
issues caused by using floating point numbers. For example,
the server may have an internal, integer representation in ms, and store
100 as an int (0.1 sec) in the local storage; the sever sends "0.1" as
the interval size to the client; the client may use a float/double to store
the JSON number, which may not represent 0.1 precisely. To improve
interoperability, the server and the client SHOULD use IEEE 754
double-precision floating point [IEEE.754.2008] to store this value, in unit
of seconds.
100 as an int (0.1 sec) in the local storage; the sever sends "0.1" as
the interval size to the client; the client may use a float/double to store
the JSON number, which may not represent 0.1 precisely. To improve
interoperability, the server and the client SHOULD use IEEE 754
double-precision floating point [IEEE.754.2008] to store this value, in unit
of seconds.
I am a bit ambivalent of the last sentence though.
Any comment?
> > Similarly, if you import the data format from HTTP/1.1,
> > does this not make the HTTP/1.1 RFC a normative reference?
> >
> >
> ----- [[SR]]
> Our plan is to make HTTP header fields format [RFC7231] a normative
> reference, but a question for you: since RFC7285 lists RFC7231 as an
> informative reference, if we were to follow RFC7285, 7231 would be
> informative as well, but we agree that listing 7231 as a normative
> reference makes more sense. OK?
>
> -----
>
I think your definition of calendar-start-(date|time) normatively
imports the Date/Time Format defined in section 7.1.1.1 of RFC 7231
and hence I think your document has a normative reference to RFC 7231.
It does not matter for your document what RFC 7285 has a normative /
informative references.
OK. W
Thanks a lot!
Richard
/js
--
Juergen Schoenwaelder Jacobs University Bremen gGmbH
Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
Fax: +49 421 200 3103 <https://www.jacobs-university.de/>
-- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
