Reviewer: Robert Sparks Review result: Almost Ready I am the assigned Gen-ART reviewer for this draft. The General Area Review Team (Gen-ART) reviews all IETF documents being processed by the IESG for the IETF Chair. Please treat these comments just like any other last call comments. For more information, please see the FAQ at <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>. Document: draft-ietf-ippm-model-based-metrics-10 Reviewer: Robert Sparks Review Date: 2017-03-13 IETF LC End Date: 2017-03-14 IESG Telechat date: Not scheduled for a telechat Status: Almost ready, but has significant issues to address before publication as an Informational RFC I'm guessing the goal for this document changed substantially during its development. The point and tone if the document changes depending on where you are in it. I found it very difficult to read (and I did see the shepherd's report call out that the document has already had a strong editorial pass). It really feels to me that the purpose this document is trying to serve could be better achieved with two much shorter documents. Please consider pulling out the most important observations into its own informational document, and an experimental document that succinctly presents the tests, the framework, and the feedback you want from people wo try these things. Major issues: There are issues with the equations in section 7.2. Please look at the line containing "Xa < defect_count(n) < Xb". Where is Xb defined? Was it supposed to be Xr from further up the page? (or perhaps that Xr should have been Xb? The document starts using 2119 MUSTs in section 7.3. None of the uses are appropriate 2119 keywords - they aren't talking about protocol. Please rewrite the prose without using the 2119 keywords. Section 8.2.4 seems to be missing an actual description/definition of the test it is trying to talk about. Minor issues: The abstract says this document does not define tests. The rest of the document says it does. (See phrases like "The tests described in this note") Where the document claims it is neccessary to "suppress the effects of TCP congestion control", it should say "avoid the effects". The mechanism of using precalculated traffic patterns discussed in the document avoids, but does not suppress. At the end of section 1, you call out "multiple standardized versions of TCP". Please clarify what you're trying to point to? In the definition of "traffic patterns", you note that the goal is to "mimic the range of common patterns". Consider qualifying that to "current common patterns", and add some discussion later in the document when you talk about pregenerating traffic patterns to point out that these change over time. At the next to last paragraph of section 4.1, you say "metrics have entirely thwarted the analytic framework". I don't think that's what you mean. I suspect you mean to say attempts to create metrics have been unsuccessful (I suggest you avoid the word "thwarted"). At the 3rd paragraph on page 19, you say "There is no model". Do you mean no model exists in the world for this, or only that this document does not provide a model. I strongly object to the attempt to use "infinitesimal" to describe the conditions on the test network you are trying to convey. At the very least, you are dealing with a finite (if arbitrarily large) set of states that network can be in, not a continuum. You should be looking to raid terms from discrete mathematics instead, but I don't think even that's the right thing to do. Please replace the infinitesimal concept here with it's definition (which you have to call out anyway). Just _say_ "has the tightest available constraints that allow the tests to pass." Consider providing some rational for the numbers you came up with in the strawman in the second paragraph on page 33. 1mS seems pretty arbitrary otherwise. I'm not asking for you to _defend_ the choice as the right number - just talk about why you picked it instead of something else. Nits: - Consider pointing to the models you are actually using in the first sentence of the first paragraph on page 9 - The third paragraph of the abstract is particularly hard to read. Please consider breaking the compound sentences into simpler ones. - You use "engineering test" before you say what you mean by that (see sections 8.2.4 and 8.3.2 for examples) - There are several places where you talk about tests being inconclusive because the precomputed traffic was not accurately generated. Are you trying to say "if you have an inconclusive test and can't otherwise figure out why, look closely at your precomputed traffic", or "Double-check your precomputed traffic before you run your test or risk garbage-in/garbage out"?