Reviewer: Stewart Bryant Review result: Ready with Nits I am the assigned Gen-ART reviewer for this draft. The General Area Review Team (Gen-ART) reviews all IETF documents being processed by the IESG for the IETF Chair. Please treat these comments just like any other last call comments. For more information, please see the FAQ at <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>. Document: draft-ietf-bmwg-sdn-controller-benchmark-meth-07 Reviewer: Stewart Bryant Review Date: 2018-01-30 IETF LC End Date: 2018-02-02 IESG Telechat date: Not scheduled for a telechat Summary: This is a well written comprehensive test set for SDN controllers. It could be published as is, but some thought about how to address the issues below might be helpful to the user of this technology. Major issues: None Minor issues: I find the large amount of text on Openflow that appears out of the blue in the appendix somewhat strange. The test suit is controller protocol agnostic, so I wonder why so much text is devoted to this specific SDN control protocol. If they are there by way of illustrative example of packet exchanges, it might be useful to the reader to point to them from the measurement text. Something I am slightly surprised by is the lack of statistical sophistication. Average is a very crude metric giving no information on the distribution of the results. I imagine that it is now ingrained in this aspect of the industry to specify graphs and tables, but I would have expected that the results would be specified in some machine readable format such as xml for input to a database rather than in the human readable format that is hard coded into this specification. Nits/editorial comments: Abstract This document defines the methodologies for benchmarking control plane performance of SDN controllers. Terminology related to benchmarking SDN controllers is described in the companion terminology document. SB> It would be convenient to the reader to provide the reference to or name of SB> the companion document - the twin of the comment in the other review. SB> it would also be useful to include such a reference early in the main text. ============= 4. Test Considerations 4.1. Network Topology The test cases SHOULD use Leaf-Spine topology with at least 1 Network Device in the topology for benchmarking. SB> Leaf-Spine could use a reference. In Fig 2 I am not sure this is SL rather than SB> a linear sequence of nodes. There is a better SL diagram later in the SB> document and it would be useful to the reader to forward reference it. ======== The test traffic generators TP1 and TP2 SHOULD be connected to the first and the last leaf Network Device. SB> I am sure I know what does first and last mean, but the meaning should be called out. ========= Procedure: 5. Stop the trial when the discovered topology information matches the deployed network topology, or when the discovered topology information return the same details for 3 consecutive queries. SB> What do you report in the latter case? ===========