Hi Mališa, please see below... > -----Original Message----- > From: Mališa Vučinić [mailto:malisa.vucinic@xxxxxxxx] > Sent: Tuesday, December 15, 2020 9:21 AM > To: MORTON, ALFRED C (AL) <acm@xxxxxxxxxxxxxxxx>; secdir@xxxxxxxx > Cc: last-call@xxxxxxxx; bmwg@xxxxxxxx; draft-ietf-bmwg-b2b- > frame.all@xxxxxxxx > Subject: Re: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03 > > Hi Al, > > Thanks, that is clear. I think that discussing the assumption of honesty > among the parties involved in benchmarking would be a useful addition to > the Security Considerations section in the draft. [acm] I don't mind explaining the requirement using the term "honesty", but I can only imagine raised eyebrows and subsequent DISCUSS/comments if we try to assert a need for/assumption of honesty anywhere in the memo. Do you have suggested wording? Do others have opinions whether or not this is needed? thanks, Al > > Mališa > > On 15/12/2020 14:45, "MORTON, ALFRED C (AL)" <acm@xxxxxxxxxxxxxxxx> wrote: > > Hi Mališa, > thanks for your review, please see below for one reply to your > question (acm]. > Al > > > -----Original Message----- > > From: bmwg [mailto:bmwg-bounces@xxxxxxxx] On Behalf Of Mališa > Vucinic via > > Datatracker > > Sent: Tuesday, December 15, 2020 6:30 AM > > To: secdir@xxxxxxxx > > Cc: last-call@xxxxxxxx; bmwg@xxxxxxxx; draft-ietf-bmwg-b2b- > > frame.all@xxxxxxxx > > Subject: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame- > 03 > > > > Reviewer: Mališa Vučinić > > Review result: Ready > > > > I reviewed this document as part of the Security Directorate's > ongoing > > effort > > to review all IETF documents being processed by the IESG. These > comments > > were > > written primarily for the benefit of the Security Area Directors. > Document > > authors, document editors, and WG chairs should treat these comments > just > > like > > any other IETF Last Call comments. > > > > Thank you for this well-written document, it was a pleasure to read > and I > > think > > it is ready to proceed. Since the document updates RFC2544 > benchmarking > > procedure for estimating the buffer time of a Device Under Test > (DUT), it > > does > > not raise any security issues. Security Considerations section is > quite > > clear > > and it stresses that these tests are performed in a lab environment. > > > > I do have a question regarding the last paragraph of the Security > > Considerations on special capabilities of DUTs for benchmarking > purposes. > > Currently, the sentence reads: "Special capabilities SHOULD NOT > exist in > > the > > DUT/SUT specifically for benchmarking purposes." Why is this a > SHOULD NOT > > and > > not a MUST NOT? Could you give an example when such special > capabilities > > in a > > DUT are appropriate? > [acm] > We can only make a strong recommendation in this area. As > testers/benchmarkers are often independent from the DUT developers and > conduct testing external to the DUT, we assume honesty among other parties > but we cannot require it. If someone constructed a DUT that recognized > test conditions and operated differently to perform better somehow, our > tests would measure the intended "better" performance. It takes a > special/additional test effort to prove that a DUT has "designed to the > test" (consider Volkswagen and fuel efficiency testing [0]). > > We simply do not have any authority in this matter, but we can let all > parties know that gaming the test can be discovered and reported (albeit > with more testing that we do not describe). > > [0] https://urldefense.com/v3/__https://www.consumerreports.org/fuel- > economy-efficiency/volkswagen-used-special-software-to-exaggerate-fuel- > economy/__;!!BhdT!0KS_VCF5ZQfIGkVyPLoJXuAxdcoS3- > xJTE0LoKZPWuSiHjQZM1u0H9M36YXByCk$ > > > > > > > > > _______________________________________________ > > bmwg mailing list > > bmwg@xxxxxxxx > > > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/bmwg__;! > > !BhdT!1JFeLsENzMU-ew89jxmJKxfp4wj5Zo3AZ6V8iULU3hWAentH1dymqJmDOvw7$ > > -- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
