Sounds good, thanks!
Mališa
On 16/12/2020 18:44, "MORTON, ALFRED C (AL)" <acm@xxxxxxxxxxxxxxxx> wrote:
Hi Mališa,
Thanks for your proposed wording, it seems sufficiently neutral and with a few small tweaks, WFM.
I see that Roman's COMMENT also supports this additional text.
So, consider it part of the next version, and thanks for your help!
Al
> -----Original Message-----
> From: Mališa Vučinić [mailto:malisa.vucinic@xxxxxxxx]
> Sent: Wednesday, December 16, 2020 7:22 AM
> To: MORTON, ALFRED C (AL) <acm@xxxxxxxxxxxxxxxx>; secdir@xxxxxxxx
> Cc: last-call@xxxxxxxx; bmwg@xxxxxxxx; draft-ietf-bmwg-b2b-
> frame.all@xxxxxxxx
> Subject: Re: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03
>
> Al,
>
> I don't have a strong opinion on using the term "honesty" here. How about
> this phrasing, just before the last paragraph in Security Considerations:
>
> The DUT developers are commonly independent from the personnel and
> institutions conducting the benchmarking.
> The DUT developers might have incentives to alter the performance of the
> DUT if the test conditions are detected.
> Procedures described in this document are not designed to detect such
> activity.
> Additional testing, outside of the scope of this document, is needed and
> has been successfully used in the past to discover such malpractices.
>
> Mališa
>
> On 15/12/2020 20:22, "MORTON, ALFRED C (AL)" <acm@xxxxxxxxxxxxxxxx> wrote:
>
> Hi Mališa,
> please see below...
>
> > -----Original Message-----
> > From: Mališa Vučinić [mailto:malisa.vucinic@xxxxxxxx]
> > Sent: Tuesday, December 15, 2020 9:21 AM
> > To: MORTON, ALFRED C (AL) <acm@xxxxxxxxxxxxxxxx>; secdir@xxxxxxxx
> > Cc: last-call@xxxxxxxx; bmwg@xxxxxxxx; draft-ietf-bmwg-b2b-
> > frame.all@xxxxxxxx
> > Subject: Re: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-
> frame-03
> >
> > Hi Al,
> >
> > Thanks, that is clear. I think that discussing the assumption of
> honesty
> > among the parties involved in benchmarking would be a useful
> addition to
> > the Security Considerations section in the draft.
> [acm]
>
> I don't mind explaining the requirement using the term "honesty", but
> I can only imagine raised eyebrows and subsequent DISCUSS/comments if we
> try to assert a need for/assumption of honesty anywhere in the memo.
>
> Do you have suggested wording?
>
> Do others have opinions whether or not this is needed?
>
> thanks,
> Al
>
> >
> > Mališa
> >
> > On 15/12/2020 14:45, "MORTON, ALFRED C (AL)" <acm@xxxxxxxxxxxxxxxx>
> wrote:
> >
> > Hi Mališa,
> > thanks for your review, please see below for one reply to your
> > question (acm].
> > Al
> >
> > > -----Original Message-----
> > > From: bmwg [mailto:bmwg-bounces@xxxxxxxx] On Behalf Of Mališa
> > Vucinic via
> > > Datatracker
> > > Sent: Tuesday, December 15, 2020 6:30 AM
> > > To: secdir@xxxxxxxx
> > > Cc: last-call@xxxxxxxx; bmwg@xxxxxxxx; draft-ietf-bmwg-b2b-
> > > frame.all@xxxxxxxx
> > > Subject: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-
> frame-
> > 03
> > >
> > > Reviewer: Mališa Vučinić
> > > Review result: Ready
> > >
> > > I reviewed this document as part of the Security Directorate's
> > ongoing
> > > effort
> > > to review all IETF documents being processed by the IESG.
> These
> > comments
> > > were
> > > written primarily for the benefit of the Security Area
> Directors.
> > Document
> > > authors, document editors, and WG chairs should treat these
> comments
> > just
> > > like
> > > any other IETF Last Call comments.
> > >
> > > Thank you for this well-written document, it was a pleasure to
> read
> > and I
> > > think
> > > it is ready to proceed. Since the document updates RFC2544
> > benchmarking
> > > procedure for estimating the buffer time of a Device Under
> Test
> > (DUT), it
> > > does
> > > not raise any security issues. Security Considerations section
> is
> > quite
> > > clear
> > > and it stresses that these tests are performed in a lab
> environment.
> > >
> > > I do have a question regarding the last paragraph of the
> Security
> > > Considerations on special capabilities of DUTs for
> benchmarking
> > purposes.
> > > Currently, the sentence reads: "Special capabilities SHOULD
> NOT
> > exist in
> > > the
> > > DUT/SUT specifically for benchmarking purposes." Why is this a
> > SHOULD NOT
> > > and
> > > not a MUST NOT? Could you give an example when such special
> > capabilities
> > > in a
> > > DUT are appropriate?
> > [acm]
> > We can only make a strong recommendation in this area. As
> > testers/benchmarkers are often independent from the DUT developers
> and
> > conduct testing external to the DUT, we assume honesty among other
> parties
> > but we cannot require it. If someone constructed a DUT that
> recognized
> > test conditions and operated differently to perform better somehow,
> our
> > tests would measure the intended "better" performance. It takes a
> > special/additional test effort to prove that a DUT has "designed to
> the
> > test" (consider Volkswagen and fuel efficiency testing [0]).
> >
> > We simply do not have any authority in this matter, but we can
> let all
> > parties know that gaming the test can be discovered and reported
> (albeit
> > with more testing that we do not describe).
> >
> > [0]
> https://urldefense.com/v3/__https://www.consumerreports.org/fuel-
> > economy-efficiency/volkswagen-used-special-software-to-exaggerate-
> fuel-
> > economy/__;!!BhdT!0KS_VCF5ZQfIGkVyPLoJXuAxdcoS3-
> > xJTE0LoKZPWuSiHjQZM1u0H9M36YXByCk$
> >
> > >
> > >
> > >
> > > _______________________________________________
> > > bmwg mailing list
> > > bmwg@xxxxxxxx
> > >
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/bmwg__;!
> > > !BhdT!1JFeLsENzMU-
> ew89jxmJKxfp4wj5Zo3AZ6V8iULU3hWAentH1dymqJmDOvw7$
> >
> >
>
>
>
--
last-call mailing list
last-call@xxxxxxxx
https://www.ietf.org/mailman/listinfo/last-call
