Hi David, Thanks for your review and comment! Please see a proposed resolution below, [acm] Al > -----Original Message----- > From: David Black via Datatracker [mailto:noreply@xxxxxxxx] > Sent: Tuesday, November 24, 2020 10:27 AM > To: tsv-art@xxxxxxxx > Cc: bmwg@xxxxxxxx; draft-ietf-bmwg-b2b-frame.all@xxxxxxxx; last- > call@xxxxxxxx > Subject: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03 > > Reviewer: David Black > Review result: Ready with Issues > ... > > This draft updates the back-to-back frame testing procedure in RFC 2544 to > take > account of experience. > > The draft is in good shape, with one notable exception in Section 5.2: > > The duration of the trial MUST be at least 2 seconds, to allow DUT > buffers to deplete. > > That duration of 2 seconds has been carried forward from RFC 2544 without > change. A 2 second duration may have been sufficient to deplete buffers in > 1999, but that is no longer reliably the case. For example, on-site > measurement of the network for the 2020 Linux Conference in Australia indicated > a at least 1.6 seconds of buffering, as indicated by Figure 1 at > https://urldefense.com/v3/__https://blog.apnic.net/2020/01/22/bufferbloat- > may-be-solved-but-its-not-over- > yet/__;!!BhdT!wOuQE0NajXs4dT7tdIMVQU5FFpb0JiU0- > yK2DOVVn0ecoYjf7mFEABLmlwDk$ , > which is slide 6 in from the complete slide deck at: > https://urldefense.com/v3/__https://blog.apnic.net/2020/01/22/bufferbloat- > may-be-solved-but-its-not-over- > yet/__;!!BhdT!wOuQE0NajXs4dT7tdIMVQU5FFpb0JiU0- > yK2DOVVn0ecoYjf7mFEABLmlwDk$ > . Experience with bufferbloat suggests that one network device was primarily > responsible. Also, see slide 14 in that slide deck. > > That 1.6 seconds measured on an actual network is entirely too close to 2 > seconds for confidence that buffers will be depleted in any tested device. > Hence, the 2 second minimum duration ought to be increased by at least a factor > of 10. I'd suggest changing it to 30 seconds or 60 seconds as convenient round > numbers, and providing the rationale that increased buffering in WiFi devices, > e.g., home "routers," as indicated by experience with bufferbloat measurements, > is the reason for the increased duration. > [acm] [acm] I agree with your observation that there are cases where trial duration should be increased to accommodate the encountered in the DUT, but not as a mandate for all testing. I have four factors in mind: 1. Some of the virtual network DUTs we are testing now have very small buffers, and the B2B stream of frames is quite short -- less than 2000 frames@10GE in some cases -- so 2 seconds fully sufficient. 2. The trial duration is a factor in total test duration, where each trial is one step in the Binary Search. We need to manage the tension between the time needed to reach a search result and confidence that we have depleted the queues. 3. The RFC 2544 Latency benchmark will tell us if bufferbloat is present. 4. The current text says "at least 2 seconds". So I suggest adding the following text: The duration of the trial MUST be at least 2 seconds, to allow DUT buffers to deplete. When RFC2544 Latency measurements indicate that large buffers are present in the DUT, the trial duration SHOULD be increased to ensure that buffer depletion takes place, without unduly extending the total test time. I hope this suggestion resolves your issue; thanks for highlighting it! Al -- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
