[Last-Call] Re: [bmwg] Tsvart last call review of draft-ietf-bmwg-benchmarking-stateful-06

["Scharf, Michael" <Michael.Scharf@xxxxxxxxxxxxxxx>]

Hi Gabor,

 

Thanks a lot for the fast reply. Your changes work for me.

 

Best regards

 

Michael

 

 

From: Gábor LENCSE <lencse@xxxxxxxxxx>
Sent: Wednesday, May 22, 2024 4:08 PM
To: Scharf, Michael <Michael.Scharf@xxxxxxxxxxxxxxx>
Cc: tsv-art@xxxxxxxx; bmwg@xxxxxxxx; draft-ietf-bmwg-benchmarking-stateful.all@xxxxxxxx; last-call@xxxxxxxx
Subject: Re: [bmwg] Tsvart last call review of draft-ietf-bmwg-benchmarking-stateful-06

 

Dear Michael,

Thank you very much for your review and comments. Please see my answers inline.

2024.05.20. 12:31 keltezéssel, Michael Scharf via Datatracker írta:

Reviewer: Michael Scharf

Review result: Ready with Issues

This document has been reviewed as part of the transport area review team's

ongoing effort to review key IETF documents. These comments were written

primarily for the transport area directors, but are copied to the document's

authors and WG to allow them to address any issues raised and also to the IETF

discussion list for information.

When done at the time of IETF Last Call, the authors should consider this

review as part of the last-call comments they receive. Please always CC

tsv-art@xxxxxxxx if you reply to or forward this review.

The TSV-ART review for this informational document was requested with a

deadline of only 4 days during Pentecost holidays in Europe. As a result, this

TSV-ART review only represents a quick check.

I agree that this is an unfortunate situation, which happened beyond our control. If you could dedicate more time for a more through review, I would be happy to receive further comments that can improve the quality of our Internet Draft. If you can do a second review, please let me know, when you can send your more detailed review and I will wait for it.

I write here the planned changes, but they will remain in my XML file, as I am waiting for several further reviews and I plan to submit a new version of the draft when all reviews are addressed and they are deemed satisfactory by the corresponding Reviewers.

At first sight, there are some issue related to transport protocols. They are

probably not hard to address in a revised draft:

- In section 8, the document assumes that HTTP(S) uses TCP. This is correct for

HTTP/1.x and HTTP/2, but not for HTTP/3, as HTTP/3 uses QUIC. The statement

regarding HTTP in Section 8 should be reworded.

Thank you for pointing it out. I have split the last paragraph of Section 8 into two paragraphs as follows:

   As a mitigation of this problem, this document recommends that

   testing with protocols using TCP (like HTTP and HTTPS up to version

   2) can be performed as described in [RFC9411].  This approach also

   solves the potential problem of protocol helpers may be present in

   the stateful DUT.

   As for HTTP/3, it uses QUIC, which uses UDP as stated above.  It

   should be noted that QUIC is treated as any other UDP payload.  The

   proposed measurement method does not aim to measure the performance

   of QUIC, rather it aims to measure the performance of the stateful

   NATxy gateway.

- Section 4 should more precisely define what a "connection" is in the context

of the benchmark test. The text probably assumes "TCP connection" in a number

of places, but this should be made very explicit. Some use of the word

"connection" probably also applies to all transport protocols, including TCP

and UDP, but this is not always true. For instance, Section 4.8 ("Connection

Tear-Down") seems to focus only on TCP. In all descriptions of tests, it should

be made very clear what transport protocol is assumed.

Indeed, the definition of the "connection" was missing from the terminology. I added the following definition prior to the definition of "Connection tracking table":

   *  Connection: Although UDP itself is a connection-less protocol,

      stateful NATxy gateways keep track of their translation mappings

      in the form of a "connection" also in the case of UDP using the

      same kind of entries as in the case of TCP.

And I hope that this definition made it clear that "connection tear-down" refers to the deletion of the "connections" (that is, translation mappings) from the connection tracking table of the DUT. And it is the same for TCP and UDP.

- The document as a whole does not consider implications of use of QUIC. Given

the increasing share of QUIC in the total Internet traffic, this is somewhat

surprising. It could make sense to have at least some short discussion on the

impact of QUIC on the benchmarking, e.g., in section 8.

- The document only mentions TCP and UDP as Internet transport protocols. It

could make sense to at least mention somewhere other that other transport

protocols exist (SCTP, possibly also DCCP), possibly as a topic outside the

scope of the document. Similarly, issues resulting from multipath transport

(e.g., MPTCP) are not discussed in the document. That could be also stated

explicitly.

We addressed the two comments together by the following paragraph in Section 8:

   Further potential transport layer protocols e.g., DCCP [RFC4340] and

   SCTP [RFC9260] are outside of the scope of this document, as the

   widely-used stateful NAT44 and stateful NAT64 implementations do not

   support them.  Although QUIC [RFC9000] is also considered a transport

   layer protocol, but QUIC packets are carried in UDP datagrams thus

   QUIC does not need a special handling.

In general, I am not sure if it makes sense to use RFC2119/RFC8174 language in

such an informational document. At least in some cases the use of

RFC2119/RFC8174 language does not really affect interoperability. An example

would be Section 6.

Even though BMWG only publishes Informational documents, previous BMWG RFCs (e.g., RFC 2544, RFC 5180) became de facto standards in benchmarking and they do use normative language. (RFC 2544 defined the terms, RFC 5180 cited RFC 2119.)

Nit: s/athours/authors/ in Secion 8

Thank you for picking it. I have corrected it.

I think it can be useful to copy here the entire text of the new Section 8 so that it can be seen in context.

8.  Limitations of using UDP as Transport Layer Protocol

   The test frame format defined in RFC 2544 exclusively uses UDP (and

   not TCP) as a transport layer protocol.  Testing with UDP was kept in

   both RFC 5180 and RFC 8219 regarding the standard benchmarking

   procedures (throughput, latency, frame loss rate, etc.).  The

   benchmarking methodology proposed in this document follows this long

   established benchmarking tradition using UDP as a transport layer

   protocol, too.  The rationale for this is that the standard

   benchmarking procedures require sending frames at arbitrary constant

   frame rates, which would violate the flow control and congestion

   control algorithms of the TCP protocol.  TCP connection setup (using

   the three-way handshake) would further complicate testing.

   Further potential transport layer protocols e.g., DCCP [RFC4340] and

   SCTP [RFC9260] are outside of the scope of this document, as the

   widely-used stateful NAT44 and stateful NAT64 implementations do not

   support them.  Although QUIC [RFC9000] is also considered a transport

   layer protocol, but QUIC packets are carried in UDP datagrams thus

   QUIC does not need a special handling.

   Some stateful NATxy solutions handle TCP and UDP differently, e.g.

   iptables uses 30s timeout for UDP and 60s timeout for TCP.  Thus

   benchmarking results produced using UDP do not necessarily

   characterize the performance of a NATxy gateway well enough when they

   are used for forwarding Internet traffic.  As for the given example,

   timeout values of the DUT may be adjusted, but it requires extra

   consideration.

   Other differences in handling UDP or TCP are also possible.  Thus,

   the authors recommend that further investigations should be performed

   in this field.

   As a mitigation of this problem, this document recommends that

   testing with protocols using TCP (like HTTP and HTTPS up to version

   2) can be performed as described in [RFC9411].  This approach also

   solves the potential problem of protocol helpers may be present in

   the stateful DUT.

   As for HTTP/3, it uses QUIC, which uses UDP as stated above.  It

   should be noted that QUIC is treated as any other UDP payload.  The

   proposed measurement method does not aim to measure the performance

   of QUIC, rather it aims to measure the performance of the stateful

   NATxy gateway.

Once again, thank you very much for your review and comments, which have helped us to improve our Internet Draft.

Best regards,

Gábor Lencse

_______________________________________________

bmwg mailing list -- bmwg@xxxxxxxx

To unsubscribe send an email to bmwg-leave@xxxxxxxx

 

<<attachment: smime.p7s>>

-- 
last-call mailing list -- last-call@xxxxxxxx
To unsubscribe send an email to last-call-leave@xxxxxxxx