[Last-Call] Re: [bmwg] Tsvart last call review of draft-ietf-bmwg-benchmarking-stateful-06

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dear Michael,

Thank you very much for your review and comments. Please see my answers inline.

2024.05.20. 12:31 keltezéssel, Michael Scharf via Datatracker írta:
Reviewer: Michael Scharf
Review result: Ready with Issues

This document has been reviewed as part of the transport area review team's
ongoing effort to review key IETF documents. These comments were written
primarily for the transport area directors, but are copied to the document's
authors and WG to allow them to address any issues raised and also to the IETF
discussion list for information.

When done at the time of IETF Last Call, the authors should consider this
review as part of the last-call comments they receive. Please always CC
tsv-art@xxxxxxxx if you reply to or forward this review.

The TSV-ART review for this informational document was requested with a
deadline of only 4 days during Pentecost holidays in Europe. As a result, this
TSV-ART review only represents a quick check.

I agree that this is an unfortunate situation, which happened beyond our control. If you could dedicate more time for a more through review, I would be happy to receive further comments that can improve the quality of our Internet Draft. If you can do a second review, please let me know, when you can send your more detailed review and I will wait for it.

I write here the planned changes, but they will remain in my XML file, as I am waiting for several further reviews and I plan to submit a new version of the draft when all reviews are addressed and they are deemed satisfactory by the corresponding Reviewers.
At first sight, there are some issue related to transport protocols. They are
probably not hard to address in a revised draft:

- In section 8, the document assumes that HTTP(S) uses TCP. This is correct for
HTTP/1.x and HTTP/2, but not for HTTP/3, as HTTP/3 uses QUIC. The statement
regarding HTTP in Section 8 should be reworded.

Thank you for pointing it out. I have split the last paragraph of Section 8 into two paragraphs as follows:

   As a mitigation of this problem, this document recommends that
   testing with protocols using TCP (like HTTP and HTTPS up to version
   2) can be performed as described in [RFC9411].  This approach also
   solves the potential problem of protocol helpers may be present in
   the stateful DUT.

   As for HTTP/3, it uses QUIC, which uses UDP as stated above.  It
   should be noted that QUIC is treated as any other UDP payload.  The
   proposed measurement method does not aim to measure the performance
   of QUIC, rather it aims to measure the performance of the stateful
   NATxy gateway.

- Section 4 should more precisely define what a "connection" is in the context
of the benchmark test. The text probably assumes "TCP connection" in a number
of places, but this should be made very explicit. Some use of the word
"connection" probably also applies to all transport protocols, including TCP
and UDP, but this is not always true. For instance, Section 4.8 ("Connection
Tear-Down") seems to focus only on TCP. In all descriptions of tests, it should
be made very clear what transport protocol is assumed.

Indeed, the definition of the "connection" was missing from the terminology. I added the following definition prior to the definition of "Connection tracking table":
   *  Connection: Although UDP itself is a connection-less protocol,
      stateful NATxy gateways keep track of their translation mappings
      in the form of a "connection" also in the case of UDP using the
      same kind of entries as in the case of TCP.

And I hope that this definition made it clear that "connection tear-down" refers to the deletion of the "connections" (that is, translation mappings) from the connection tracking table of the DUT. And it is the same for TCP and UDP.

- The document as a whole does not consider implications of use of QUIC. Given
the increasing share of QUIC in the total Internet traffic, this is somewhat
surprising. It could make sense to have at least some short discussion on the
impact of QUIC on the benchmarking, e.g., in section 8.

- The document only mentions TCP and UDP as Internet transport protocols. It
could make sense to at least mention somewhere other that other transport
protocols exist (SCTP, possibly also DCCP), possibly as a topic outside the
scope of the document. Similarly, issues resulting from multipath transport
(e.g., MPTCP) are not discussed in the document. That could be also stated
explicitly.

We addressed the two comments together by the following paragraph in Section 8:
   Further potential transport layer protocols e.g., DCCP [RFC4340] and
   SCTP [RFC9260] are outside of the scope of this document, as the
   widely-used stateful NAT44 and stateful NAT64 implementations do not
   support them.  Although QUIC [RFC9000] is also considered a transport
   layer protocol, but QUIC packets are carried in UDP datagrams thus
   QUIC does not need a special handling.

In general, I am not sure if it makes sense to use RFC2119/RFC8174 language in
such an informational document. At least in some cases the use of
RFC2119/RFC8174 language does not really affect interoperability. An example
would be Section 6.

Even though BMWG only publishes Informational documents, previous BMWG RFCs (e.g., RFC 2544, RFC 5180) became de facto standards in benchmarking and they do use normative language. (RFC 2544 defined the terms, RFC 5180 cited RFC 2119.)

Nit: s/athours/authors/ in Secion 8

Thank you for picking it. I have corrected it.

I think it can be useful to copy here the entire text of the new Section 8 so that it can be seen in context.

8.  Limitations of using UDP as Transport Layer Protocol

   The test frame format defined in RFC 2544 exclusively uses UDP (and
   not TCP) as a transport layer protocol.  Testing with UDP was kept in
   both RFC 5180 and RFC 8219 regarding the standard benchmarking
   procedures (throughput, latency, frame loss rate, etc.).  The
   benchmarking methodology proposed in this document follows this long
   established benchmarking tradition using UDP as a transport layer
   protocol, too.  The rationale for this is that the standard
   benchmarking procedures require sending frames at arbitrary constant
   frame rates, which would violate the flow control and congestion
   control algorithms of the TCP protocol.  TCP connection setup (using
   the three-way handshake) would further complicate testing.

   Further potential transport layer protocols e.g., DCCP [RFC4340] and
   SCTP [RFC9260] are outside of the scope of this document, as the
   widely-used stateful NAT44 and stateful NAT64 implementations do not
   support them.  Although QUIC [RFC9000] is also considered a transport
   layer protocol, but QUIC packets are carried in UDP datagrams thus
   QUIC does not need a special handling.

   Some stateful NATxy solutions handle TCP and UDP differently, e.g.
   iptables uses 30s timeout for UDP and 60s timeout for TCP.  Thus
   benchmarking results produced using UDP do not necessarily
   characterize the performance of a NATxy gateway well enough when they
   are used for forwarding Internet traffic.  As for the given example,
   timeout values of the DUT may be adjusted, but it requires extra
   consideration.

   Other differences in handling UDP or TCP are also possible.  Thus,
   the authors recommend that further investigations should be performed
   in this field.

   As a mitigation of this problem, this document recommends that
   testing with protocols using TCP (like HTTP and HTTPS up to version
   2) can be performed as described in [RFC9411].  This approach also
   solves the potential problem of protocol helpers may be present in
   the stateful DUT.

   As for HTTP/3, it uses QUIC, which uses UDP as stated above.  It
   should be noted that QUIC is treated as any other UDP payload.  The
   proposed measurement method does not aim to measure the performance
   of QUIC, rather it aims to measure the performance of the stateful
   NATxy gateway.

Once again, thank you very much for your review and comments, which have helped us to improve our Internet Draft.

Best regards,

Gábor Lencse

_______________________________________________
bmwg mailing list -- bmwg@xxxxxxxx
To unsubscribe send an email to bmwg-leave@xxxxxxxx


-- 
last-call mailing list -- last-call@xxxxxxxx
To unsubscribe send an email to last-call-leave@xxxxxxxx

[Index of Archives]     [IETF Annoucements]     [IETF]     [IP Storage]     [Yosemite News]     [Linux SCTP]     [Linux Newbies]     [Mhonarc]     [Fedora Users]

  Powered by Linux