Thank you very much for your review and comments. Please see my answers inline.
2024.05.20. 12:31 keltezéssel, Michael
Scharf via Datatracker írta:
Reviewer: Michael Scharf Review result: Ready with Issues This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information. When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC tsv-art@xxxxxxxx if you reply to or forward this review. The TSV-ART review for this informational document was requested with a deadline of only 4 days during Pentecost holidays in Europe. As a result, this TSV-ART review only represents a quick check.
I agree that this is an unfortunate situation, which happened beyond our control. If you could dedicate more time for a more through review, I would be happy to receive further comments that can improve the quality of our Internet Draft. If you can do a second review, please let me know, when you can send your more detailed review and I will wait for it.
I write here the planned changes, but they will remain in my XML file, as I am waiting for several further reviews and I plan to submit a new version of the draft when all reviews are addressed and they are deemed satisfactory by the corresponding Reviewers.
At first sight, there are some issue related to transport protocols. They are probably not hard to address in a revised draft: - In section 8, the document assumes that HTTP(S) uses TCP. This is correct for HTTP/1.x and HTTP/2, but not for HTTP/3, as HTTP/3 uses QUIC. The statement regarding HTTP in Section 8 should be reworded.
Thank you for pointing it out. I have split the last paragraph of Section 8 into two paragraphs as follows:
As a mitigation of this problem, this document recommends that testing with protocols using TCP (like HTTP and HTTPS up to version 2) can be performed as described in [RFC9411]. This approach also solves the potential problem of protocol helpers may be present in the stateful DUT. As for HTTP/3, it uses QUIC, which uses UDP as stated above. It should be noted that QUIC is treated as any other UDP payload. The proposed measurement method does not aim to measure the performance of QUIC, rather it aims to measure the performance of the stateful NATxy gateway.
- Section 4 should more precisely define what a "connection" is in the context
of the benchmark test. The text probably assumes "TCP connection" in a number
of places, but this should be made very explicit. Some use of the word
"connection" probably also applies to all transport protocols, including TCP
and UDP, but this is not always true. For instance, Section 4.8 ("Connection
Tear-Down") seems to focus only on TCP. In all descriptions of tests, it should
be made very clear what transport protocol is assumed.
Indeed, the definition of the "connection" was missing from the terminology. I added the following definition prior to the definition of "Connection tracking table":
* Connection: Although UDP itself is a connection-less protocol,
stateful NATxy gateways keep track of their translation mappings
in the form of a "connection" also in the case of UDP using the
same kind of entries as in the case of TCP.
And I hope that this definition made it clear that "connection
tear-down" refers to the deletion of the "connections" (that is,
translation mappings) from the connection tracking table of the DUT.
And it is the same for TCP and UDP.- The document as a whole does not consider implications of use of QUIC. Given the increasing share of QUIC in the total Internet traffic, this is somewhat surprising. It could make sense to have at least some short discussion on the impact of QUIC on the benchmarking, e.g., in section 8. - The document only mentions TCP and UDP as Internet transport protocols. It could make sense to at least mention somewhere other that other transport protocols exist (SCTP, possibly also DCCP), possibly as a topic outside the scope of the document. Similarly, issues resulting from multipath transport (e.g., MPTCP) are not discussed in the document. That could be also stated explicitly.
We addressed the two comments together by the following paragraph in Section 8:
Further potential transport layer protocols e.g., DCCP [RFC4340] and
SCTP [RFC9260] are outside of the scope of this document, as the
widely-used stateful NAT44 and stateful NAT64 implementations do not
support them. Although QUIC [RFC9000] is also considered a transport
layer protocol, but QUIC packets are carried in UDP datagrams thus
QUIC does not need a special handling.
In general, I am not sure if it makes sense to use RFC2119/RFC8174 language in such an informational document. At least in some cases the use of RFC2119/RFC8174 language does not really affect interoperability. An example would be Section 6.
Even though BMWG only publishes Informational documents, previous BMWG RFCs (e.g., RFC 2544, RFC 5180) became de facto standards in benchmarking and they do use normative language. (RFC 2544 defined the terms, RFC 5180 cited RFC 2119.)
Nit: s/athours/authors/ in Secion 8
Thank you for picking it. I have corrected it.
I think it can be useful to copy here the entire text of the new Section 8 so that it can be seen in context.
8. Limitations of using UDP as Transport Layer Protocol The test frame format defined in RFC 2544 exclusively uses UDP (and not TCP) as a transport layer protocol. Testing with UDP was kept in both RFC 5180 and RFC 8219 regarding the standard benchmarking procedures (throughput, latency, frame loss rate, etc.). The benchmarking methodology proposed in this document follows this long established benchmarking tradition using UDP as a transport layer protocol, too. The rationale for this is that the standard benchmarking procedures require sending frames at arbitrary constant frame rates, which would violate the flow control and congestion control algorithms of the TCP protocol. TCP connection setup (using the three-way handshake) would further complicate testing. Further potential transport layer protocols e.g., DCCP [RFC4340] and SCTP [RFC9260] are outside of the scope of this document, as the widely-used stateful NAT44 and stateful NAT64 implementations do not support them. Although QUIC [RFC9000] is also considered a transport layer protocol, but QUIC packets are carried in UDP datagrams thus QUIC does not need a special handling. Some stateful NATxy solutions handle TCP and UDP differently, e.g. iptables uses 30s timeout for UDP and 60s timeout for TCP. Thus benchmarking results produced using UDP do not necessarily characterize the performance of a NATxy gateway well enough when they are used for forwarding Internet traffic. As for the given example, timeout values of the DUT may be adjusted, but it requires extra consideration. Other differences in handling UDP or TCP are also possible. Thus, the authors recommend that further investigations should be performed in this field. As a mitigation of this problem, this document recommends that testing with protocols using TCP (like HTTP and HTTPS up to version 2) can be performed as described in [RFC9411]. This approach also solves the potential problem of protocol helpers may be present in the stateful DUT. As for HTTP/3, it uses QUIC, which uses UDP as stated above. It should be noted that QUIC is treated as any other UDP payload. The proposed measurement method does not aim to measure the performance of QUIC, rather it aims to measure the performance of the stateful NATxy gateway.Once again, thank you very much for your review and comments, which have helped us to improve our Internet Draft.
Best regards,
Gábor Lencse
_______________________________________________ bmwg mailing list -- bmwg@xxxxxxxx To unsubscribe send an email to bmwg-leave@xxxxxxxx
-- last-call mailing list -- last-call@xxxxxxxx To unsubscribe send an email to last-call-leave@xxxxxxxx
