Hi, I read this document again as part of its second Last Call. I have a few comments that should ideally be fixed before passing the draft on to the RFC Editor. (I ran out of steam around Section 6, sorry.) Thanks, Adrian === I wondered about the implementation status of this document. One might say that an Informational I-D has nothing to be implemented, but this document seems to be telling us which elements of other RFCs to use and combine to make a working system. Seeing that some of my comments note that the text appears to recommend using a deprecated code point, and that the BESS wiki notes "Implementation Status" as one of the working group last call checklist items, I thought it might be nice if this document has an RFC 7942 section to help us know how solid the processes are. --- Please fix John Drake's contact details. --- You have a different number of people on the front page and in the Authors' Addresses section. --- The RFC Editor is going to ask you whether you can find a different way of saying "traditional" in the many cases in this document. Mainly, I think you can just delete the word. --- The running footer seems to be broken ("xxx, et al.") --- The document title and abstract should not assume that the reader is familiar with the abbreviation "SD-WAN" --- Why does the document title say "overlay networks" while the Abstract says "multiple scenarios". --- Why isn't [MEF70.1] a normative reference? It seems that this document leans on it heavily for the definition of SD-WAN and for other material. --- 1. s/IP Packets/IP packets/ --- 1. - Some traffic can be forwarded by edge nodes, based on their application identifiers instead of destination IP addresses I think this is unintentionally ambiguous. Presumably it is not the application identifiers of the edge nodes. I believe you are talking about traffic steering, although "forwarding" may be an acceptable term. We normally think about forwarding onto a link or toward a next hop, and steering onto a path. --- 1. - Some traffic can be forwarded by edge nodes, based on their application identifiers instead of destination IP addresses, by placing the traffic onto specific overlay paths based on the application-specific policies. An "application identifier" in this document refers to one or multiple fields of the IP header of the packets. I think this use of "application identifier" (and, later, "recognizing applications") is significantly misleading. At best, what you have here is a "flow identifier". Further, you say that this is done "instead of the destination IP address", yet the destination IP address is surely a "field of the IP header of the packet". (By the way, by the time you get to Section 3.3, you are talking about flows.) You go on to say: More detailed attributes that can be used for identifying applications are described in the Table7 and Table 8 of [MEF70.1] Table 7 includes fields that are not part of the IP header. - In general, matching on source and destination port IDs is considered an acceptable way of identifying traffic flows. But this is contrary to what you have previously said and is, in any case, problematic with some forms of encryption. - The concept of "Custom match including heuristic/algorithmic matching" is clearly not intended to relate purely to fields in the IP header, but includes some form of DPI. It is arguable that an SD-WAN edge node is part of the customer's network, and that the customer is allowed to inspect any user's traffic within their network. This, however, probably only applies to certain types of customer network. It is also likely to fail in the presence of end-to-end encryption. I do not find this topic discussed further in the draft, and I think it needs to be specifically excluded, or explained in some detail. I refer the AD to the discussion of "APN" within the IETF and to the careful efforts that have been made to attempt to achieve application flow identification for different traffic treatments and for traffic steering at the edge of the network without recourse to DPI. This is, I think a serious problem with the document. Table 8 also includes fields that are not part of the IP header. - Using the Ethertype of the encapsulating Ethernet frame may be a tool that can be used, but you would need to make that clear as it is obviously not part of the IP header. Further, as the note in the table says, not all implementations will have access to the Ethertype of the received frame. It is possible that including a reference to Section 8 of MEF 70.1 might be helpful here, but I note that that document says: Although the term "Application Flow" includes the word "Application" there is only a loose connection between the two. So, while you could continue to use the term "application" in your draft, I think that would be a bad idea. Sure, reference that fact that MEF 70.1 uses the term, but say something like: As noted in [MEF70.1] there is only a loose connection between an "Application" and what [MEF70.1] calls an "Application Flow". To avoid any confusion and remove any implication that individual applications can be identified by SD-WAN edge nodes, this document uses the term "traffic flow" (or simply "flow"). A flow is a collection of packets between the same source and destination pair that are subject to the same forwarding and policy decisions at the ingress SD-WAN edge node, and are identified by the settings of one or more fields in the packet headers. --- 1. [Net2Cloud-Problem] describes the network-related problems relating to connecting enterprises' branch offices to dynamic workloads in different Cloud Data Centers (DC). SD-WAN has been positioned as a flexible way to solve those issues; however, this can create significant scaling issues when hundreds or thousands of nodes need to be interconnected by SD-WAN overlay networks. I was distracted by this paragraph. I'm not sure it adds to the document, and it seems to present some challenges: for example, this document purports to address "large-scale SD-WAN overlay networks" (Abstract) yet here you say that there is a scaling problem for large networks. Asides from this, the only mention of scalability is in Section 5.1 My suggestion would be to scrap this paragraph since it is really only saying that you can use SD-WAN to address some problems of, erm, SD-WAN deployments. If you *do* decide to keep the paragraph, then I suggest adding a new section to the draft for "Scalability Considerations." --- 1. BGP for SD-WAN overlay is a different layer from the underlay networks' BGP control plane instances. I think, more importantly, it is a different BGP instance. Or is it? --- 2. Controller: Used interchangeably with SD-WAN controller to manage SD-WAN overlay path creation/deletion and monitor the path conditions between sites. The overlay paths are somewhat trivial, I believe, seeing that in the overlay all edges are adjacent and the path is a single hop. Reading ahead, the more important (the only?) roles of the controller are to manage subscription of edge nodes to the SD-WAN, to assist with ZTP, and to determine which edges should be connected to which other edges. --- 2. s/A SD-WAN/An SD-WAN/ --- 2. CPE-Based VPN is defined in this section, but the term is not used anywhere in the document. This is somewhat confusing, and looking at the scenarios in section 3, it appears that you intend that all deployments use CPE-CPE security of some form. This appears to be a fundamental component of the SD-WAN that you are describing. Perhaps you need to bring this point out very clearly in the description of what an SD-WAN is. --- 2. SD-WAN IPsec SA: IPsec Security Association between two SD-WAN ports or nodes. The "nodes" in this case are SD-WAN edges (or CPEs). And the "ports" are ports on those nodes. Possibly "WAN ports" q.v. --- 2. SD-WAN over Hybrid Networks The term used in the document seems to be "Hybrid Underlay" --- 2. s/an Network/a Network/ --- 2. It seems to me to be confusing to define a new term "C-PE" which: - doesn't seem to stand for anything - means "SD-WAN Edge node" which is already defined - "can be Customer Premises Equipment (CPE)" which is a very similar abbreviation Why can you not stick with "SD-WAN Edge node"? --- 2. and 3.1.4 Decide between "Zero Touch Provisioning" and "zero-touch provisioning" --- 3.1.1 Please expand VRF and provide a citation --- 3.1.1 As SD-WAN is an overlay network arching over multiple types of networks, MPLS L2VPN/L3VPN or pure L2 underlay can continue using the VPN ID, VN-ID, or VLAN in the data plane to differentiate packets belonging to different SD-WAN VPNs. There are some unexplained abbreviations here. --- 3.1.2 Please expand UNI --- 3.1.2 I had a lot of trouble working out what this section is trying to say. The client service requirements describe the port interface requirement at the SD-WAN edge to connect the client network to the SD-WAN service. The requirements describe the requirement? And what are those requirements? The client interface ports can support IPv4 & IPv6 address prefixes and Ethernet (as described in [IEEE802.3] standard). How does a port support an address prefix? It is worth noting that this "interface" Which interface? is called SD-WAN UNI in [MEF 70.1] with a set of attributes (described in Section 11 in MEF 70.1); these attributes (in MEF 70.1) describe the expected behavior and requirements to support the connectivity to the client network. I presume that this is focused on the case that the SD-WAN edge is a PE not a CPE? The client service should support the SD-WAN UNI service attributes at the SD-WAN edge as described in MEF 70.1, Section 11. What is the "client service"? What does "should" mean here? Isn't it the case that the attributes in MEF 70.1/11 apply at an interface not at a node? Do these attributes apply to the configuration/management of the client service interface, or to the marking of packets on the interface, or to the handling of packets on the interface? --- 3.1.3 For example, a retail business requires the point-of-sales (PoS) application to be on a different topology from other applications. The PoS application is routed only to the payment processing entity at a hub site; other applications can be routed to all other sites. The second sentence is true, but does not justify the asserted requirement in the first sentence. --- 3.1.3 The figure in this section needs to be tidied up, should be labeled, and should be referred to by number in the text. Subsequent figures in the document will need to be renumbered. The figure doesn't really make clear the differences in the topologies. For example, in the figure, and considering the "tree topology" it looks like Site 1 and Site 2 could be connected. Part of the problem here may be that the "topology" relates to the underlay (which is not the business of the SD-WAN service or customer), while what you probably want to describe is the connectivity services in the two cases (which are multipoint-to-point, and any-to-any). Maybe "connectivity matrix" is the term you need in place of "topology". The final paragraph of the section seems to be talking about both different connectivity requirements for different traffic flows, as well as different service demands for those flows. --- 3.1.4 It might be helpful to move the definition of ZTP from the last para in the section to be the first. --- 3.1.5 OLD edges' loopback address NEW1 edges' loopback addresses NEW2 edge's loopback address END --- 3.1.5 One SD-WAN edge node may only be authorized to communicate with a small number of other SD-WAN edge nodes. In this circumstance, the property of the SD-WAN edge node cannot be propagated to other nodes that are not authorized to communicate. Why "cannot" instead of "need not"? after all, if two nodes are not authorized to communicate then any attempt to communicate will fail authorization policy checks. The knowledge of edge node properties is useless, but harmless (except possibly for scaling issues). Unless, of course, you are proposing "authorization by obscurity". --- 3.1.5 s/insecure/unsecured/ --- 3.1.5 The text should make reference to the figure. The figure is the first mention of "peer groups" --- I feel that the references to [SECURE-EVPN] are (or are very close to being) Normative. --- 3.2 Figure 2 needs some tidying up. The figure should be referenced from the text. The key to the figure is the first and only mention of NVo3 in the document. The IETF appears to use "NVO3". You should either correct that, expand the abbreviation, and provide a citation, or simply drop the text. --- 3.3 Please expand "SLA" --- 3.3 Since IPsec requires additional processing power and the encrypted traffic over the Internet does not have the premium SLA commonly offered by Private VPNs, especially over a long distance, it is more desirable for traffic over a private VPN to be forwarded without encryption. This seems to be putting it too strongly! s/is more desirable/may be acceptable/ Actually, the SLA of traffic over the Internet has nothing to do with how traffic is handled on the Private VPN. What you are possibly saying is that the high performance SLAs commonly offered by Private VPNs mean that it may not be possible to deliver traffic that both meets the SLA and is subject to edge-to-edge encryption. By the way, the term "private VPN" (used in various places in the document) is a little odd. "Private Virtual Private Network"? I suspect that a "private VPN" may be a VPN that is supported wholly by a single network service provider without using any elements of the public Internet and without any traffic passing out of the immediate control of that service provider. Perhaps you could add the term to Section 2. --- 3.3 3) Some flows, especially Internet-bound browsing ones, can be handed off to the Internet without any encryption. That is probably "without any further encryption" because such flows are probably already encrypted. --- 3.3 Suppose a flow traversing multiple segments, such as A<->B<->C<- >D, has Policy 2) above. The flow can cross different underlays in different segments, such as over Private underlay between A<->B without encryption or over the public Internet between B<->C protected by an IPsec SA. This is not the same use of "segment" as in 3.1.1 or 3.1.3, or is it? In any case, surely the SD-WAN edge only has control over the edge function. Thus, if there is the possibility that one segment will be over the public Internet, the SD-WAN edge is going to need to perform end-to-end encryption (meaning that traffic over the private segments is also encrypted. That is, the SD-WAN edge, while it can choose the forwarding path out of its various ports, cannot control whether some transit PE (possibly an ASBR) provides additional function (such as encryption over the segment). Of course, it may be that you do not mean "Suppose a flow traversing multiple segments, such as A<->B<->C<->D". You might mean, "Suppose a service includes flows traversing multiple segments, such as A<->B, A<->D, C<->B, and C<->D as shown in Figure 3". But I am only guessing. --- 3.4 This scenario refers to an existing VPN (e.g., EVPN or IPVPN) Need citations for EVPN and IPVPN, and should probably expand EVPN. --- 3.4 Figure 4 is a bit messy. It should be referenced from the text. --- 4.1 Client service provisioning can follow the same approach as MPLS VRFs. A client VPN can establish the communication policy by specifying the Route Targets to be imported and exported. Alternatively, traditional Match and Action ACLs can identify the specific routes allowed or denied to or from the client VPN. "Route Target" needs to be set in the context of BGP. Please expand "ACL" and give a citation. --- 4.3 Section title needs to be in Title Case. --- 4.3 SD-WAN edge nodes must negotiate various cryptographic parameters to establish IPsec tunnels between them. Except, of course, those that don't use IPsec tunnels. --- 4.3 Each SD-WAN edge may have the default values assigned to them for the respective attributes. Maybe... Each SD-WAN edge may have default values configured for the IPsec parameters. --- 4.3 For a BGP-controlled SD-WAN, BGP UPDATE messages can propagate each node's IPsec-related attribute values for peers to choose the common values supported, traditionally done by IPsec IKEv2 [RFC7296]. The codicil to this paragraph seems misplaced. I guess you are saying that the choice is notified to the peer using IKEv2? Or something else? --- Section 5 reminded me of the name of this document. But page 14 is a long way in before we get to the point. So, I think that, possibly, the document title (and Abstract and Introduction) are a bit off target. Probably you have something like... An Introduction to SD-WAN Use Cases, Provisioning, and Forwarding --- 5.1 When multiple IPsec tunnels are established between two pairwise edge nodes I don't know what a "pairwise edge node" is. Do you mean: When multiple IPsec tunnels are established between two edge nodes --- 5.1 In a traditional IPsec VPN, separate routing protocols must run in parallel in each IPsec Tunnel Surely not separate routing protocols. Probably not even separate routing protocol instances. Perhaps, separate routing protocol adjacencies? --- 5.2 I think RFC 9012 calls it the "Tunnel Encapsulation attribute" not "Tunnel-Encap Path Attributes" --- 5.2 Alternatively, the C-PE2 can use two separate BGP UPDATE messages to reduce the size of the BGP UPDATE messages, especially for IPsec tunnels terminated at edge nodes or WAN ports, as IPsec SA tunnels have many attributes which can change at different frequencies than clients' routes updates, such as IPsec SA keys periodical changes. I think that is s/updates/update/ and s/periodical/periodic/ --- 5.2 - Suppose that a given packet "C" destined towards the client addresses attached to C-PE2 (e.g., prefix 192.0.2.4/30) can be carried by any IPsec tunnels terminated at C-PE2. It doesn't matter, but any reason why the packet is called "C"? s/tunnels/tunnel (since the packet can ultimately only be carried by one tunnel) --- 5.2 s/UPdATE U2:/UPDATE U2:/ s/as described in the [SECURE-EVPN]/as described in [SECURE-EVPN]/ s/Color-Extended-Community/Color Extended Community/ s/client routes policies/client routes' policies/ s/UPDATE messages propagation/UPDATE message propagation/ --- In 5.2, 5.3, and 5.4 you have lines such as: Encapsulation Extended Community: TYPE = IPsec But I think this should be: BGP Tunnel Encapsulation Attribute Tunnel Type = IPsec That said, https://www.iana.org/assignments/bgp-tunnel-encapsulation/ shows IPsec to be deprecated by RFC 9012. This seems to leave you with a bit of a problem! Skimming draft-ietf-idr-sdwan-edge-discovery, it looks like it handles IPsec by offering sub-TLVs to the SDWAN-Hybrid Tunnel Encapsulation Attribute. Perhaps you just need to rewrite around this? --- Looks like you have used [SD-WAN-EDGE-Discovery] in a normative way. That is, you can't do the things suggested in this document until that draft is an RFC. On the other hand, I did wonder what is the difference between 25 SDWAN-Hybrid and 20 Any-Encapsulation While draft-ietf-bess-bgp-multicast-controller that defines Any-Encapsulation seems to be about multicast, I think that the encapsulation type is defined to support multicast or unicast. It could be that the answer is how you handle IPsec. Depends on the answer to the previous point. --- 6. The procedures described in Section 6 of RFC8388 are applicable for the SD-WAN client traffic. This is true, but surely it only applies to Ethernet-based client services. What about IP services? --- 6.2.2 The figure is messed up The figure needs a title and a number The figure should be cited from the text --- 10.1 The reference text of BCP 195 is unusual --- 10.2 The reference for 802.3 seems incomplete --- -----Original Message----- From: IETF-Announce <ietf-announce-bounces@xxxxxxxx> On Behalf Of The IESG Sent: 01 February 2024 16:58 To: IETF-Announce <ietf-announce@xxxxxxxx> Cc: andrew-ietf@xxxxxxxxxxx; bess-chairs@xxxxxxxx; bess@xxxxxxxx; draft-ietf-bess-bgp-sdwan-usage@xxxxxxxx; matthew.bocci@xxxxxxxxx Subject: Last Call: <draft-ietf-bess-bgp-sdwan-usage-19.txt> (BGP Usage for SD-WAN Overlay Networks) to Informational RFC The IESG has received a request from the BGP Enabled ServiceS WG (bess) to consider the following document: - 'BGP Usage for SD-WAN Overlay Networks' <draft-ietf-bess-bgp-sdwan-usage-19.txt> as Informational RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the last-call@xxxxxxxx mailing lists by 2024-02-15. Exceptionally, comments may be sent to iesg@xxxxxxxx instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract The document discusses the usage and applicability of BGP as the control plane for multiple SD-WAN scenarios. The document aims to demonstrate how the BGP-based control plane is used for large- scale SD-WAN overlay networks with little manual intervention. SD-WAN edge nodes are commonly interconnected by multiple types of underlay networks owned and managed by different network providers. The file can be obtained via https://datatracker.ietf.org/doc/draft-ietf-bess-bgp-sdwan-usage/ No IPR declarations have been submitted directly on this I-D. _______________________________________________ IETF-Announce mailing list IETF-Announce@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf-announce -- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call