Ack. Agree. Dino > On Feb 12, 2015, at 6:56 AM, Black, David <david.black@xxxxxxx> wrote: > > [A] and [B] are handled in other messages. On UDP source port selection: > >>> Please state that a 5-tuple hash is used. ECMP/LAG is among the important >> >> Well there can be other ways to hash and that is detail not needed at this level IMO. We suggest a 5-tuple hash in RFC6830. >> >>> reasons why, but that doesn't need to be stated if you prefer not to. An >>> example of something that needs to be excluded is that using a random >>> number generator to set the source port would be wrong - I could suggest >>> citing draft-ietf-dart-dscp-rtp for related discussion (and lots more >>> details), but I don't think that's necessary. >> >> How about stating the source-port should not change for a flow or it causes an underlay router to >> resequence packets over lags? > > This is for ECMP in addition to LAG - if you go this route, please do cite the dart draft (informative reference) for its supporting discussion about transport protocol (mis)behavior in the face of within-flow resequencing. It would also be helpful to say that a 5-tuple hash is one way to achieve this (and see RFC 6830 for details). > > Thanks, > --David > >> -----Original Message----- >> From: Dino Farinacci [mailto:farinacci@xxxxxxxxx] >> Sent: Wednesday, February 11, 2015 11:40 PM >> To: Black, David >> Cc: Joel M. Halpern; Albert Cabellos; Damien Saucez; ops-dir@xxxxxxxx; >> ietf@xxxxxxxx; lisp@xxxxxxxx >> Subject: Re: [lisp] OPS-Dir review of draft-ietf-lisp-introduction-11 >> >>> Dino - thanks for the response. >>> >>> On the major issues, it looks like both [A] and [B] involve only the text >>> in this draft and nothing beyond, which is good news. I have a simple text >>> suggestion for [A], but it looks like [B] is going to require some careful >>> editing, as one of the primary causes is that the draft is sloppy in using >>> the same symbol "G" to represent both EID and RLOC multicast groups. >> >> Okay for [A] but not true for [B]. In RFC6831, a multicast address G is not in >> the mapping database because signaling is performed from ETR to ITR. What's in >> the mapping database is the EID S from the (S,G) the source sends from and to. >> >>> On the minor issues, I have text suggestions for three of the four, and >>> I'd like to temporarily defer further discussion the IPv6 UDP zero >>> checksum "tarpit" in favor of resolving everything else first. >> >> Sounds good David. >> >>> On the nits/editorial comments, all the suggestions in your email are fine >>> with me. FWIW, I regard that portion of a review as almost entirely >>> subject to the draft authors' discretion (and editorial taste). >>> >>>>>> [A] EID mobility vs. EID prefixes >>>> >>>> ... from the start of the LISP design (circa 2007), an prefix is what >> moves. >>>> And a specific EID is simply a /32 or /128 prefix. Here is a practical >>>> example: >>> >>> A statement that the mobility use cases need to employ /32 and /128 >> prefixes, >>> and not anything coarser should suffice. That should be added to Section 5. >> >> Well I think it is not true. Because EID-prefixes are moved but is outside of >> the VM-mobility use-case. >> >>> >>>>>> [B] LISP Multicast vs. EID/RLOC separate >>>>>> >>>>>> - 6. Multicast >>>>>> >>>>>> This is interesting, multicast addresses (G) look like they're an >> exception >>>> >>>> They are really not. >>> >>> My concern is that as I read the draft, it leaves me with the strong >> impression >>> that the same multicast addresses (G) are being used in both the overlay >>> (as EIDs) and the underlay (as RLOCs). From your response, I conclude that >> this >> >> Understand. We state in RFC6831 that it can map one-to-one or many-to-one. >> We'll make that more clear in the introduction document. >> >>> is not the case (and I have no argument with that). Rather, Section 6 needs >> to >>> bluntly state that multicast addresses are mapped between EID and RLOC space >> at >>> both ITRs and ETRs, so that the following inference is obvious from the text >>> (it's currently not obvious): >> >> Right, agree. >> >>> So it makes perfect sense to register multicast addresses to the mapping >>>> system as EIDs and they can map to RLOCs of sites that have joined the >> group. >>> >>> As part of this, I strongly recommend moving away from use of "G" to refer >> to >>> multicast groups in both the overlay and underlay. Careful use of G-EID >>> and G-RLOC would significantly improve clarity. >> >> Well we have not used G-EID in any documentation. And since we want to >> encourage the use of SSM in the underlay and how we signal in the overlay, we >> simply call the "eid" the 2-tuple (S,G). >> >>> --- >>> If the above are done for [A] and [B] in Sections 5 and 6, then the text for >> the >>> use cases in Section 7 should not need further attention. >>> --- >>> >>>>>> -- Minor Issues -- >>>>>> >>>>>> There seems to be an implicit assumption that the end host and its >>>>>> ITR (xTR) are in the same domain or Autonomous System. For incremental >>>> >>>> This is true when you call the domain a "LISP site". But if the site is >>>> unchanged and one uses PITRs, maybe even close to the site, like in a PE >>>> router, then the PITR is definitely in another AS. >>> >>> Looking at the text, it seems that "LISP site" and "domain" are the same >>> concept for this draft. That would be useful to state, IMHO but I'll leave >>> the decision on whether to do so to you and the draft authors. >>> >>> On rereading, my concerns seem to be triggered mostly by this sentence in >>> Section 3.2: >>> >>> The edge consists of LISP sites (e.g., an >>> Autonomous System) that use EID addresses. >>> >>> I think a small change to the last sentence in that paragraph would resolve >>> my concern without distracting from the narrative: >>> >>> OLD >>> EIDs do not >>> contain inter-domain topological information and because of this, >>> EIDs are usually routable at the edge (within LISP sites) or in the >>> non-LISP Internet. >>> NEW >>> EIDs do not >>> contain inter-domain topological information and because of this, >>> EIDs are usually routable at the edge (within LISP sites) or in the >>> non-LISP Internet; see Section 3.5 for discussion of LISP site >>> internetworking with non-LISP sites and domains in the Internet. >> >> Ack. >> >>> >>>>>> Despite multiple mentions of incremental deployment, I did not >>>>>> see a discussion of how that might be accomplished. >>>> >>>> There are PxTRs and NATs. And references to the LISP interworking RFC. >>> >>> Ok, can we just say so in Section 3.5? Adding the following sentence >>> to the end of the section would suffice: >>> >>> PITRs, PETRs and LISP-NAT support incremental deployment of LISP >>> by providing significant flexibility in location of the boundaries >>> between the LISP and non-LISP portions of the network, and making >>> it reasonable to change those boundaries over time. >> >> Yes. >> >>> - 3.3.1. LISP Encapsulation >>>>>> >>>>>> the source port is selected by >>>>>> the ITR and ignored on reception. >>>>>> >>>>>> Please mention multipathing (e.g., ECMP and LAG) as possible influences >>>>>> on how source ports are selected, as this imposes some limits on what an >>>>>> ITR can reasonably do. >>>> >>>> ECMP/LAG don't influence which source port is selected. It is a 5-tuple >> hash >>>> of the inner header that selects a source port that influences how an >> underlay >>>> router would load-split traffic. >>> >>> Please state that a 5-tuple hash is used. ECMP/LAG is among the important >> >> Well there can be other ways to hash and that is detail not needed at this >> level IMO. We suggest a 5-tuple hash in RFC6830. >> >>> reasons why, but that doesn't need to be stated if you prefer not to. An >>> example of something that needs to be excluded is that using a random >>> number generator to set the source port would be wrong - I could suggest >>> citing draft-ietf-dart-dscp-rtp for related discussion (and lots more >>> details), but I don't think that's necessary. >> >> How about stating the source-port should not change for a flow or it causes an >> underlay router to resequence packets over lags? >> >>> -- IPv6 zero UDP checksum >>> >>>> My head spins every time I hear about this subject. This subject has been >>>> talked about from 100s of people for a decade. We have CRC on links, we >> have >>>> apps that use TCP and UDP checksums. Nuf said. >>> >>> Understood - there's more than one set of scars on this one :-(. Let's come >> back >>> to this topic after we've resolved everything else, and please keep in mind >>> that I tagged this as a minor issue, not a major one (e.g., the above >> changes >>> for [A] and [B] are far more important, IMHO). >> >> Ack. Thanks again. >> >> Dino >> >>> >>> Thanks, >>> --David >>> >>>> -----Original Message----- >>>> From: Dino Farinacci [mailto:farinacci@xxxxxxxxx] >>>> Sent: Wednesday, February 11, 2015 2:19 PM >>>> To: Joel M. Halpern >>>> Cc: Black, David; Albert Cabellos; Damien Saucez; ops-dir@xxxxxxxx; >>>> ietf@xxxxxxxx; lisp@xxxxxxxx >>>> Subject: Re: [lisp] OPS-Dir review of draft-ietf-lisp-introduction-11 >>>> >>>>> I will leave most of these for the authors to comment on. >>>> >>>> See my comments inline. Thanks David for your detailed review and >> commentary. >>>> >>>>> With regard to your question about incremental deployment, that is the >>>> domain of the LISP Deployment document, and was deliberately only lightly >>>> covered here. I am not sure what we can do to address your comment without >>>> duplicating the entirety of that document. >>>> >>>> That is the risk we may have with many of your comments. We have a lot of >>>> detail in the already 9 published RFCs and this document really is to take >>>> all that detail and summarize as an easily understandable description of a >>>> cohesive design. >>>> >>>>> With regard to UDP Zero, this was approved by the IESG and published as an >>>> RFC. It is part of the way the protocol is defined. If there are specific >>>> changes you would like to see in the explanatory text, I am sure >>>> >>>> Definitely agreed. In fact we instigated UDP zero. And I continually talk >> to >>>> hardware engineers and they all believe we made the right decision. So hats >>>> off to the IETF for being practical. >>>> >>>>> we could include them. If you are looking for a change in the behavior, >>>> this document can not make changes to the LISP behavior. >>>> >>>> Yes, an important point. >>>> >>>>>> I found a couple of major issues that I hope arise from the >>>>>> summarization of LISP in this draft, as opposed to being problems in >>>>>> the actual LISP protocols. I also found a few minor issues, the most >>>>>> important of which is the need for additional security considerations >>>>>> discussion on misdelivery, with particular attention to VPNs. >>>> >>>> Thanks a ton. >>>> >>>>>> -- Major issues -- >>>>>> >>>>>> [A] EID mobility vs. EID prefixes >>>>>> >>>>>> - 5. Mobility >>>>>> >>>>>> I understand how this works when mapping is per-EID, but how does this >> work >>>>>> when the EID of the system that moves is part of an EID prefix, as >>>> discussed >>>>>> in Section 3.4.1? Even if the answer is a long version of "Don't do >> that!" >>>>>> it should be explained. >>>> >>>> No, from the start of the LISP design (circa 2007), an prefix is what >> moves. >>>> And a specific EID is simply a /32 or /128 prefix. Here is a practical >>>> example: >>>> >>>> You have a cluster of servers that communicate together for a particular >>>> application. They application cluster is running in a set of VMs. Those VMs >>>> are assigned EIDs from a common power-of-2 EID-prefix. Those VMs are >> currently >>>> running in a brick-and-mortar data center. Now there is a desire to move >> the >>>> VM cluster to a cloud provider. What is moved is the EID-prefix of the >>>> cluster. The mapping system is told that the EID-prefix is changing its >> RLOC- >>>> set from the brick-and-mortar xTRs to the cloud providers xTRs. >>>> >>>>>> >>>>>> - 7.4. LISP for Virtual Machine Mobility in Data Centers >>>>>> >>>>>> I don't understand how this works when EID prefixes are used, as each VM >>>>>> has its own EID or EIDs, hence the entire prefix range does not move when >>>>>> the VM moves. >>>>>> >>>>>> For OPS-Dir, this EID prefix issue [A] falls under A.1.1 in Appendix A >>>>>> of RFC 5706: Has deployment been discussed? and specifically under: >>>>>> >>>>>> * Is the proposed specification deployable? If not, how could >>>>>> it be improved? >>>>>> >>>>>> as EID prefixes appear to be undeployable for Mobility and VM Mobility >>>> usage. >>>> >>>> See above example. >>>> >>>>>> [B] LISP Multicast vs. EID/RLOC separate >>>>>> >>>>>> - 6. Multicast >>>>>> >>>>>> This is interesting, multicast addresses (G) look like they're an >> exception >>>> >>>> They are really not. Since multicast addresses *identify* a group of >>>> receivers, it is very much an EID and aheres to the definition of an EID. >>>> Multicast addresses never had topological signficance but the state >>>> representing a distribution tree does tell you were the members are (but >> the >>>> identity of the members are not know in multicast). >>>> >>>> So it makes perfect sense to register multicast addresses to the mapping >>>> system as EIDs and they can map to RLOCs of sites that have joined the >> group. >>>> See draft-farinacci-signal-free-multicast as just one example. RFC6831 and >>>> draft-farinacci-lisp-mr-signaling are other examples. >>>> >>>>>> to the EID/RLOC separation as the same destination IP multicast address >>>>>> is used for both purposes. This could use some more discussion, as it's >>>>>> unexpected based on the contents of the draft up to this point. >>>> >>>> I believe the level of detail we have in the introduction document is at >> the >>>> right level or we'll err on having way too many details crop in. >>>> >>>>>> - 7.2. LISP for IPv6 Co-existence >>>>>> >>>>>> LISP encapsulations allows to transport packets using EIDs from a >>>>>> given address family (e.g., IPv6) with packets from other address >>>>>> families (e.g., IPv4). >>>>>> >>>>>> How does that work for multicast traffic, where the destination address >>>>>> (G) has to be the same in both the inner and outer headers? Are ETRs >>>>>> and ITRs expected to map IPv6 multicast addresses to IPv4 and v.v.? >>>> >>>> The mapping system can map an (S-EID-ipv6, group-ipv6) 2-tuple to a RLOC >> set >>>> that looked like this (ipv4-multicast, ipv4-unicast) mean the ITR that >>>> receives the packet from S-EID-ipv6 would replicate the packet and >> multicast >>>> encapsulate to ipv4-multicast and unicast encapsualte to ipv4-unicast. >>>> >>>>>> - 7.3. LISP for Virtual Private Networks >>>>>> >>>>>> This also has multicast problems, as there is only one instance of each >>>>>> multicast address (G) in the underlay network. I think I can figure out >>>> how >>>> >>>> You can map from EID-G to RLOC-G one to one. But we have seen over the last >>>> decade in a half that with general multicast deployment that many-to-1 is >>>> desirable. Hence, now that we have a way to map with a network-based >> database, >>>> we can map multiple EID-Gs to a single (or multiple) RLOC-Gs. >>>> >>>>>> to make multicast work for this use case, but it's not immediately >> obvious, >>>>>> and the result when the same underlay multicast address is used by more >>>>>> than one VPN could well deliver some traffic to ETRs that have to discard >>>> >>>> This is a necessary evil when the underlay is state challenged. But it is a >>>> state/bandwidth tradeoff. We have found with MVPN deployment that the >> network >>>> admin configures the underly or simply unicasts. >>>> >>>>>> it because the Instance ID is wrong (and that excessive delivery is a >>>>>> security consideration, see minor issue on Section 8 below). I think an >>>>>> explanation is in order. >>>> >>>> There are just too many combinations to make a high-level description >> simple >>>> to understand. The current introduction text does a find job providing >>>> references for someone to go off and get the details. >>>> >>>>>> -- Minor Issues -- >>>>>> >>>>>> There seems to be an implicit assumption that the end host and its >>>>>> ITR (xTR) are in the same domain or Autonomous System. For incremental >>>> >>>> This is true when you call the domain a "LISP site". But if the site is >>>> unchanged and one uses PITRs, maybe even close to the site, like in a PE >>>> router, then the PITR is definitely in another AS. But note I said PITR and >>>> not ITR. The reason being is because an ITR is configured with database- >>>> mapping prefixes that is uses to encapsulate packets from such addresses. >>>> Versus the PITR being an ITR with *no database-mappings* providing a much >> more >>>> larger/or more aggregtable service. >>>> >>>>>> deployment, I don't think that's always the case, but I think that only >>>>>> has editorial impact on this document, as I don't think any of the >>>>>> fundamental LISP mechanisms are affected. The authors should look for >>>>>> use of "domain" and "Autonomous System" and ensure that the text is >>>>>> generalized to the case where the end host and ITR are more widely >>>>>> separated. >>>> >>>> We are overloaded with terms that create topological or organization >> boundary. >>>> Hence why we created "LISP site" which is also the same as a "LISP VPN >> site". >>>> Where a "LISP site" that has multiple tennants would be multiple "LISP VPN >>>> sites". >>>> >>>>>> Despite multiple mentions of incremental deployment, I did not >>>>>> see a discussion of how that might be accomplished. There is some >>>> >>>> There are PxTRs and NATs. And references to the LISP interworking RFC. >>>> >>>>>> useful content in Section 3.5, but that's at best an incomplete >>>>>> explanation. This is an OPS-Dir review concern - it falls under >>>>>> A.1.3 in Appendix A of RFC 5706: Has the migration path been discussed? >>>>>> >>>>>> - 3.3.1. LISP Encapsulation >>>>>> >>>>>> the source port is selected by >>>>>> the ITR and ignored on reception. >>>>>> >>>>>> Please mention multipathing (e.g., ECMP and LAG) as possible influences >>>>>> on how source ports are selected, as this imposes some limits on what an >>>>>> ITR can reasonably do. >>>> >>>> ECMP/LAG don't influence which source port is selected. It is a 5-tuple >> hash >>>> of the inner header that selects a source port that influences how an >> underlay >>>> router would load-split traffic. >>>> >>>>>> For OPS-Dir, this multipathing concern falls under A.1.4 in Appendix A of >>>>>> RFC 5706: Have the Requirements on other protocols and functional >>>>>> components been discussed? >>>>>> >>>>>> This decision was made because the >>>>>> typical transport protocols used by the applications already include >>>>>> a checksum, by neglecting the additional UDP encapsulation checksum >>>>>> xTRs can forward packets more efficiently. >>>>>> >>>>>> Groan! I have an exquisite set of scars on UDP zero checksums for IPv6 >>>>>> from working on the MPLS in UDP draft, so I may be overly sensitive to >>>>>> this concern. The downside of this efficiency is that there is no >>>>>> checksum coverage of the IPv6 header when zero UDP checksums are used. >>>>>> That should at least be mentioned here, with a summary of why that's ok >>>>>> - the detailed justification for why that's ok can be left to other >>>>>> documents. >>>> >>>> My head spins every time I hear about this subject. This subject has been >>>> talked about from 100s of people for a decade. We have CRC on links, we >> have >>>> apps that use TCP and UDP checksums. Nuf said. >>>> >>>>>> >>>>>> -- Nits/Editorial Comments -- >>>>>> >>>>>> - Top of p.4: >>>>>> >>>>>> The initial motivation in the LISP effort is to be find in the >>>>>> >>>>>> "find" -> "found" >>>>>> >>>>>> - Section 3.1, first bullet item >>>> >>>> We will certainly fixe these. Thanks. >>>> >>>>>> >>>>>> Devices are assigned with relatively opaque identity >>>>>> meaningful addresses that are independent of their topological >>>>>> location. >>>>>> >>>>>> I don't understand "relatively opaque identity meaningful" and >>>>>> suggest rewriting the sentence. In particular - opaque to what? >>>>>> meaningful to what in what manner? >>>> >>>> Well beacuse it is as accurate as it can be. If automobiles are going to be >>>> assigned EIDs from a VIN number allocation from a manufacture, the address >> is >>>> relatively opaque. If a VM in a data-center is going to be assigned an EID >>>> from the set of prefixes already being used and allocated to that data- >> center, >>>> then there is a good chance that address is in a power-of-2 block that is >>>> summarizable in the IGP. >>>> >>>>>> >>>>>> - Section 3.2, second paragraph >>>>>> >>>>>> Judging from the figure, xTRs are the common case, with single- >>>>>> function ITRs and ETRs being rare. It might be good to say that >>>>>> and discuss when ITRs and ETRs that are not xTRs are appropriate >>>>>> to use. >>>> >>>> When you want egress path selection to happen further out in the toplogical >>>> from the source location, then you put an ITR-only system there. Where >> ingress >>>> to the same source (destination in this direction), the ETR can be closer >> to >>>> the destination. >>>> >>>>>> >>>>>> - 3rd paragraph on p.7: >>>>>> >>>>>> >>>>>> Finally, the LISP architecture emphasizes a cost effective >>>>>> incremental deployment. >>>>>> >>>>>> I'd delete "cost effective" here and look for other occurrences >>>>>> of "cost" as candidates for deletion. This is supposed to be >>>>>> a technical document, so discussion of costs is a bit off-target. >>>> >>>> Fair enough. >>>> >>>>>> - First item after Figure 2: >>>>>> >>>>>> 1. HostA retrieves the EID_B of HostB, typically querying the DNS >>>>>> and obtaining and A or AAAA record. >>>>>> >>>>>> "and A" -> "an A" (spelling checkers don't catch everything). >>>> >>>> Already noted and will be fixed. >>>> >>>>>> >>>>>> - 3.3.1. LISP Encapsulation >>>>>> >>>>>> On the other hand, Recursive >>>>>> tunnels are nested tunnels and are implemented by using multiple LISP >>>>>> encapsulations on a packet. >>>>>> >>>>>> The above sentence seems out of place in the middle of a paragraph about >>>>>> Re-encapsulating tunnels and routers - I suggest moving it down into its >>>>>> own paragraph and perhaps adding a sentence about where/how Recursive >>>>>> tunnels may be useful. >>>> >>>> Good suggestion and makes sense. >>>> >>>>>> - 3.3.2. LISP Forwarding State >>>>>> >>>>>> In the LISP architecture, ITRs keep just enough information to route >>>>>> traffic flowing through it. >>>>>> >>>>>> "it." -> "them." >>>>>> >>>>>> Meaning that, ITRs retrieve from the >>>>>> LISP Mapping System mappings between EID prefixes and RLOCs that are >>>>>> used to encapsulate packets. >>>>>> >>>>>> This is the first use of the notion of EID prefixes. That concept should >>>>>> be explained before it is used, although a forward reference to section >>>>>> 3.4.1 may suffice. It might be better to rewrite this paragraph in terms >>>>>> of EIDs and leave the notion of EID prefixes to section 3.4.1. >>>> >>>> Hmm, I'll let Albert and Damien decide if we should state "EID-prefixes" >>>> everywhere instead of just "EID". >>>> >>>>>> >>>>>> - 4.4. MTU Handling >>>>>> >>>>>> Additionally, LISP also recommends inferring reachability of locators >>>>>> by using information provided by the underlay, in particular: >>>>>> >>>>>> It'd be useful to add a sentence or two about how LISP and the techniques >>>>>> in this section interact with host use of PMTUD and PLPMTUD. >>>> >>>> This is a lot of detail because in RFC6830 we have 3 positions or options >> on >>>> the subject. And we did provide a reference to RFC6830 for this topic. >>>> >>>>>> - Next to last paragraph on p.17: >>>>>> >>>>>> Additionally, LISP also recommends inferring reachability of locators >>>>>> by using information provided by the underlay, in particular: >>>>>> >>>>>> This looks like it's a paragraph early and needs to be moved down to >>>>>> after the paragraph that follows it. >>>> >>>> Agree. >>>> >>>>>> idnits 2.13.01 didn't find any nits. >>>>>> >>>>>> Thanks, >>>>>> --David >>>> >>>> Thanks again David. >>>> >>>> Dino >>> >