> I think we are in agreement and I will edit the document to reflect > your comments shortly. I think Adrian is AD on this and Loa the > shepard, so I'll wait for a go-ahead from either of them before > submiting the edits. Yes, we're in agreement, and indeed, "sandwich" is a term I coined in my review - it seemed descriptive of an MPLS-TP label in a stack between two ELI/EL pairs. Thanks, --David > -----Original Message----- > From: Curtis Villamizar [mailto:curtis@xxxxxxxxxxxxxx] > Sent: Tuesday, January 21, 2014 8:08 PM > To: Black, David > Cc: curtis@xxxxxxxxxxxxxx; ops-dir@xxxxxxxx; ietf@xxxxxxxx; draft-ietf-mpls- > multipath-use.all@xxxxxxxxxxxxxx > Subject: Re: OPS-Dir review of draft-ietf-mpls-multipath-use-03 > > > In message <8D3D17ACE214DC429325B2B98F3AE712026F047CC2@xxxxxxxxxxxxxxxxxx> > "Black, David" writes: > > > > Curtis, > > > > > Thanks for the review and for pointing out lack of clarity due to > > > underlying assumptions about the use of RFC 4206 (and perhaps use of > > > RFC 4201 as well for large MPLS LSP over MPLS-TP). > > > > > > Sometimes it is too easy to forget that not all readers are going to > > > be as steeped in MPLS "as deployed today" details as a relatively > > > small number of long time MPLS participants are. > > > > > > Adding a few paragraphs to the introduction and a few concluding > > > paragraphs may significantly improve clarity for people who have not > > > been following MPLS closely for a decade or more without requiring > > > them to read the references and figure out usage details themselves. > > > > > > Please let me know how you feel about the suggested additions to the > > > document. > > > > Most of them look good - details below. Many thanks for the prompt and > > comprehensive response. I have a few more suggestions in what follows > > - anything that I don't comment on below is fine with me. > > Thank you for the quick turnaround on this as well. > > > -- Introduction > > > > > There is an underlying assumption in the draft that the reader > > > understands that MPLS-in-MPLS exists today, most commonly but not > > > exclusively in the form of RFC 4206 hierarchy. RFC 4206 provides a > > > means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some > > > level of Packet Switch Capable (PSC) hierarchy. It is also possible > > > to make use of the management plane, using CLI or NETCONF or the MPLS > > > cross connect MIB to program a multiple label MPLS push operation. > > > > > > If a very experienced reviewer could miss that, then the introduction > > > could be improved by better stating assumptions. It is also the case > > > that many participants in MPLS, particularly those primarily > > > interested in MPLS-TP, began participation long after RFC 4206 was > > > written and would benefit by better stated assumptions in the intro. > > > > > > I suggest that the following paragraphs be added to the introduction. > > > > The suggested new paragraphs will definitely improve the introduction. > > > > -- [A] Summary text in Introduction appears to be wrong. > > > > > > Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above > > > > exception appears to concern the reverse, and Section 4 on carrying MPLS > > > > over MPLS-TP allows for MPLS LSPs that exceed component link capacity. > > > > Was the exception intended to be for an MPLS-TP LSP that exceeds the > > > > capacity of any single component link? > > > > [... snip ...] > > > > > To answer your question directly: the exception is a very large MPLS > > > LSP can't be carried by a single MPLS-TP LSP. The solution is to use > > > multiple MPLS-TP LSP. The MPLS-TP LSR need not be not "aware" that > > > this is occurring. > > > > It looks like my confusion stemmed from the original text only > > discussing only what won't work and referencing a section that > > concerns a different encapsulation order than the one that causes > > the problem. Including the "MAY NOT" -> "SHOULD NOT" change, I suggest: > > > > OLD > > When an MPLS LSP > > exceeds the capacity of any single component link it MAY be carried > > by a network using multipath techniques, but MAY NOT be carried by a > > single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation > > imposed by MPLS-TP Operations, Administration, and Maintenance (OAM) > > fate sharing constraints and MPLS-TP LM OAM packet ordering > > constraints (see Section 3.1). > > NEW > > When an MPLS LSP > > exceeds the capacity of any single component link it MAY be carried > > by a network using multipath techniques, but SHOULD NOT be carried by a > > single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation > > imposed by MPLS-TP Operations, Administration, and Maintenance (OAM) > > fate sharing constraints and MPLS-TP LM OAM packet ordering > > constraints (see Section 3.1). Instead, multiple MPLS-TP LSPs SHOULD > > be used to carry a large MPLS LSP (see Section 4). > > OK. Good addition. > > > -- [B] Section 3.2 needs a summary. > > > > The proposed summary looks good, and helps with [C], but the phrase > > "access and edge" needs qualification in the following sentence: > > > > However access and edge is the least likely to support > > RFC 6790 in the near term. > > > > "access and edge" what? e.g., areas of the network? LSRs? > > Oh. Good point. > > Core, edge, aggregation, and access are terms commonly used for the > tiers in provider networks. Core is central, high traffic. Access is > furthest out with lower traffic (think of subscriber connections for > consumer Internet). > > Perhaps I can just change the sentence to: > > However equipment furthest from a provider's network core is the > least likely to support RFC 6790 in the near term. > > I think we all know "core" at this point so furthest from core might > be easier to understand. > > > -- [C] Entropy label "sandwich" seems peculiar and at least needs an > > explanation. > > That was your term. I didn't propose adding Entropy label "sandwich" > to the text. I think this is just an excerpt from your prior email as > a marker in following up on it. Please correct me if I am wrong. > > > > The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following > > > paragraph. > > > > > > Alternately, the need for requirement MP#1 can be eliminated if > > > every MPLS-TP LSP created by an MPLS-TP ingress makes use of an > > > Entropy Label Indicator (ELI) and Entropy Label (EL) below the > > > MPLS-TP label [RFC6790]. This would require that all MPLS-TP LSR > > > in a deployment support Entropy Label, which may render it > > > impractical in many deployments. > > > > > > Please not the "Alternately" at the beginning and "impractical in many > > > deployments" at the end. > > That should have said "Please note". > > > > The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs > > > above that but assumes knowledge of RFC 6790. I think the following > > > change within this paragraph would clear things up: > > > > That change is helpful, but my "sandwich" concern was actually raised > > by text two paragraphs further down: > > > > MPLS-TP LSPs can be carried as client LSPs within an MPLS server LSP > > if an Entropy Label Indicator (ELI) and Entropy Label (EL) is pushed > > below the server layer LSP label(s) in the label stack, just above > > the MPLS-TP LSP label entry [RFC6790]. > > > > Read after the "Alternately" paragraph, this text appeared to suggest > > a "sandwich" that would have an ELI/EL pair both above and below the > > MPLS-TP label. That was clearly not intended, so I suggest inserting > > "When Entropy Label Indicators (ELIs) and Entropy Labels (ELs) are not > > applied by MPLS-TP ingresses," before the start of the first sentence > > above, so that the result is: > > > > When Entropy Label Indicator (ELIs) and Entropy Labels (ELs) are > > not applied by MPLS-TP ingresses, MPLS-TP LSPs can be carried as > > client LSPs within an MPLS server LSP if an ELI and EL are pushed > > below the server layer LSP label(s) in the label stack, just above > > the MPLS-TP LSP label entry [RFC6790]. > > OK. I will make that replacement. > > > -- [D] Discussion of multipath impacts on MPLS OAM > > > > > > Section 3.2 discusses OAM, and contains this important > > > > requirement for OAM traffic: > > > > > > > > An Entropy Label must be used to insure that all of the > > > > MPLS-TP payload and OAM traffic are carried on the same component, > > > > except during very infrequent transitions due to load balancing. > > > > > > > > [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in > > > > Section 4; that should be added. > > > > > > There is no impact on MPLS OAM traffic. There are three cases to > > > consider. > > > > > > 1. Where MPLS is carried over a single MPLS-TP all traffic flows on > > > one link. MPLS OAM is unaffected and need not use multipath > > > support in LSP Ping. > > > > > > 2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP > > > LSP is carried over one link thanks to the fixed EL value. > > > MPLS-TP OAM is unaffected. > > > > > > 3. Where MPLS is carried over MPLS (an existing case) or over > > > multiple MPLS-TP, the multipath support in LSP Ping (as > > > imperfect as it may already be) is used and LSP Ping (RFC 4379) > > > operation is unaffected. > > > > > > If needed something can be added to state the above (maybe with less > > > informal wording). What is obvious to some people may not be obvious > > > to everyone. Briefly stating the almost obvious is usually not a bad > > > thing. > > > > Points 1-3 are useful explanation as complement to the Entropy label > > requirement to correctly handle OAM traffic for the other encapsulation. > > > > I like your suggestion to add this text as part of a new concluding > > Section 5, and the rest of the proposed conclusion section text looks > > good. > > > > Thanks, > > --David > > Thanks again. > > I think we are in agreement and I will edit the document to reflect > your comments shortly. I think Adrian is AD on this and Loa the > shepard, so I'll wait for a go-ahead from either of them before > submiting the edits. > > Curtis > > > > > -----Original Message----- > > > From: Curtis Villamizar [mailto:curtis@xxxxxxxxxxxxxx] > > > Sent: Tuesday, January 21, 2014 2:23 PM > > > To: Black, David > > > Cc: ops-dir@xxxxxxxx; ietf@xxxxxxxx; draft-ietf-mpls-multipath- > > > use.all@xxxxxxxxxxxxxx > > > Subject: Re: OPS-Dir review of draft-ietf-mpls-multipath-use-03 > > > > > > > > > In message <8D3D17ACE214DC429325B2B98F3AE712026F047819@xxxxxxxxxxxxxxxxxx> > > > "Black, David" writes: > > > > > > > > I have reviewed this document as part of the operations directorate's > > > > ongoing effort to review all IETF documents being processed by the > > > > IESG. These comments were written primarily for the benefit of the > > > > operations area directors. Document editors and WG chairs should treat > > > > these comments just like any other last call comments. > > > > > > > > Summary: > > > > This draft is on the right track but has open issues, described in the > > > review. > > > > > > > > I apologize for this review showing up a couple of days after the end of > > > > IETF Last Call. > > > > > > > > This draft discusses considerations for what are effectively MPLS-in- > MPLS > > > > tunnels for multipathing - an LSP carries another LSP, with the inner > > > > (client) LSPs being distributed across multiple outer (server) LSPs that > > > > use different paths. This draft is motivated by considerations specific > > > > to MPLS-TP that complicate multipathing when one of the two MPLS > instances > > > > is MPLS-TP. > > > > > > Skipping to the discussion quite a ways down, there seems to be some > > > confusion about what is being proposed here. While MPLS-in-MPLS is > > > being used, no new form of MPLS-in-MPLS is being proposed. > > > > > > There is an underlying assumption in the draft that the reader > > > understands that MPLS-in-MPLS exists today, most commonly but not > > > exclusively in the form of RFC 4206 hierarchy. RFC 4206 provides a > > > means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some > > > level of Packet Switch Capable (PSC) hierarchy. It is also possible > > > to make use of the management plane, using CLI or NETCONF or the MPLS > > > cross connect MIB to program a multiple label MPLS push operation. > > > > > > If a very experienced reviewer could miss that, then the introduction > > > could be improved by better stating assumptions. It is also the case > > > that many participants in MPLS, particularly those primarily > > > interested in MPLS-TP, began participation long after RFC 4206 was > > > written and would benefit by better stated assumptions in the intro. > > > > > > I suggest that the following paragraphs be added to the introduction. > > > > > > MPLS LSPs today are able to function as a server layer and carry > > > client MPLS LSPs. When control plane signaling is used, forwarding > > > adjacency (FA) advertisements are used to inform the set of LSR of > > > Packet Switching Capable (PSC) LSP within the MPLS topology > > > [RFC4206]. Client MPLS LSP at a higher layer (lower PSC number) > > > may signal their intention to use PSC LSP as hops in the RSVP-TE > > > Explicit Route Object (ERO). LSR with no explicit support for RFC > > > 4206 see the PSC LSP as ordinary links and therefore use them. > > > > > > An MPLS LSP that has been set up using RSVP-TE appears to its > > > ingress LSR as a viable IP next hop to a distant LSR. If LDP is > > > used and bidirectional RSVP-TE LSP connectivity is available, then > > > LDP signaling can be set up among the RSVP-TE LSP endpoints and LDP > > > can make use of the RSVP-TE LSP as an LDP hop. This is another > > > form of existing MPLS-in-MPLS use. > > > > > > MPLS LSPs may also make use of hierarchy that is configured through > > > the management plane rather than signaled using RSVP-TE. > > > > > > These existing forms of MPLS-in-MPLS may traverse multipath hops > > > such as Ethernet LAG [IEEE802.1AX] or MPLS Link Bundling [RFC4201]. > > > MPLS-TP brings with it a new set of requirements not considered in > > > past deployments of the various forms of MPLS-in-MPLS where > > > multipath was in use. This document merely discusses use of > > > existing forwarding and protocol mechanisms that can support the > > > case where either the client layer LSPs or the server layer LSPs > > > are MPLS-TP and where multipath is used. > > > > > > If there are no objections I will add these paragraphs. > > > > > > Now on to the specific comments. > > > > > > > I consider the following to be open issues that need attention (each > > > > letter is used to tag the issue in the text that follows - [B] occurs > > > > multiple times, as there are multiple aspect to that issue): > > > > > > > > [A] Summary text in Introduction appears to be wrong. > > > > [B] Section 3.2 needs a summary. > > > > [C] Entropy label "sandwich" seems peculiar and at least needs an > explanation. > > > > [D] Discussion of multipath impacts on MPLS OAM should be added to > Section > > > 4. > > > > > > > > --- Comments on draft text > > > > > > > > [A] Section 1 (Introduction) 2nd paragraph: > > > > > > > > RFC 5654 requirement 33 requires the capability to carry a client > > > > MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer [RFC5654]. > > > > This is possible in all cases with one exception. When an MPLS LSP > > > > exceeds the capacity of any single component link it MAY be carried > > > > by a network using multipath techniques, but MAY NOT be carried by a > > > > single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation > > > > imposed by MPLS-TP Operations, Administration, and Maintenance (OAM) > > > > fate sharing constraints and MPLS-TP LM OAM packet ordering > > > > constraints (see Section 3.1). > > > > > > > > Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above > > > > exception appears to concern the reverse, and Section 4 on carrying MPLS > > > > over MPLS-TP allows for MPLS LSPs that exceed component link capacity. > > > > Was the exception intended to be for an MPLS-TP LSP that exceeds the > > > > capacity of any single component link? > > > > > > There is no problem today with MPLS LSP carried by MPLS-TP LSP except > > > the existing case where MPLS LSP capacity is too big. For example, a > > > 500 Gb/s LSP is possible today carried over N x 100 Gb/s or N x 10 > > > Gb/s multipath links but could not be carried over a single MPLS-TP > > > LSP which would be limited to a single 100 Gb/s or 10 Gb/s link. So > > > the exception is where the MPLS LSP is too big to fit in any possible > > > single MPLS-TP LSP. > > > > > > To answer your question directly: the exception is a very large MPLS > > > LSP can't be carried by a single MPLS-TP LSP. The solution is to use > > > multiple MPLS-TP LSP. The MPLS-TP LSR need not be not "aware" that > > > this is occurring. > > > > > > > Also, "MAY NOT" is not an RFC 2119 term (for good reason - it's > equivalent > > > > to "MAY OR MAY NOT"). I think "cannot" was intended here, but "MUST > NOT" > > > > is another possibility. > > > > > > Good point. I suggest we replace the MAY NOT with SHOULD NOT. Many > > > people would insist that it should be MUST NOT, but in reality > > > networks SHOULD NOT be congested but sometimes they are. > > > > > > > [B] Section 3.2 contains a lot of useful details, but it could really > use > > > > a summary at the end on the options and recommendations for meeting the > > > > four requirements (this summary could be a new Section 3.3). It's a bit > > > > difficult to discern what's going on, as I had to read this section more > > > > than once in order to discern the following: > > > > > > OK. A Summary paragraph can be added. See below. Perhaps conclusion > > > section can be added. See far below. > > > > > > > [C] In Section 3.2, use of an MPLS entropy label to meet requirement > MP#1 > > > > requires that it be "below the MPLS-TP label" whereas, to meet > requirement > > > > MP#2, the MPLS entropy label needs to be "just above the MPLS-TP LSP > label". > > > > This combination appears to suggest "sandwiching" the MPLS-TP label > between > > > > two entropy labels that effectively carry the same information (albeit, > > > > applied and removed at different places in the network) - was that > intended? > > > > This is important, as there seems to be no reasonable alternative > > > > to the entropy label for meeting requirement MP#2, and the draft's > > > > alternative to the entropy label for meeting requirement MP#1 is "some > form > > > > of configuration" - the entropy label would seem to be preferred to that > > > > based on my reading of this draft and the comment in RFC 5706 section > 2.2 > > > > that "Anything that can be configured can be misconfigured." > > > > > > The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following > > > paragraph. > > > > > > Alternately, the need for requirement MP#1 can be eliminated if > > > every MPLS-TP LSP created by an MPLS-TP ingress makes use of an > > > Entropy Label Indicator (ELI) and Entropy Label (EL) below the > > > MPLS-TP label [RFC6790]. This would require that all MPLS-TP LSR > > > in a deployment support Entropy Label, which may render it > > > impractical in many deployments. > > > > > > Please not the "Alternately" at the beginning and "impractical in many > > > deployments" at the end. > > > > > > The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs > > > above that but assumes knowledge of RFC 6790. I think the following > > > change within this paragraph would clear things up: > > > > > > OLD > > > > > > MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI) > > > and Entropy Label (EL) are added to the label stack [RFC6790]. For > > > those client LSP that are MPLS-TP LSP, a single EL value must be > > > chosen. For those client LSP that are MPLS LSP, per packet entropy > > > below the top label must, for practical reasons, be used to > > > determine the entropy label value. > > > > > > NEW > > > > > > MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI) > > > and Entropy Label (EL) are added to the label stack by the midpoint > > > LSR for the client MPLS-TP LSP, at the ingress of the MPLS LSP > > > [RFC6790]. For those client LSP that are MPLS-TP LSP, a single EL > > > value must be chosen. For those client LSP that are MPLS LSP, per > > > packet entropy below the top label must, for practical reasons, be > > > used to determine the entropy label value. The resulting label > > > stack contains the server MPLS LSP label, ELI, EL and the client > > > LSP label. > > > > > > The above rewording is simply more explicit about the label stack > > > order and which LSR puts the ELI and EL in the label stack. > > > > > > > [B] A summary of requirements for implementation and deployment at the > end > > > > of Section 3.2 would have made this much more obvious. That should be > > > > provided in addition to addressing the above question about duplicate > use > > > > of the MPLS entropy label. > > > > > > The following summary can be added to the end of Section 3.2: > > > > > > Two methods of adding an Entropy Label are described above. The > > > MPLS-TP ingress must have a means to determine which links can > > > support MPLS-TP in selecting a path (MP#4). Administrative > > > attributes can satisfy that requirement. If the MPLS-TP LSR is > > > capable of adding ELI/EL to the label stack, this method is > > > preferred. However access and edge is the least likely to support > > > RFC 6790 in the near term. For portions of the topology where an > > > MPLS-TP is carried within a server layer MPLS LSP the ingress of > > > the server layer MPLS LSP can add ELI/EL using a fixed EL value per > > > client LSP, except those known not to require MPLS-TP treatment. > > > There are numerous ways to determine which client LSP are MPLS-TP > > > LSP and which are not. While this determination is out of scope > > > and will vary among deployments, configuration or the presence of > > > specific attribute affinities in RSVP-TE signaling are among the > > > likely means to do so. > > > > > > If there are no objections I will add the above paragraph. > > > > > > > --- Relevant Q&A based on RFC 5706 Appendix A > > > > > > > > I'm omitting management concerns, as both MPLS and MPLS-TP have > extensive > > > > management infrastructure, and this draft's notion of stacking LSPs so > > > > that one instance of MPLS (or MPLS-TP) can be a server to (i.e., carry) > > > > another is not novel to this draft. > > > > > > > > A.1.1. Has deployment been discussed? > > > > > > > > [B] Yes, although Section 3.2 is weak on deployment requirements (some > of > > > > which are implementation requirements) and could use a summary. > > > > > > The use of RFC 4206 is assumed throughout the discussion of signaling > > > in section 3.2. This assumption is made more clear by suggested > > > additions to the introduction. > > > > > > > A.1.2. Has installation and initial setup been discussed? > > > > > > > > No, although because this draft reuses existing functionality in > different > > > > configurations, this really reduces to the A.1.9 configuration question > > > > below. In practice, quite a bit of operator knowledge and insight is > > > > required to get initial setup right, e.g., as traffic engineering LSPs > to > > > > avoid congestion problems (needed to meet requirement MP#3) requires > > > > significant knowledge of network structure and expected traffic flows. > > > > This should be obvious to anyone who works w/MPLS, but it's probably > > > > worth stating for completeness. > > > > > > FA advertisements defined in RFC 4206 make it possible for ingress to > > > signal viable paths. Either the use of Ethernet Link Aggregation or > > > MPLS Link bundling using the all-ones component link (RFC 4201) allows > > > the total capacity of a multipath to be advertised. Meeting the > > > requirement to balance traffic over the component links in a multipath > > > (MP#3) is then a local matter and requires no further signaling > > > extensions. > > > > > > > A.1.6. Have suggestions for verifying correct operation been discussed? > > > > > > > > Yes and no. Section 3.2 discusses OAM, and contains this important > > > > requirement for OAM traffic: > > > > > > > > An Entropy Label must be used to insure that all of the > > > > MPLS-TP payload and OAM traffic are carried on the same component, > > > > except during very infrequent transitions due to load balancing. > > > > > > > > [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in > > > > Section 4; that should be added. > > > > > > There is no impact on MPLS OAM traffic. There are three cases to > > > consider. > > > > > > 1. Where MPLS is carried over a single MPLS-TP all traffic flows on > > > one link. MPLS OAM is unaffected and need not use multipath > > > support in LSP Ping. > > > > > > 2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP > > > LSP is carried over one link thanks to the fixed EL value. > > > MPLS-TP OAM is unaffected. > > > > > > 3. Where MPLS is carried over MPLS (an existing case) or over > > > multiple MPLS-TP, the multipath support in LSP Ping (as > > > imperfect as it may already be) is used and LSP Ping (RFC 4379) > > > operation is unaffected. > > > > > > If needed something can be added to state the above (maybe with less > > > informal wording). What is obvious to some people may not be obvious > > > to everyone. Briefly stating the almost obvious is usually not a bad > > > thing. Please tell me if a small subsection on "Imopact on MPLS and > > > MPLS-TP OAM" is needed. > > > > > > Note that the limitations of LSP Ping are currently being discussed in > > > the MPLS WG in the context of extensions to LSP Ping to better support > > > Entropy Label. A few limitations not related to EL may get fixed in > > > the process. > > > > > > > A.1.9. Is configuration discussed? > > > > > > > > [B] Not much. Configuration is required to meet requirements MP#3 and > MP#4, > > > > and is one of the options for MP#1. A summary of what has to be > configured > > > > as part of the summary at the end of Section 3.2 would be a good idea. > > > > > > Perhaps a worked example appendix could be added but I don't think > > > this is common and may not be necessary with some of the > > > clarifications suggested so far. > > > > > > > A.2.* [skipped, see above] > > > > > > > > A.3 Documentation > > > > > > > > There's quite a bit of operational material contained in this draft; a > > > > separate section on operations and management considerations would not > > > > be appropriate. > > > > > > > > There is a useful implementation status section which appears to imply > > > > that MPLS-TP over MPLS is not currently deployable due to absence of > > > > entropy label support, and because deployed MPLS equipment does not > > > > generally support multipathing. Nonetheless, data center experience > > > > indicates significant value and widespread use of multipathing based > > > > on link aggregation and ECMP; corresponding benefits can be expected > > > > for use of multipathing in the Internet beyond data centers. > > > > > > If you would like I can add a conclusion section as Section 5 with > > > these paragraphs: > > > > > > MPLS equipment deployed in the core currently supports multipath. > > > For large service providers, core LSR must support some form of > > > multipath to be deployable. Deployed MPLS access and edge > > > equipment is often oblivious to the use of multipath in the core. > > > It is expected that at least first generation MPLS-TP equipment > > > will be oblivious to the use of multipath in the core. This first > > > generation MPLS-TP equipment is deployable in a core using > > > multipath, with no adverse impact to RSVP-TE signaling, if the edge > > > equipment can support administrative attributes (RFC 3209) and the > > > core equipment can support ELI/EL and put a per-LSP fixed EL value > > > on any LSP that indicates a particular attribute affinity or can > > > identify client MPLS-TP LSP through some other means. > > > > > > There are no issues carrying MPLS over MPLS-TP, except when the > > > MPLS LSP is too big to be carried by a single MPLS-TP LSP. Most > > > MPLS core equipment and some edge equipment can configure an MPLS > > > Link Bundle [RFC4201] over multiple component links where the > > > component links are themselves MPLS LSP. This existing capability > > > can be used to carry large MPLS LSP and overcome the limited > > > capacity of any single server layer MPLS-TP LSP. > > > > > > If this conclusion section is added, then the three bullets above > > > regarding impact on OAM can be put there. The three bullets can be > > > added as follows: > > > > > > MPLS OAM and MPLS-TP OAM are unaffected in the following cases > > > proposed in this document: > > > > > > 1. Where MPLS is carried over a single MPLS-TP all traffic flows > > > on one link. MPLS OAM is unaffected and need not use > > > multipath support in LSP Ping. > > > > > > 2. Where MPLS-TP is carried over MPLS, all traffic for that > > > MPLS-TP LSP is carried over one link thanks to the fixed EL > > > value. MPLS-TP OAM is unaffected. > > > > > > 3. Where MPLS is carried over MPLS (an existing case) or over > > > multiple MPLS-TP, the multipath support in LSP Ping is used > > > and LSP Ping (RFC 4379) operation is unaffected. > > > > > > I think adding the above as a conclusion section as Section 5 would > > > improve clarity and therefore is worth doing. > > > > > > > --- Editorial comments/nits: > > > > > > > > - Section 3.1, after RFC 5960 quotes: > > > > > > > > [RFC5960] paragraph 3 requires that packets within a specific ordered > > > > > > > > Insert "Section 3.1.1.", before "paragraph 3" and likewise for > "paragraph 6" > > > > later in the same paragraph in the draft. > > > > > > GenArt review caught this. Done. > > > > > > > - Section 3.2, requirement MP#3: > > > > > > > > MP#3 It SHOULD be possible to insure that MPLS-TP LSPs will not be > > > > > > > > "insure" -> "ensure" or "assure". Surely this draft does not envision > > > > operators taking out insurance policies on MPLS TP LSP behavior ;-). > > > > > > Good catch. I think "ensure" is the right word. > > > > > > > - Section 3.2, last paragraph on p.8: > > > > > > > > An MPLS-TP LSP may not traverse multipath links on the path where > > > > MPLS-TP forwarding requirements cannot be met. > > > > > > > > "may not" is meaningless - I believe "MUST NOT" was intended here. > > > > > > I suggested SHOULD NOT above. > > > > > > > - Section 4, middle of 3rd para: > > > > > > > > Editorial suggestions for clarity: > > > > > > > > OLD > > > > For those LSP that are larger than component link > > > > capacity, their capacity are not integer multiples of convenient > > > > capacity increments such as 10Gb/s. > > > > NEW > > > > For those LSPs that are larger than a component link > > > > capacity, the LSP capacities need not be (and often are not) > > > > integer multiples of convenient capacity increments such as 10Gb/s. > > > > > > Good suggestion. GenArt review suggested adding "generally" making > > > the statement read "their capacity are generally not integer > > > multiples". Your suggested rewording may be more clear. > > > > > > Also "10Gb/s" should be "10 Gb/s". > > > > > > > In particular, "are not" seems wrong, as the "integer multiples" > > > > case is possible, so I changed "are not" to "need not be (and often > > > > are not)" in the suggested new text. > > > > > > > > Thanks, > > > > --David > > > > ---------------------------------------------------- > > > > David L. Black, Distinguished Engineer > > > > EMC Corporation, 176 South St., Hopkinton, MA 01748 > > > > +1 (508) 293-7953 FAX: +1 (508) 293-7786 > > > > david.black@xxxxxxx Mobile: +1 (978) 394-7754 > > > > ---------------------------------------------------- > > > > > > Thanks for the review and for pointing out lack of clarity due to > > > underlying assumptions about the use of RFC 4206 (and perhaps use of > > > RFC 4201 as well for large MPLS LSP over MPLS-TP). > > > > > > Sometimes it is too easy to forget that not all readers are going to > > > be as steeped in MPLS "as deployed today" details as a relatively > > > small number of long time MPLS participants are. > > > > > > Adding a few paragraphs to the introduction and a few concluding > > > paragraphs may significantly improve clarity for people who have not > > > been following MPLS closely for a decade or more without requiring > > > them to read the references and figure out usage details themselves. > > > > > > Please let me know how you feel about the suggested additions to the > > > document. > > > > > > Curtis >