Joel, Sorry for the late reply. Bufferbloat. Thank you very much for the review. Uploaded first round fixes for your review to github, will push out -14 when i am through with the other reviews. Version: https://raw.githubusercontent.com/anima-wg/autonomic-control-plane/master/draft-ietf-anima-autonomic-control-plane/draft-ietf-anima-autonomic-control-plane-14.txt Diff to -13: http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=https://www.ietf.org/id/draft-ietf-anima-autonomic-control-plane-13.txt&url2=https://raw.githubusercontent.com/anima-wg/autonomic-control-plane/master/draft-ietf-anima-autonomic-control-plane/draft-ietf-anima-autonomic-control-plane-14.txt Comments inline below. (n) additional explanations at end of mail. Cheers Toerless On Thu, Mar 01, 2018 at 06:18:45PM -0800, Joel Halpern wrote: > Reviewer: Joel Halpern > Review result: Ready > > I have been selected as the Routing Directorate reviewer for this draft. The > Routing Directorate seeks to review all routing or routing-related drafts as > they pass through IETF last call and IESG review, and sometimes on special > request. The purpose of the review is to provide assistance to the Routing ADs. > For more information about the Routing Directorate, please see > ???http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir > > Although these comments are primarily for the use of the Routing ADs, it would > be helpful if you could consider them along with any other IETF Last Call > comments that you receive, and strive to resolve them through discussion or by > updating the draft. > > Document: draft-ietf-anima-autonomic-control-plane-13 > Reviewer: Joel Halpern > Review Date: 1-March-2018 > IETF LC End Date: N/A > Intended Status: Proposed Standard > > Summary: I have some minor concerns about this document that I think should be > resolved before publication. > > Major: N/A (Note that three comment is marked *** as they may be more > significant.) > > Minor: > The description of the ACP In the introduction should probably note that > the addition of Management capability is beyond the definition provided in > section 5 of RFC 7575, and is included to complete the needed set of > functionality. Added this to the end of introduction before discussion of stable connectivity, seems to fit best there: <t>The ACP provides secure IPv6 connectivity, therefore it can not only be used as the secure connectivity for self-management as required for the ACP in <xref target="RFC7575"/>, but it can also be used as the secure connectivity for traditional (centralized) management. The ACP can be implemented and operated without any other components of autonomic networks, except for the GRASP protocol which it leverages.</t> > The definition of data-plane seems to assume that all forwarding (and > control?) in all nodes is structured in terms of VRFs. That seems to > overly constrain implementation architecture. Using VRF to to describe the > ACP functionality is a little odd, but likely defnesible. Declaring that > all other functionality is in VRFs seems a step too far. Oh... rathole ;-) I spent a lot of time pondering this problem. Given how i think the classical way VRFs are implemented in routers is actually NOT a good way to implement the ACP, i tried long to figure out better terminology. But: VRF is actually the most logical way to specify the interoperability, routing and forwarding aspects of the ACP without really implying a specific implementation approach. For example: If we would have used a term like "virtual router", we would have run the risk of excluding the option to implement ACP as a "classical router OS VRF". On the other hand, by using the term VRF (including our definitions what we mean with it), i can perfectly implement the ACP as a virtual router, as a function in a hypervisor or as a classical router OS VRF. VRF only says separate routing and forwarding (see section 6.9). Note too that we have explicitly specified in the terminology section that we're NOT implying any of the MPLS/VPN-PE specific functions of a VRF. Like othrer RFCs as well that use the term VRF without having anything to do with MPLS/VPN. In response to you bringing up the naming issue, i went through the doc and removed "VRF" where it was used for the data-plane, so as not to imply the use of VRF other than for the ACP itself. There is still some amount of VRF in the "VRF selection" section, thats really too hard to describe without VRF - would just make it too convoluted. > The definition of Intent in the terminology section seems to be different > from the cited definition in the anima reference model. "Intent" there > seems to refer to a policy language, not an interface. Thanks. Good catch. Fixed. > It is unclear how the flexible policy defined in section 5 bullet 2 (about > which nodes are ACP peer candidates) is consistent with autonomic > operation. It seems that the flexibility is important, so there should be > some explanation here about how this is consonant with the stated goals. I > understand that the bootstrap comes from BRSKI, but I do not think that is > where the policy comes from? Would rather not like to add more suggestive text, and thats at best what i could add. The default policy is the best "autonomic" behavior we know how to make work: aka: try to connect ACP to all neighbors you can discover. And we have only defined with DULL GRASP how to find subnet adjacent neighbors. The main reason to mention policy is so that there is some leeway to do more or even (sigh) less than all direct neighbors. See also (1) at end. > The BNF is section 6.1.1 seems to have some oddities. It may be that none > of these can happen for reasons that are described elsewhere, but as > written: > If the local-info is empty, the localpart will be "rfcxxxx." and thus the > domain info will be "rfcxxxx.@domain". Is the period really required in the > absence of the local-info. modified local-part = key "." local-info to local-part = key [ "." local-info ] > If the rsub is empty, but the extensions are not > empty, it looks like there will be two plus signs (following the optional > acp-address) before the first extension. Is that intended as the way to > indicate the absence of the rsub? Yes. Otherwise it would be ambiguous to parse. Haven't written something about this because i think readers eithrer don't care or the can figure it out easily. > It also seems odd that the rsub (which is a > domain extension) appears in the local-part of the domain information, not in > the domain name. Even though for hashing purposes it is concatenated, with a > period, with the domain name. > If these oddities are intentional, then there ought to be explanations so the > reader is not confused. The difference between domain and routing-subdomain are explained directly after the BNF: <t>"domain" is used to indicate the ACP Domain across which all ACP nodes trust each other and are willing to build ACP channel to each other. See <xref target="certcheck"/>. <t><t>"routing-subdomain" is the autonomic subdomain that is used to calculate the hash for the ULA prefix of the ACP address of the node. ... Added: "rsub" needs to be in the "local-part"; it could not syntactically be separated from "domain-name" if "domain" is just a domain name. It also makes it easier for domain name to be a valid e-mail target. > I presume there is not an assumption that all ULA addressed communicating > with a node which supports the ACP will be over the ACP. As I understand > it, ULAs may well have other uses outside of ANIMA / the ACP. Which leads > to a question about how the text in 6.1.3 "If the CDP URL uses an IPv6 ULA, > the ACP node will try to reach it via the ACP." is expected to be supported? The reason is that the ACP certificate maintenance runs across the ACP so that we have a simple security model for the ACP certificate lifecycle. We do not want to have to figure out the impact of attacks against connectivity to the CDP when it runs across the data plane. After all, these are ACP certificates are also renewed via EST across the ACP. I added the following explanation sentence: Reaching the CDP across the ACP implies that the CDPs need to be reachable via the ACP, for example by running CDPs on registrars or on devices connected to the ACP via ACP connect (see <xref target="ACPconnect">).</t> See also (2) at end. > The discovery text in section6.3 states that discovery should be run in > every physical interface. Is this intended to be restricted by policy, as > described in section 5, bullet 2? As said above for 5 bullet 2, it is permissible to be modified, the ACP draft does not suggest or recommends any modifications other than those in 6.3, aka: physical interfaces. > *** The explanation in section 6.5 on channel selection (and security > protocol selection) is worded as if the described behavior will lead to > reliable interoperation. The text on why there are limitations (reasonable) > doe not explain why mandating dTLS support would not be a reasonable step. (I > hope I missing something, as otherwise this is a major problem rather than > being minor.) It is also hard to align this with the requirements in section > 6.7.3. I think there is no issue. Let me explain: We beat ourselves up for quite a long time about MTI channel options (Mandatory To Implement) and figured there is no one-size-fits-all. The first challenge for dTLS was that there is no RFC specifying how to run IPv6 over dTLS. There are widely supported / deployed variations like OpenConnect etc. running IP/IPv6 over dTLS, but they all havesome home-brewed complex session maintenance mechanisms because they where built for hub&spoke user VPN solutions. We do not only not need that additional complexity, there is also no RFC for them (only expired individual drafts). In the absence of a simple standardized specification and deployment experience in non-constrained equipment with simple IPv6 over dTLS, we did not want to make it MTI for devices that had a better alternative. I then tried to figure out what would need to be written to specify how to run IPv6 over dTLS, and that became 6.7.2. Many type of constrained devices want to run everything over dTLS because they already got that code for CoAP etc. pp, and they do often not have IPsec code in their software. Therefore dTLS is the better MTI for those devices. I have modified the dTLS MUST sentence to make it clearer: A constrained ACP node that can not support IPsec MUST support dTLS. ^^^^^^^^^^^^^^^^^^^^^^^^^^ > 6.8.2.1 has text about "hop-by-hop reliable retransmission as provided for > by TCP". I am unable to determine what this is saying. Changed sentence to: <t>Such shorter periodic retransmission of datagrams would result in more traffic and processing overhead in the ACP than the hop-by-hop reliable retransmission mechanism by TCP and duplicate elimination by GRASP.</t> > If I have used section 6.10.3 properly, when the Zone-ID is 0, all nodes > within the ULA are within a flat address space. that does turn the address > into an identifier. It is, as far as I can tell from the description, > still a locator in that packets will be routed to the node using the > address including the node identifier. It just means that routing has to > work in /127 addresses. (There is also a reference to "identifier" in > 6.10.3.1. From the usage there, I think that the intention is that this is > the generic "name" for the node. Given that it is routable, it is simply > an addresses, not an identifier or a locator.) My driver license number is (intended to be) an identifier. It stays an indentifier even if someone starts building a network with a routing table for driver license numbers. Likewise, the intent of zone=0 numbers is to be identifiers. And yes, they do double a locators when you have a flat routing table which is what we do in this doc. We just didn't describe options where zone=0 would not double as a locator, that is meant to be left for future work if we see sufficient use cases for it. We just tried to make the definitions extensible and define the intent of the format to be that of identifiers. > The description in section 6.10 / 6.10.1 / 6.10.2 is about the schema > identification is very confusing, and arguably a bad design. I guess you mean 6.10.2 / 6.10.3 and 6.10.4 ? > The two schemas use the same schema identifier. ^^^^^^^^^^ Type ? The difference in interpretation > is actually provided by the last bit in the upper 64. If these are the > same schema, then it should not need a bit to differentiate them. They > should be explained as a single schema. Which would presumably result in > the Z bit being part of the zone / subnet field. If they are two > different schemas, then move the differentiation to part of the schema > identifier (making it 3 bits). We did not want to make Z part of the base schemas 6.10.2 "Type" code because that would also make the Vlong scheme one bit shorter and that would reduce virtualization space in Vlong by a factor of 2. So, yes, encoding wise, 6.10.4 manual sub addressing space was carved out of the 6.10.3 zone address space via the Z bit, but thats just an encoding aspect. Functionally, 6.10.3/6.10.4 are quite different, and therefore it is IMHO textually correct to have them in different subsections. Carving out bits like Z isn't ideal, but in between the ULA prefix, the 64 bit local part we must have for external interfaces and best utilization of bits, we tried to rather optimize address space utilization and not beauty of encoding. Having at the end allows (as written) use of /63 instead of 2 * /64 routes. I am not sure how important this would be in the future. If you feel strongly about more beauty in encoding, i can move Z to the left and remove the notion of the possible /63 route optimization possible through the placement of Z. > This sort-of different but sort-of the same > makes implementation "interesting". Even if this is extra bit is only > needed for schema 00b, it should be in front of the Zone / subnet, not > after it. This leads to the obvious observation that either we need two > bits, of which we will have 3 settings, and we will use the fourth setting > as an extension mechanism if we ever need more formats, or we need at least > three bits all the time because we seriously think there will be more > formats. I very much hope there is no "interesting" implementation aspect; nothing in the current scheme that makes implemenation harder. If you have an example that you fear, i would like to hear it. To me, we are just defining address space allocation schemes. > In section 6.10.5, after stating that it is not even known what the needed > usage is for more V values Which sentence are you referring to when you say "not even known what the needed usage is" ? I can't find it, but i would like to rectify it if it actually is still there. The two addressing space Zone and Vlong do certainly address two very different network and node design considerations. In the Zone addressing format, we wanted to have a scheme for potentialy very large networks. Enterprises plus their attached IoT networks with hundreds of thousands of nodes. Worldwide enterprise WN/campus/branches. Or massive DCs. The one V bit in this space is just to solve the single most likely problematic implementation consideration of ANI: Allowing ANI to have a complete port number space of itself, therefore not conflicting with traditional management port number usage (if needed). Or similar. The Vlong addressing space goes the other way, allowing to build out management plane or ASA more easily as containers or other lightweight component models where each such component has its own address. We also had brainstormed some network layer functions (proxies) that had to eat up address space, so would use V bits. Both approaches are IMHO very valid and important options that we wanted to be able to support and let developers and deployments choose which one fits them best. > the document goes on to introduce the > complexity of classful nodes numbers, with the leading bit indicating how > long the node-number is. Yes, the rest of the text in that bullet tries to > explain why you need both sizes. It seems like needless complexity that > begs for mistakes in implementation. I was taking the example use cases we brainstormed into account, and some of them would require a lot more addresses in the node than others, therefore these two choices. Ultimately, we need to break through the chicken and egg problem of how to build more self-managing networks. These will require a different number of addresses inside nodes. Anything more intelligent than address space allocations would become a complex dependency making implementation/adoption even slower. I don't think that offering 2, 256, 64k addresses per node is too much flexibility. It IMHO necessary and sufficient to explore and we'll see what sticks. > In section 6.11.1.11 in describing the prefix lengths, I thought that the > point of zone addressing was to allow the use of /64 prefixes. But it > seems here that we will not do so ? As mentioned above: This document does not describe the routing setup for /64 (or /63 with Z-bit) routing aggregation. There are multiple options, but we could not conclude on a single solution that we felt to be appropriate for this document. Instead it was only very important to carve out addressing space to support this. See also (3) at end. > Section 6.12.5 says "ACP nodes MUST NOT derive their ACP virtual interface > IPv6 link local address from their IPv6 link local address used on the > underlying interface". But it does not then seem to provide any advice on > what is expected to be used. Is the loopback ACP supposed to be used > somehow? Or? Changed paragraph to following: <t>The ACP virtual interface IPv6 link local address can be derived from any appropriate local mechanism such as <xref target="EUI48"/> or <xref target="EUI64"/>. It MUST NOT depend on something that is attackable from the data-plane such as the IPv6 link-local address of the underlying physcial interface, which can be attacked by SLAAC, or parameters of the secure channel encapsulation header that may not be protected by the secure channel mechanism.</t> > The "configuration" description in 8.2.1 provides a nice description of > what information is needed, and what choices are allowed. It does NOT > describe a method for configuring the information. Please tune the > introductory material. Hope i am guessing correctly what you're alluding to. Fixed text: <t>On the ACP node, remote ACP neighbors are configured explicitly. The parameters of such a "connection" are described in the following ABNF. ... <t>Explicit configuration of a remote-peer according to this ABNF ^^^^^^^^^^^^^^^^^^^^^ provides all the information to build a secure channel > *** Section 10.3.2 paragraph 2 says that devices should change the meaning > of "admin down" to mean "down for everything except ACP / Anima". I can > understand why, in an autonomic network, such a state is desirable. However, > that really should be a different state from "admin down". Operators already > understand "admin down" as meaning that this interface will not be used for any > traffic. Changing that is fairly drastic. "admin limited" or "admin ACP-Only" > would seem much better than changing the meaning of "admin down". The > justification in the text seems to be a desire to prevent an operator from > doing what he intends. that seems backwards. (Note that the distinction > between administrative state vs operational state, aka failed, is already well > understood.) Well, this is in an informative section, so hopefully something we can agree to disagree. I strongly believe that the most easy way to operationalize the ACP and achieve its goals is what we have written. The historic equivalence of "down" = "admin down" = "physical down" is one of the the most fundamental problem of remote management in networks. We really need to start thinking of separate layers of (remote) management. Network services on one hand and physical infra on the other. The first thing to do this is to separate operations such as "down" into "admin down" that affects networks services and "phy down" that affects the physcial infra. If you have existing commands like "shutdown" that today do both, the safe mapping is to let them do the safe thing in the future, e.g: only "admin down", and introduce new commands for the phy operations, e.g.: "phy shutdown" or the like. And then protect those dangerous commands even further against unintentional or intentional misuse. Think of ACP/ANI also as a seamless inband version of a DCN. As an operator of the actual data network, you also didn't have any access to bring down the DCN and cut yourself off from the network you manage. YOu could only screw up that network you're meant to manage. And if your network consists of VMs in a DC, a "shutdown" on their interfaces will also not bring down physical interfaces necessary to reach the VM. In optical networks you often have an inband physcial ethernet management channel. Making/breaking that one is completely different from making/breaking your data-plane, aka: data fiber colors. > *** The notion in 10.3.6 that the device should refuse to disable > functionality when an authorized administrator directs such seems flatly wrong. The authorization to break your own remote management connectivity would be a property of the certificate. clearly whoever enrolled the device with a certificate denying that capaility to the operator did NOT authorize the administrator to break connectivity. > Editorial / Nits: > Section 6.3, in talking about DULL GRASP refers to the grasp internet > draft. It refers to section 3.5.2.2. there is no 3.5.2.2 in the grasp > I-D. As best I can guess the reference should be 2.5.2. Thanks > In the various formats in section 6.10, the lowest bit of the upper 64 bits > is mandated to be 0. Presumably, there is some reason for doing this. It > would be nice to explain it. Sorry, not clear what you're referring to. Can you give me an explicit reference ? > In section 6.10.5, in the text about how many V values are allowed, the > text reads: "2^16 - 65536" which would be 0. I believe that the intention > is "2^16 (i.e. 65536)" Thanks. Meant to say "2^16 = 65536", but now its your i.e. > Section 9.2.2 paragraph 2 begins "Group members must overall the secured". > While I can guess what is intended, I would prefer that were in grammatical > English so as to avoid guessing. Hmm.. not sure i catched the grammatical error i did, but i figured that the text was missing some better explanation what it meant. Added the following: <t>Group members must be protected against attackers so that there is no easy way to compromise them, or use them as a proxy for attacking other devices across the ACP. For example, management plane functions (transport ports) should only be reachable from the ACP but not the data-plane. Especially for those management plane functions that have no good protection by themselves because they do not have secure end-to-end transport and to whom ACP does not only provides automatic reliable connectivity but also protection against attacks. > Section 10.5 provides the requirements that were met by choosing RPL for > the routing protocol. Likely RPL is a reasonable choice. Since this > section does not provide comparison or list any alternatives, I would > suggest changing "explanation" to "motivation". Done. -------------------------------------------------------------------------- (1) Examples of good additional discovery would be methods to do it via L3 or where additional parameters ae needed, we discussed some options to help bring up interfaces like PPP requiring credentials or the like. On the "challenged product" side, we have seen products unwilling to bring up interfaces during bootstrap because those interfaces require additional license keys for activation *sigh*. (2)side note: We started from classical PKI model with CDP/OCSP but didn't bother much about it, so this is a bit of a step-child: Most of the ANI authors are fans of short lived certificates to eliminate CRL/CDP because through ACP and BRSKI, we can make lifetimes shorter than typical CDP validity intervals: ACP adds reliability, EST automation, BRSKI can provides re-enrollment even when certificate has expired so ACP can not be vbuilt. Therefore we didn't bother much in making CDPs as easy to use as we could, e.g.: no GRASP objective to auto-announce/discover CDP like we did for EST renewal. but i didn't want to blackhole the text with these considerations even further. ] (3) The most simple option we have seen asked for in actual deployments was for manual config of such zone edges, not IMHO really appropriate for this document. More like something for informational docs. Even in that case you could explore the traditional method where you actually hand out zone != 0 address via the domain cert, set up RPL roots for each zone and connect the roots. Or you do simply rely on zone 0 addresses, rely only on the default route in each zone and use something like lisp between zone roots to map from zone=0 as identifier to remote root locator. Aka: everything worth describing in other documents.
