Thanks, more discussion inline.
David
On Apr 18, 2023, at 7:00 PM, touch@xxxxxxxxxxxxxx wrote:
Hi, David and Magnus,
Replies below, cutting to the remaining issues…
(Trying again - I got reports of mail failures)
Joe
—
Dr. Joe Touch, temporal epistemologist
www.strayalpha.com
> On Apr 18, 2023, at 5:26 PM, David Schinazi <dschinazi.ietf@xxxxxxxxx> wrote:
>
> Thanks Joe and Magnus for the replies.
> Some more responses inline.
> David
>
> On Tue, Apr 18, 2023 at 1:23 AM Magnus Westerlund <magnus.westerlund@xxxxxxxxxxxx> wrote:
>> Hi,
>>
>> Please see inline. Prefix with “MW:”
...
>> Sec 4.7, although defining new capsules, is really about tunnel configuration.
>> It should be moved to a top-level section and presented as such.
>>
>> Except for Sec 4.7, which explains how IP addresses and routing can be obtained
>> over the HTTP connection, the document views the necessary behavior from only
>> the perspective of the tunnel ingress as an HTTP client and the tunnel egress
>> as an HTTP server.
>> That's not right, most of the document is written in terms of "IP proxying endpoints"
>> because most of the mechanisms are symmetric. Do you have a specific example?
>> The abstract, which talks about the server acting a proxy but not the client:
>>
>> ...this document defines a protocol that allows an HTTP client to create an IP tunnel through an HTTP server that acts as an IP proxy
>>
>>
>> Sec 2 definitions, again focusing on the behavior of the server, not the client:
>>
>> In this document, we use the term "IP proxy" to refer to the HTTP server that responds to the IP proxying request.
>> MW: So the HTTP usage does enforce a client server behavior. That can’t be ignored. Only a client can request the establishment of a tunnel and what potential filtering and other parameters/functions to apply. The tunnel is then created and then it becomes symmetric in that tunnel context.
Agreed, but the role of the client isn’t clearly described other than initiating the tunnel. It still acts as an ingress/egress at that endpoint. That’s not very clear.
That's fair. In the GitHub issue, Magnus suggested defining "IP proxying endpoints" more clearly in the terminology section. I think if we get this definition right it'll help a lot.
>> https://github.com/ietf-wg-masque/draft-ietf-masque-connect-ip/issues/167
>>
>> Sec 7 describes clients sending packets to servers; I could not easily locate the converse:
>>
>> Clients MAY optimistically start sending proxied IP packets ...MW: So there is no potential for the converse to happened. The client requests a tunnel, and might have packets to send for that tunnel, they can be sent prior to the HTTP response as well as any capsule arrives back at the client. But the server will have the request to create a tunnel, and will answer that before sending anything in that tunnel context. Thus, there are asymmetry here.
>
> There is some asymmetry (only the client can initiate the tunnel), but most of the document speaks in terms of "IP proxying endpoints" - e.g. the section about encapsulation. That makes it clear that both of them can send encapsulated packets.
It does, but there should be some checks to make sure it’s more clear that things happen in both direction.
Fair, some of this is currently implicit and should be made explicit for improved clarity.
>> It is missing the way in which these ingress/egress
>> components are viewed a their endpoints, e.g., to be useful as an IP tunnel,
>> these need to appear as attached to (possibly virtual) network interfaces,
>> i.e., to appear as a link, which allows them to then be used for local
>> processes (via sockets), packet forwarding, etc.
>> That's an implementation detail that doesn't belong in this document. Most
>> implementations will indeed use virtual TUN interfaces, but it's not a requirement.
>> There is a known implementation with transport protocols in userspace that doesn't
>> do what you describe.
>> Whether a TUN interface is used or some other method, there needs to be a method by which these applications (client, server) present something that accepts IP packets.
>>
>> That aspect of how this is actually used is ignored and needs to be addressed. It does not need to be implementation specific, but it would not hurt to give an example like TUNs.
>>
>> (The fact that other user-space IP systems ignore this issue is not rationale for this document also ignoring it)
>> MW: I don’t see how any text can be other than informational. Considering the below discussion. Are you asking for a general discussion of the boundaries between the routing and the link the tunnel that this construct results in, especially as it puts some traffic filtering rules in front of the encapsulation that affects the routing?
I’m asking for both the routing and endpoint behaviors to be described in relation to the tunnel.
>> https://github.com/ietf-wg-masque/draft-ietf-masque-connect-ip/issues/165
> I think I now understand what Joe was saying. We didn't make it clear enough that this document specifies a (virtual) link with routers attached to it. The IP proxying endpoints both act as routers that are connected by this virtual link. That's why we talk about having them send ICMP. Adding some text to clarify this should help. We'll cover this in issue 165.
If that’s what you’re defining, it is incorrect. It can’t be a router. If it were and traffic were to go from tunnel to real interface, it would have its IP decremented twice, which is inconsistent with RFC1812.
This is a tunnel. It should not try to be a router or a host. It can’t issue ICMPs properly esp. because it can’t take into consideration the relation of this tunnel to other interfaces, the endpoint and its rules for ICMP (per RFC1122) or a router and its rules for ICMP (per RFC1812).
What you are describing is not a convenience. It’s *incorrect*. I’ve noted that in the text deleted between here and the next issue (see past emails for that detail).
Perhaps another way to present this is that connect-ip is a virtual link with a half router on each side, but that gets harder to reason about. In practice, we do need some router functionality here to ensure that a packet that loops between virtual connect-ip links will have its TTL decremented to prevent infinite loops. That can be implemented by sending packets through the kernel, but some implementations might want to handle that all in the connect-ip process to improve performance - and we need to make sure those implementers don't forget to decrement the TTL.
>> As other reviewers have noted, Sec 10 on nested congestion control is quitethin. The current statement is equivalent to “if you KNOW congestion is nested,
>> turn it off” – it should be the opposite, i.e., “turn congestion ON only if you
>> KNOW congestion is NOT nested”.
>> Fair enough. We're tweaking that section to be more permissive:
>> https://github.com/ietf-wg-masque/draft-ietf-masque-connect-ip/pull/162It’s not about allowing congestion control to be disabled; that needs to be a SHOULD, with the caveat that when it is not, performance can suffer in ways that are difficult to predict.
>> MW: I agree, and we could actually say SHOULD in that text under those constraints rather than MAY. However, doing this disabling first of all requires support of the DATAGRAM extension and will require changes to the QUIC stack that might not be possible in all deployment scenarios. And it can’t be done in general as some usage of this tunnel specification might happen over other HTTP versions than HTTP/3 that uses TCP.
>> https://github.com/ietf-wg-masque/draft-ietf-masque-connect-ip/issues/164
“It can’t be done” in some cases - that can be used as a rationale for implementers not following the SHOULD (i.e., the reason for an exception).
> I personally don't think a SHOULD is reasonable here. We don't have enough data to demonstrate that this advice is always sound. My personal experience is that nested congestion control loops work fine in practice. (Note that this is nested congestion control, not nested loss recovery.) Whether this should be done or not is very dependent on the deployment environment. Adding some text warning of the risks is always good, but a normative recommendation is a step too far.
This isn’t an issue for anecdotal discussion; it’s been proven in the literature. Nested control loops are never stable unless they’re specifically designed to do so, and two instances of the same control with the same parameters (TCP on TCP, esp. if the same variant) is particularly bad.
I’m suggesting normative SHOULD with specific exceptions that can include places where it can’t be done.
I've seen literature about nested TCP, which is both nested congestion control and nested loss recovery. In my understanding, the majority of the issues come from the two layers retransmitting the same data, not from the nested congestion controllers. I haven't seen literature about nested congestion control without nested loss recovery. If you squint hard enough, any IP router connected to heterogeneous links is a congestion controller because if the input link is getting more packets in than the output link can handle, the router will drop some of them. In that world view, almost every TCP connection on the Internet involves nested congestion controllers, and it's working quite well.
>> Section 11.1 refers to fragmented packets; it should refer to them as not being
>> able to be “re-fragmented”; source-generated fragments are still fragmented and
>> can cross the tunnel subject to the tunnel MTU.
>> The use of "fragmented" in that section refers to QUIC datagram frames, which
>> cannot be fragmented - this isn't about IP fragmentation.
>> The section talks about whether IP packets can fit inside QUIC datagram frames.
>>
>> Fragmentation of those packets can - and will - happen when those packets are generated on the host where the packets enter the tunnel, unless the host decides to force “don’t fragment” on those packets. That’s a decision that happens (could happen or should happen, depending on your viewpoint) before the packets ever get to the tunnel ingress.
>>
>> On-path fragmentation of IPv4 packets relayed to the IP proxy happens (could happen or should happen, again depending on your viewpoint) before those packets ever get to the tunnel ingress.
>>
>> Either of those can happen - even if QUIC datagrams sit inside IP packets with DF=1 (or IPv6) that are also not source fragmented.
>>
>> MW: So I think this may need a bit of wording clean up and also clarification of the conceptual model. If I understand Joe correct here a reasonable way of looking on this is that when a packet arrive at the router part with this tunnel as one of its interfaces, the router part will have some knowledge of the current tunnel MTU. The tunnel itself will not refragement the data, but it will support a particular MTU. Thus, the router part can actually fragment an IPv4 packet that doesn’t have the DF bit set and send it into the tunnel. And in relation to discussion about ICMP generation. It will be the router part that generates the ICMP when the routing decision says send it over the tunnel, but the tunnel MTU is too small for the packet to fit.
And, FWIW, that’s another reason why the routing part belongs outside of the tunnel. It’s not just routing, it’s also endpoint (source) behavior. The tunnel has no business trying to replicate this behavior when it’s already part of the endpoint system (if it weren’t, then HTTP over X over IP wouldn’t be available as a tunnel mechanism).
>> https://github.com/ietf-wg-masque/draft-ietf-masque-connect-ip/issues/165
> I agree with Magnus, if we clarify the split between link and router then this becomes natural.
I agree with the split, but the router cannot be part of this mechanism.
In practice, implementers of connect-IP will often implement part of the router function, so it's useful to mention it so folks don't forget important parts. Phrasing it as not a part of the mechanism and instead as a part of the overall environment is reasonable.
>> Section 11.1 should require that the IP tunnel be created with a given MTU and
>> to indicate that to the endpoint where the client/servers reside; issuing ICMP
>> PTBs is the responsibility of those endpoints when they are seeking to use
>> those tunnel endpoints, NOT of the tunnel endpoint itself. Tunnels are links;
>> links do not issue ICMPs.
>> This use of ICMP is a pragmatic solution to ensure that the link doesn't violate
>> the minimum IPv6 link MTU. Some working group members felt strongly that
>> we needed a solution for this.It is incorrect. ICMPs can be issued by hosts or routers, but not by links (tunnels). Again, see draft-ietf-intarea-tunnels sec 3.2 as to why.
>>
>> The solution is to explain how this IP tunnel ties into the end hosts or router - again, that’s the gap in this document.
>> MW: I do believe I see your point here. There are parts of this specification that impacts the router part. The scoping is basically traffic filters for routing decision as well as what you brought up with how ICMP is handled. To address this can have some significant impact on the document and I think we need further discussion among the authors for how to handle this.
IMO, the only viable path forward is to remove this from the tunnel. There’s no way to spec this as a tunnel function and also how it interacts with existing endpoint / router functions. Removing it is both simpler and correct.
>> Sec 12 should indicate how access to the HTTP proxy itself should be access
>> limited, i.e., to avoid presenting an arbitrary traffic injection point.
>> Are you talking about things like <<don't set your root SSH password to
>> "password">>?
>> Not quite; it’s more like “don’t run root SSH with no password”.
>>
>> Regardless, it creates a huge DOS opportunity - how is the endpoint protected?
>>
>> MW: So from my perspective the whole first paragraph talks about this.
>>
>> There are significant risks in allowing arbitrary clients to establish a tunnel that permits sending to arbitrary hosts, regardless of whether tunnels are scoped to specific hosts or not. Bad actors could abuse this capability to send traffic and have it attributed to the IP proxy. HTTP servers that support IP proxying SHOULD restrict its use to authenticated users. Depending on the deployment, possible authentication mechanisms include mutual TLS between clients and proxies, HTTP-based authentication via the HTTP Authorization header [HTTP], or even bearer tokens. Proxies can enforce policies for authenticated users to further constrain client behavior or deal with possible abuse. For example, proxies can rate limit individual clients that send an excessively large amount of traffic through the proxy. As another example, proxies can restrict address (prefix) assignment to clients based on certain client attributes such as geographic location.¶
Ahh - thanks. I missed that. That’s what I was looking for.
>> This is about limiting access to the functionality of establishing tunnels, as well as potential having additional resource constraints on a tunnel.
>>
>> It should also address the potential use of HTTP-level encryption (e.g., TLS,
>> DTLS) to protect the tunnel contents and tunnel configuration exchanges.
>> We require the scheme to be https, which means that QUIC or TLS is mandatory.
>> I see that in the examples, but it doesn’t appear to be a requirement anywhere. Can you indicate where it explains that “http://…” would be considered invalid or that ONLY “https” can be used to start?
>>
>> MW: I think Joe is correct that this is only implicit required when using HTTP/3. We don’t actually have a requirement to use HTTPS.
>>
>> https://github.com/ietf-wg-masque/draft-ietf-masque-connect-ip/issues/163
>
> Oh you're right, the use of the https scheme isn't actually mandated. That said, we shouldn't mandate use of HTTPS, since there can be other future schemes. Instead we should RECOMMEND running HTTP over TLS or QUIC to provide confidentiality, integrity, and authentication. Given the existence of HTTP intermediaries, there will be scenarios where the client-to-intermediary security is provided by QUIC and the intermediary-to-proxy security is provided by other mechanisms such as IPsec or proprietary hardware-offloaded security.
It might be useful to augment the discussion of limiting access to include a “SHOULD NOT” use this without TLS or IPsec or something like that, or explain how to limit access if TLS/IPsec/DTLS are not used.
We're adding a "MUST use TLS or QUIC or equivalent"
-- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
