Brandon Williams and I reviewed this draft with Kathleen and Al via the IETF etherpad during last call. The link is here:
https://etherpad.tools.ietf.org/p/krose-review-draft-mm-wg-effect-encrypt-13
I have attempted to cull the discussion down to something consumable on the mailing list. Apologies in advance for formatting issues. Note that I have not subsequently added any comments to the etherpad: I'll respond to this thread instead with any follow-ups.https://etherpad.tools.ietf.org/p/krose-review-draft-mm-wg-effect-encrypt-13
this mode is deployed. IPsec with authentication has many useful
applications and usage has increased for infrastructure applications
such as for virtual private networks between data centers.
KR> The above paragraph doesn't seem to have a clear point. It's mostly about opportunistic security, but has a few unrelated points inserted (e.g., the final sentence about IPsec with authentication).
KM> Looks like final sentence provides contrast with the previous sentence
"OS has been implemented as NULL Authentication with IPsec..."
the application user, and hosting service providers lease computing,
storage, and communications systems in datacenters. In practice,
many companies perform two or more service provider roles, but may be
historically associated with one.
KR> Honestly, all of section 1.1 seems to be a grab bag lacking a thesis, maybe motivating the vague title "Additional Background". I think the information there could be organized in a better way to answer the question posed in the reader's mind after the introduction, which is: give me some examples of monitoring/manipulation for operability that have been defeated by encryption, with evidence that they aren't solvable without additional telemetry or cooperation with middleboxes. The following section is closer to what I wanted after the intro.
KM> Initially, we had some examples in the draft, but were asked to remove them. I take your point and am in process of reworking this text to make it more cohesive. The problem in much of this draft was the number of contributions, so your sweep of the draft for points like this are very helpful.
AM> As I read it, section 1.1 is describing changes to the communication landscape re: encryption. The exception is the very last paragraph, which could be moved above section 1.1. I agree that the purpose of section 1.1. could be made more clear in the introductory paragraph. I do think that it's useful background to high-light these changes in the encryption landscape, since that's what motivates the draft.
KM> Thanks. It is also to ensure the draft is not read as being just about TLS, so I am making that point more clear as well. Moving the last paragraph to the introduction makes sense.
Following the Snowden revelations, application service providers
responded by encrypting traffic between their data centers (IPsec) to
prevent passive monitoring from taking place unbeknownst to them
(Yahoo, Google, etc.). Large mail service providers also began to
KR> IIRC, companies were already doing this for infrastructure traffic on the public internet: it was their own private backbones on which they started using encryption universally.
KM> Yes, good point. I clarified the text to distinguish that, but was careful not to say that they pretty all infrastructure traffic over the Internet was encrypted even though I think that's in line with reality - at least where I worked which included an ISP and a big financial data provider early on.
The EFF reported [EFF2014] several network service providers taking
steps to prevent the use of SMTP over TLS by breaking STARTTLS
KR> I think it's important to use the phrase "downgrade attack" when describing something like this. The word "downgrade" first appears in the references list.
KM> Done, thanks.
(section 3.2 of [RFC7525]), essentially preventing the negotiation
process resulting in fallback to the use of clear text. In other
cases, some service providers have relied on middle boxes having
access to clear text for the purposes of load balancing, monitoring
for attack traffic, meeting regulatory requirements, or for other
purposes. These middle box implementations, whether performing
functions considered legitimate by the IETF or not, have been
impacted by increases in encrypted traffic. Only methods keeping
with the goal of balancing network management and PM mitigation in
[RFC7258] should be considered in solution work resulting from this
document.
KR> I feel like this section could be better organized by:
* Moving the examples to 1.1 as a bulleted list of sample situations in which network operators attempted to and/or succeeded in defeating encryption to preserve existing operational mechanisms, or in which performance suffered for users (whether of the encrypted flows or of other flows impacted by encrypted flows).
KM> Interesting point, but we'd need more examples. I'll think about this more and chat with Al in case he has ideas. For now, I went with Brandon's easier suggestion, but moving to this would be nice for the document readers.
AM> Although I see how these examples could be part of the background, I think those who will
eventually remove their objections will prefer the reduced emphasis on these examples where
they are (in section 2). In one view, the entire memo is background, since nothing new is proposed.
KR>
* Using this section as an introduction to the methodology for cataloging operational mechanisms depending on cleartext traffic monitoring, with the various caveats on what will be considered (e.g., only mechanisms required heretofore for operability), and for describing the approach to seeking mitigations and/or substitutions.
KM> Hmm, interesting point. I'll have to think about this more as it could be alot of work at this stage.
AM> Unfortunately, we've already implemented many AD-level suggestions on the organization of Section 2.
We're at the stage of *what can everybody live with", and re-re-re-org falls out now, IMO.
Network service providers use various techniques to operate, manage,
and secure their networks. The following subsections detail the
purpose of each technique and which protocol fields are used to
accomplish each task. In response to increased encryption of these
fields, some network service providers may be tempted to undertake
undesirable security practices in order to gain access to the fields
in unencrypted data flows. To avoid this situation, ideally new
methods could be developed to accomplish the same goals without
service providers having the ability to see session data.
BW> I think the above paragraph is the core point of the section; describing what the whole of section 2 is about. The previous paragraphs, while important information, don't seem to belong in this section. Perhaps a separate section focused on observed bad behavior would be better.
KM> That would get at both your point and Kyle's, thanks.
AM> I agree this last paragraph could be moved-up (after the definition of Network SP).
The Snowden and EFF paragraphs would be best positioned as footnotes to the
first sentence "Network service providers use various techniques ...", but we don't
have that mechanism available.
Also, the neutral exposition that we've been asked to provide a million times actually
comes from multiple perspectives expressed in contributions that we would combine
in a balanced way, without value judgements (no good or bad).
Where we lack balance, we lack specific contributions.
(So, I think the new -14 section 1.2 does not have an appropriate title.)
heuristics grows, and accuracy suffers. For example, the traffic
patterns between server and browser are dependent on browser supplier
and version, even when the sessions use the same server application
(e.g., web e-mail access). It remains to be seen whether more
complex inferences can be mastered to produce the same monitoring
accuracy.
KR> This might be too formal of an approach for this doc, but it might be possible to construct a taxonomy of layers of metadata made unavailable by encryption at each layer to show the completeness/comprehensiveness of the survey. So, for instance:
* Protocol and port number are still available as a way of characterizing traffic over the public internet even if the payload is encrypted, but this information is lost if (e.g.) the traffic is traversing an IPsec tunnel or if radically different kinds of traffic all use port 443/tcp without any other way to distinguish between them.
* TCP is open to optimization/measurement even if using TLS, except when tunneled encrypted: congestion signals (like rexmits) previously transparent to the middlebox, for instance, are then lost.
* Encrypting the payload defeats attempts to survey traffic by user agent (if there's no other way to distinguish, e.g., by fingerprinting).
KM> I think this would be a really helpful follow on document. I'd be willing to work on it if you're game. I've been thining about something similar, specific to TLS, but should be broadened.
It is important to note that the push for encryption by application
providers has been motivated by the application of the described
techniques. Some application providers have noted degraded
performance and/or user experience when network-based optimization or
enhancement of their traffic has occurred, and such cases may result
in additional operator troubleshooting, as well.
KR> Observation: additionally, I think you'll encounter the argument that the responsibility for diagnosing bad interactions between applications and networks falls on the application owner rather than the network operator. Basically, I feel like the desire among protocol designers is for operators to provide a pipe with certain key characteristics that interact well with established transport protocol mechanisms, and otherwise to leave the traffic alone and let the application developers do what they want to within the expected constraints. If that's infeasible (e.g., in edge cases, or with respect to new technologies that interact badly with existing transports, such as the loss=congestion assumption of TCP that interacts badly with wifi), that's precisely the case needs to be made by this document.
KM> We have encountered this argument already. It's a tough one as SPs have the SLAs with customers, so they are the first call. Many don't know how to get in touch with APP providers. I understand the application developers perspecive, but also see that there has to be some ability to troubleshoot. Sure, providers could wrap the protocols for transport to provide some way of measuring, but information is lost. IPv6 with flow identifiers is another way to do it, but you might not be able to prioritize a call or protocol that has little tolerance for delay over one that does for instance. And I realize that app providers just want all traffic to have the same priority, but emergency calls are important.
BW> I think the point made by the document is correct though: operators are nearly always the first call, not the application provider.
KM> We were asked to remove text that said that. I agree that it is the case as the providers have the SLAs and you don't typlically have a number for App providers.
BW> The operators are looking for ways to demonstrate that they did not cause the problem (or determine that they did) for efficient hand-off to the correct party for resolution. There are certainly problems an approach that changes the behavior of the protocol, but it's difficult to argue with the diagnostic need.
AM> Using Netflix as an example, the first source of problem they mention is the network when
addressing the question "Why doesn't Netflix work?":
"If Netflix isn’t working, you may be experiencing a network connectivity issue, an issue with your device, or an issue with your Netflix app or account."
from https://help.netflix.com/en/node/461?ui_action=kb-article-popular-categories
They previously had even stronger wording, something like "First, make sure your network connection meets the Netflix requirements ... URL"
One of the causes of re-buffering are CDN-related pauses when accessing the next segment: completely hidden from users so far.
Additional frequent cause: the unlicensed WiFi network owned and operated by the customer.
Another way to look at this strategy: App providers are transferring as much overhead cost to the network operators as possible
(troubleshooting customer problems is expensive - rolling a truck negates months of revenue), while preserving as
much value/control/revenue as they can for themselves. The greed-thingy plays poorly over time.
A user-focused strategy would be to form partnerships for troubleshooting of shared customers, but that might result in exposing
the real causes and some would rather hide for now, it seems.
For example, browser fingerprints are comprised of many
characteristics, including User Agent, HTTP Accept headers, browser
plug-in details, screen size and color details, system fonts and time
zone. A monitoring system could easily identify a specific browser,
and by correlating other information, identify a specific user.
KR> Subsections of 2.1 cover the following in what feels like arbitrary and inconsistent order: technique description, justification for the technique, reason why the technique is bad (for privacy), how the technique is defeated by protocol designers, and examples of the technique. It really reads like a laundry list rather than a systematic analysis of the problems faced, the metadata required for diagnostics, and how these techniques are defeated by encryption.
KM> Hmm, this section was reorganized by others in the last IESG review, so that's probbaly part of the problem. I'll read through and see what I can do to help it out more. It cleared a discuss to make the changes.
** didn't tacle this one and will go back to it.
AM> To re-iterate: this isn't the optimization phase. We've done 10 months of that.
We've reached "what can you live with" phase, IMO.
packet is able to provide stateless load balancing. This ability
confers great reliability and scaleability advantages even if the
flow remains in a single POP, because the load balancing system is
not required to keep state of each flow. Even more importantly,
there's no requirement to continuously synchronize such state among
the pool of load balancers.
KR> An important point is that an integrated load balancer repurposing limited existing bits in transport flow state must maintain and synchronize per-flow state occasionally: using the sequence number as a cookie only works for so long given that there aren't that many bits available to divide across a pool of machines.
KM> I added in this point, but have to check back on flow of text.
Current protocols, such as TCP, allow the development of stateless
integrated load balancers by availing such load balancers of
additional plain text information in client-to-server packets. In
case of TCP, such information can be encoded by having server-
generated sequence numbers (that are ACK'd by the client), segment
values, lengths of the packet sent, etc.
KR> Is it worth mentioning that the use of some of these mechanisms for load balancing negates some of the security assumptions associated with those primitives (e.g., that an off-path attacker guessing valid sequence numbers for a flow is hard)?
KM> I added the above in as it may offer some balance to the discussion.
KR> A dedicated mechanism for storing load balancer state, such as QUIC's proposed connection ID, is strictly better from the load balancer's point of view, and is probably even better from a privacy perspective than bolting it on to an unrelated transport signal because it can be tightly controlled by one of the endpoints and rotated to avoid roving client linkability: in other words, being a specific, separate signal, it can be governed in a way that is finely targeted at that specific use-case. (I'm thinking the advantages of separate mechanisms belongs in a different part of the doc; this section is more like the problem statement than the solution statement.)
KM> This (above) needs to be reworded to be neutral and this does go towards solution space, which we were trying to avoid. How about:
Another possibility is a dedicated mechanism for storing load balancer state, such as QUIC's proposed connection ID to provide visibility to the load balancer. An identifier could be used for tracking purposes, but this may provide an option that is an improvement from bolting it on to an unrelated transport signal. This method allows for tight control by one of the endpoints and can be rotated to avoid roving client linkability: in other words, being a specific, separate signal, it can be governed in a way that is finely targeted at that specific use-case.
In future Network Function Virtualization (NFV) architectures, load
balancing functions are likely to be more prevalent (deployed at
locations throughout operators' networks)[. NFV environments will
require some type of identifier (IPv6 flow identifiers, the Proposed
QUIC connection ID, etc.) for managing]
traffic using encrypted tunnels.[ The shift to increased encryption
will have an impact to visibility of flow information and will require
adjustments to perform similar load balancing functions within an NFV.]
KR> I'm not sure what architecture this paragraph is discussing: are you talking about encrypted tunnels between NFV nodes? Is this something obvious to people involved in NFV? A diagram (or informational reference) would be helpeful to me here.
KM> I see your point, the langauage here could be more clear. Do the above adjustments (ed: in []) help?
2.2.2. Differential Treatment based on Deep Packet Inspection (DPI)
...
These effects and potential alternative solutions have been discussed
at the accord BoF [ACCORD] at IETF95.
KR> This section is labeled DPI, but really, the underlying issue is what you stated in the first paragraph: different kinds of traffic have different QoS needs, yet a network provider can't rely on a voluntary signal from an untrusted device to decide on QoS or every packet is simply going to be marked "high importance" and so we're back to treating all traffic equivalently. I'd argue against one of the memes I heard at the accord BoF, that it's down to latency vs. throughput, by pointing out that some applications (e.g., live video with low hand-wave latency) need both.
Even after reading this, I'm still skeptical of the need for any more granularity than flow, and using AQM on a per-flow (e.g., 5-tuple) or flow-aggregate (some subset of the 5-tuple) to prevent an application or user from consuming resources unfairly. What, for instance, prevents a carrier from privileging VoIP traffic by looking at endpoints? Would there be a way for someone else to masquerade non-VoIP traffic as VoIP traffic given this kind of setup? This is the kind of question that I need answered by this doc.
BW> It might be useful to note in this section that QUIC and H2 both combine multiple micro-flows, possibly of different types, within a single encrypted transport-layer flow. They share this with IPsec tunnels and the like. IOW, the increased use of encrypted aggregating encapsulation can hide even the the most basic representation of a flow from the differentiated service element. This same concern applies to load balancing elements discussed in section 2.2.1.
KM> ** Want to talk with Al on this set of comments.
AM> We were asked not to refer to QUIC, for various reasons (e.g., still under development).
There will always be areas where network can make the best decision, because of the
information available to the network operators (and the lack of that same info at end-points).
When network resources are constrained, only the network can manage priorities.
This has been organized according to applications that can be identified, but there
can be other solutions requiring cooperation between user devices and the network
according to subscription to a special service (QCI above).
2.2.3. Network Congestion Management
For User Plane Congestion Management (3GPP UPCON) - ability to
understand content and manage network during congestion. Mitigating
techniques such as deferred download, off-peak acceleration, and
outbound roamers.
KR> This seems like a special case of 2.2.2.
KM> Al - is there a reason this shouldn't get moved into
AM> I think there is some text missing here.
The text seems to have been one list item in old
section 7.2, dating back to version 11. The list decription was:
"7.2. Effect of Encrypted Transport Headers
When the Transport Header is encrypted, it prevents the following
mobile network features from operating:
<and then a list of many items> "
I suggest to delete this text, but...
Kathleen - if you delete this section, Please leave the section header
marked "Blank - to be deleted" to keep the section numbering as-is,
and the diffs/comments will still correlate easily. Thanks!
2.2.4. Performance-enhancing Proxies
Due to the characteristics of the mobile link, performance-enhancing
TCP proxies may perform local retransmission at the mobile edge. In
TCP, duplicated ACKs are detected and potentially concealed when the
proxy retransmits a segment that was lost on the mobile link without
involvement of the far end (see section 2.1.1 of [RFC3135] and
section 3.5 of [I-D.dolson-plus-middlebox-benefits]).
BW> Starting the first paragraph in this way suggests that such use cases are for mobile links only, which is not correct. Performance enhancing proxies of this sort can be used on any long RTT path to improve performance over a constrained uplink.
KM> How about: Performance-enhancing TCP proxies may perform local retransmission at the network edge, this also applies to mobile networks.
This optimization at network edges measurably improves real-time
transmission over long delay Internet paths or networks with large
capacity-variation (such as mobile/cellular networks).
AM> FYI - The folowing sentence was added here in the -14pre version I sent on Nov 19:
However, such optimizations can also cause problems with performance,
for example if the characteristics of some packet streams begin to vary
significantly from those considered in the proxy design.
This was intended to address one of Mark Nottingham's comments.
An application-type-aware network edge (middlebox) can further
control pacing, limit simultaneous HD videos, or prioritize active
videos against new videos, etc.
KR> Observation: This subsection provides the first really compelling argument I've seen for exposing flow metadata to the path. On long paths, physics gets in the way of tight control feedback loops. If nothing else, this should provide motivation for protocol designers and operators to break down the characteristics of different kinds of flows, determine where control points are needed in each of them, and figure out how to implement those.
I think there is this conceit among protocol designers that quality problems can all be solved at the endpoints without any cooperation from path elements; the really killer arguments are examples of where that cannot possibly be the case. ECN is a great example of this, and is a signal explicitly targeted at middleboxes with opt-in by the endpoints: it allows a middlebox to report congestion without dropping packets, which produces measurably better QoS for the user.
KM> Ack, thanks. You're not looking for additional text here, is that right? If so, what are you thinking should be added?
Content replication in caches (for example live video, DRM protected
content) is used to most efficiently utilize the available limited
bandwidth and thereby maximize the user's Quality of Experience
(QoE). Especially in mobile networks, duplicating every stream
through the transit network increases backhaul cost for live TV. The
Enhanced Multimedia Broadcast/Multicast Services (3GPP eMBMS) -
trusted edge proxies facilitate delivering same stream to different
users, using either unicast or multicast depending on channel
conditions to the user.
KR> There are on-going efforts to support multicast inside carrier networks while preserving end-to-end security: AMT, for instance, allows CDNs to deliver a single (potentially encrypted) copy of a live stream to a carrier network over the public internet and for the carrier to then distribute that live stream as efficiently as possible within its own network using multicast.
KM> Text added, thanks.
Alternate approaches such as blind caches [I-D.thomson-http-bc] are
being explored to allow caching of encrypted content; however, they
still need to intercept the end-to-end transport connection.
KM> [s/need to intercept the end-to-end transport connection/require cooperation between the content owners/CDNs and blind caches and fall outside the scope of what is covered in this document/
Content delegation solves a data visibility problem with the delegated cache, the impact remains for the use case where HTTPS encryption limits visibility to offload from congested links.]
KR> This last point isn't strictly speaking true: many proposals (including I believe Martin's) require cooperation between content owners/CDNs and these blind caches. From Martin's draft:
q( This document describes a method for conditionally delegating the
hosting of secure content to the same server. This delegation allows
a client to send a request for an "https" resource via a proxy rather
than insisting on an end-to-end TLS connection. This enables shared
caching for a limited set of "https" resources, as selected by the
server. )
BW> I'm not sure that use cases where there is explicit cooperation between the content provider and the cache are necessarily relevant for this document, since in those cases the cache is an extension of the content provider (by some definition) and the cache will most likely not be inhibited by increased encryption. The more relevant caching case is one meant for network offload on the receiver side where there is no explicit cooperation between the content provider and the cache. That's the case where the use of HTTPS inhibits the cache's ability to offload from congested links. IOW, content delegation solves a data visibility problem with the delegated cache; it does not solve a problem introduced to the cache through the use of encryption.
2.2.6. Content Compression
In addition to caching, various applications exist to provide data
compression in order to conserve the life of the user's mobile data
plan and optimize delivery over the mobile link. The compression
proxy access can be built into a specific user level application,
such as a browser, or it can be available to all applications using a
system level application. The primary method is for the mobile
application to connect to a centralized server as a proxy, with the
data channel between the client application and the server using
compression to minimize bandwidth utilization. The effectiveness of
such systems depends on the server having access to unencrypted data
flows.
KR> Observation: given the side channels exposed by data compression that is blind to content, the inability to compress arbitrary payloads is likely to be regarded as a feature of encryption. (Though I recognize this is a catalog, not an endorsement.) Furthermore, in most cases eliminating compression is still 2-competitive with compression, so I'm not sure it's a really compelling use-case.
BW> Per-object content compression might not be a compelling use case here. Aggregated data stream content compressions that spans objects and data sources is compelling, though. If there is a network element close to the receiver that sees all content destined for the receiver and can treat it all as part of a unified compression scheme (e.g., through the use of a shared segment store) will often be much more effective at providing data off-load.
KM> Thanks, we'll add this text (modified) to make those helpful points clear.
How about:
Aggregated data stream content compression that spans objects and data sources that can be treated as part of a unified compression scheme (e.g., through the use of a shared segment store) is often effective at providing data offload when there is a network element close to the receiver that has access to see all the content.
Another form of content filtering is called parental control, where
some users are deliberately denied access to age-sensitive content as
a feature to the service subscriber. Some sites involve a mixture of
universal and age-sensitive content and filtering software. In these
cases, more granular (application layer) metadata may be used to
analyze and block traffic. Methods that accessed cleartext
application-layer metadata no longer work when sessions are
encrypted. This type of granular filtering could occur at the
endpoint. However, the lack of ability to efficiently manage
endpoints as a service reduces providers' ability to offer parental
control.
KR> It might be worth discussing the typical opt-in strategy for these things in the presence of TLS, adding a new intercept CA to willing clients, which has the downside that it potentially exposes every https connection to an active MitM.
BW> +1
KM> OK, we hadn't done that before since the option doesn't change, but you make a good point, so I'll add in text. Thanks.
I added the following:
This method is also used by other types of network providers enabling
traffic inspection, but not modification.</t>
<t>Content filtering via a proxy can also utilize an intercepting
certificate where the client's session is terminated at the proxy
enabling for cleartext inspection of the traffic. A new session
is created from the intercepting device to the client's
destination, this is an opt-in strategy for the client. Changes to
TLSv1.3 do not impact this more invasive method of interception, where
this has the potential to expose every HTTPS session to an active
man in the middle (MitM). </t>
KR> Random comment: especially with respect to government content filtering, I'm worried that the IETF's current approach of playing chicken with regulators on end-to-end encryption is going to result in normalization of intercept CAs, which will be strictly worse than a compromise solution in which a subset of traffic can be inspected (but not modified) with the user's knowledge and consent (e.g., distinct optics in the browser). I wouldn't like either outcome, frankly, but it would be nice if we had a game plan for what to do for user privacy if intercept CAs become a requirement for using the web in large parts of the world (something we might be one "crisis" away from), and an honest evaluation of the alternatives. Fundamentally, I don't like it when discussion gets shut down because people want to bury their heads in the sand in the name of ideology.</rant>
BW> +1. I also note that this concern applies to some of the other performance related use cases too.
KM> I think the real argument here is a control one between the application and management folks and not security/privacy even though that's what is often discussed. This is all about control.
In addition, mobile network operator often sell tariffs that allow
free-data access to certain sites, known as 'zero rating'. A session
to visit such a site incurs no additional cost or data usage to the
user. This feature is impacted if encryption hides the details of
the content domain from the network.
KR> There's the related issue that zero-rating by-implementation typically applies only to direct connections to a particular endpoint (e.g., by IP): if a user accidentally tunnels traffic from Spotify through a corporate VPN, that traffic won't be zero-rated, encrypted tunnel or not. (This goes back to the taxonomy of metadata layers comment I made near the top.) Carriers aren't going to trust e.g., a Host header for zero-rating, because that provides a simple way to tunnel traffic for free: consequently, determination of zero-rating will always involve some hard-to-impersonate credential, like an IP address or server certificate in the public trust web.
KM> Not sure what to add here, any ideas, AL?
When RTSP stream content is encrypted, the 5-tuple information within
the payload is not visible to these ALG implementations, and
therefore they cannot provision their associated middelboxes with
that information.
KR> I would argue that this is a protocol design issue. This was originally a problem with firewalls and NATs, with content inspection as a hack to work around the protocol/network impedance mismatch. I'm not the only one who would argue the right solution today is to design protocols to not require linkage across connections by middleboxes that do basic filtering.
KM> I think we are in agreement here for solution direction, but the document specificly tries to avoid solutions. This example has been raised in the IESG by Warren and the apps side hadn't considered his view of it previously. It would be good for protocols to have these considerations in their designs, they were mostly thinking it didn't matter and were end-to-end. But poor video streaming sessions are an issue. Not sure we should add any text here???
2.3.4. HTTP Header Insertion
Some mobile carriers use HTTP header insertion (see section 3.2.1 of
[RFC7230]) to provide information about their customers to third
parties or to their own internal systems [Enrich]. Third parties use
the inserted information for analytics, customization, advertising,
to bill the customer, or to selectively allow or block content. HTTP
header insertion is also used to pass information internally between
a mobile service provider's sub-systems, thus keeping the internal
systems loosely coupled. When HTTP connections are encrypted, mobile
network service providers cannot insert headers to accomplish the
functions above.
KR> See my first comment re: compression. I'm dithering on how best to present these cases that are going to trigger some folks. ;-)
KM> Yes, this one is a hot button. For the compression one, I clarified with added text and a use case that Brandon provided.
3.1. Management Access Security
...
Application service providers, by their very nature, control the
application endpoint. As such, much of the information gleaned from
sessions are still available on that endpoint. However, when a gap
exists in the application's logging and debugging capabilities, this
has led the application service provider to access data-in-transport
for monitoring and debugging.
BW> How is DLP part of the management access discussion? It seems like a separate use case to me. The above two paragraphs seem out of place in this section.
KM> Good point, I moved this to the SP Content monitoring of Applications subsection and added a bullet for DLP.
Overlay networks (e.g. VXLAN, Geneve, etc.) may be used to indicate
desired isolation, but this is not sufficient to prevent deliberate
attacks that are aware of the use of the overlay network. It is
possible to use an overlay header in combination with IPsec, but this
adds the requirement for authentication infrastructure and may reduce
packet transfer performance. Additional extension mechanisms to
provide integrity and/or privacy protections are being investigated
for overlay encapsulations. Section 7 of [RFC7348] describes some of
the security issues possible when deploying VXLAN on Layer 2
networks. Rogue endpoints can join the multicast groups that carry
broadcast traffic, for example.
BW> I'm a little confused about the overall point of this section. I think that it might be "Hosted environment sometimes use content inspection to differentiate between management traffic and service traffic." but I don't think this point is very clearly stated. Or is there a different central point?
KM> It was supposed to be management access, but text got added and the DLP doesn't fit, so that has been moved. I can reach out to Alia who contributed the VXLAN text to work that better into the intent of this section as I can see your point on lack of flow.
Data center operators may also maintain packet recordings in order to
be able to investigate attacks, breach of internal processes, etc.
In some industries, organizations may be legally required to maintain
such information for compliance purposes. Investigations of this
KR> I think you'll get a "[citation needed]" from folks on the TLS mailing list.
KM> I suspect this is one you have that recorded text, you have to maintain it for chain of custody with investigation handling. I'll have to figure out if there is anything that would require the capture, I suspect not, but could be wrong.
3.2. Hosted Applications
Organizations are increasingly using hosted applications rather than
in-house solutions that require maintenance of equipment and
software. Examples include Enterprise Resource Planning (ERP)
solutions, payroll service, time and attendance, travel and expense
reporting among others. Organizations may require some level of
management access to these hosted applications and will typically
require session encryption or a dedicated channel for this activity.
KR> I'm not sure how encryption of the management session is relevant to this doc. The way I've framed this document in my mind is "What information from flows is being used for network management by entities other than the endpoints, who have a need-to-know for the cleartext payload?", where "management" includes things like compliance and satisfying regulatory requirements.
KM> Leaving as-is for now. While there is no impact since these sessions are already encrypted, it is one more connection point to using encryption successfully?
Al?
3.2.2. Mail Service Providers
...
STARTTLS ought have zero effect on anti-SPAM efforts for SMTP
traffic. Anti-SPAM services could easily be performed on an SMTP
gateway, eliminating the need for TLS decryption services. The
impact to Anti-SPAM service providers should be limited to a change
in tools, where middle boxes were deployed to perform these
functions.
KR> Here you're discussing a potential change to the operational technique, which doesn't match the rest of the subsections.
KM> You're right. This text came in from Stephen when he was the sonsoring AD. The point is valid, but doen't fit exactly, but I'm hesitant to remove it.
3.3. Data Storage
BW> I'm having trouble with the Data Storage section at large. For the most part, it seems to be describing use cases for the deployment of encryption, as opposed to things that people are doing today that would be harder if the network flows were encrypted.
KM> Flow were encrypted in recent years per customer demand and the engineers worked to ensure monitoring was possible, improving logging, etc. I'll try to make that more clear in the introduction. This is a positive example, at least it was meant that way and for completeness.
3.3.1. Object-level Encryption
KR> End-to-end or object encryption seems like a better description here: host-level encryption implies that anything on the originating or target host has access to it, when (for instance) it could be encrypted to a TPM resident key or an SGX enclave key. The distinguishing question seems to be "Do middleboxes/intermediate nodes have access to the cleartext?"
KM> You're right. The term host-level is an internal one and shouldn't be in the document. I'll fix that, thanks. No, middleboxes don't have access in the EMC use cases at least. This is specific to object level encryption.
3.3.1.1. Monitoring for Hosted Storage
BW> This is one of the few subsections that seems to be describing an method in current use that will be made more difficult if all the data flows are encrypted.
3.3.2.1. Monitoring Session Flows for DAR Solutions
Monitoring for transport of data to storage platforms, where object
level encryption is performed close to or on the storage platform are
similar to those described in the section on Monitoring for Hosted
Storage. The primary difference for these solutions is the possible
exposure of sensitive information, which could include privacy
related data, financial information, or intellectual property if
session encryption via TLS is not deployed. Session encryption is
typically used with these solutions, but that decision would be based
on a risk assessment.
BW> What would be a monitoring use case in current use that will be made more difficult due to the increased use of encryption? The previous sentence seems to be suggesting that session encryption is already prevalent, so I would think that people deploying DAR solutions have already come up with different monitoring approaches.
KM> The storage engineers improved monitoring capabilities through logging, etc. This was a requirement, so they made it happen with full agreement on direction.
There are use cases where DAR or disk-level
encryption is required. Examples include preventing exposure of data
if physical disks are stolen or lost.
KR> I don't see these last two sentences are relevant, as they have nothing to do with the network flows.
KM> I'm happy to remove. DO they help a reader who is not familiar with the technology to understand the layers of encryption used at all or is it better to remove the sentence?
In the case where TLS is in
use, monitoring and the exposure of data is limited to a 5-tuple.
KR> This is an example of implicit use of the taxonomy I referred to earlier. I feel like this should be used systematically throughout the survey (i.e., what metadata does your technique rely on, and why?).
KM> That was the intent, but we didn't get that information from enough contributors. I provided the above in a very early draft. We'll have to go back through and see if that's possible at this point. I know it would be helpful.
3.3.3.1. Monitoring Of IPSec for Data Replication Services
Monitoring for data replication services are described in this
subsection.
Monitoring of data flows between data centers may be performed for
security and operational purposes and would typically concentrate
more on operational aspects since these flows are essentially virtual
private networks (VPN) between data centers. Operational
considerations include capacity and availability monitoring. The
security monitoring may be to detect anomalies in the data flows,
similar to what was described in the "Monitoring for Hosted Storage
Section". If IPsec tunnel mode is in use, monitoring is limited to a
2-tuple, or with transport mode, a 5-tuple.
BW> What monitoring is done when encryption is not in use? Is there something being done that requires access to higher layer protocols? If this traffic is already most-often encrypted, then maybe the use case isn't relevant for this document.
KM> It's just here for completeness
Security monitoring in the enterprise may also be performed at the
endpoint with numerous current solutions that mitigate the same
problems as some of the above mentioned solutions. Since the
software agents operate on the device, they are able to monitor
traffic before it is encrypted, monitor for behavior changes, and
lock down devices to use only the expected set of applications.
Session encryption does not affect these solutions. Some might argue
that scaling is an issue in the enterprise, but some large
enterprises have used these tools effectively.
KR> This is another example of mixing proposed solutions in among the problem statement. I would argue for a clear separation, which may mean that this document needs to have a single-minded focus on "here are the problems and here's how enterprises currently address them."
BW> Also, enterprises increasingly allow BYOD programs for their employees, and such programs make it more difficult to ensure that adequate endpoint-based defenses are active. This is especially true when the area of risk in question is the above #5 "track misuse and abuse by employees". Note too that endpoint-based defenses can be less effective when the device is already compromised, in which case detection of the compromised device and effective remediation can be made more effective through the additional use of an on-path element.
KM> [made some subsequent edits to this section]
4.1.3.2. TCP Pipelining/Session Multiplexing
TCP Pipelining/Session Multiplexing used mainly by middle boxes today
allow for multiple end user sessions to share the same TCP
connection. [This rasises several points of interest with an
increased use of encryption. TCP session multiplexing should still
be possible when TLS or TCPcrypt is in use since the TCP header
information is exposed leaving the 5-tuple accessible. The use
TCP session multiplexing of an IP layer encyption, e.g. IPsec,
that only exposes a 2-tuple would not be possible. Troubleshooting
capabilities with encrypted sessions from the middlebox may limit
troubleshooting to the use of logs from the end points performing
the TCP multiplexing or from the middleboxes prior to any
additional encryption that may be added to tunnel the TCP multiplexed
traffic.]
KM> I deleted the following, after adding in some of this content above where I think it makes more sense:
[Today's network troubleshooter often relies upon session
decryption to tell which packet belongs to which end user, since the
logs are currently inadequate for the analysis performed.]
KR> It's not clear to me how these two sentences are related.
KM> Good point. I think deleting the second sentence is important as it's a general point and not specific to this section. E2E with FS should prevent the use of this multiplexing as I read it, so that is an impact that should be fine to document on it's own. [edits subsequently above; maybe best viewed from the etherpad]
Increased use of HTTP/2 will likely further increase the prevalence
of session multiplexing, both on the Internet and in the private data
center.[ HTTP pipelining requires both the client and server to participate
and visibilty of packets once encrypted will hide the use of HTTP pipelining for any monitoring that takes place outside of the endpoint or proxy solution. Visibility for middleboxes includes anything exposed by TLS and the 5-tuple.
Note: left the text like this as SNI encryption will be optional, so it may also be exposed and this could vary by version used.]
KR> And further complicate analysis of cleartext payloads from individual packets.
4.2. Techniques for Monitoring Internet Session Traffic
...
(PII), or personal health information (PHI). Various techniques are
used to intercept HTTP/TLS sessions for DLP and other purposes, and
are described in "Summarizing Known Attacks on TLS and DTLS"
[RFC7457]. Note: many corporate policies allow access to personal
financial and other sites for users without interception. Another
option is to terminate a TLS session prior to the point where
monitoring is performed.
KR> The last two sentences seem like a non-sequitur.
KM> Leaving as-is for now.
5.4. Botnets
Botnet detection and mitigation is complex and may involve hundreds
or thousands of hosts with numerous Command and Control (C&C)
servers. The techniques and data used to monitor and detect each may
vary. Connections to C&C servers are typically encrypted, therefore
a move to an increasingly encrypted Internet may not affect the
detection and sharing methods used.
KR> This is one of a general category of traffic that is intentionally protected from interference by the application.
BW> For that reason, this one almost seems like a counter example ... botnets encrypt to evade many of the earlier described methods, therefore many of the earlier described methods are already inadequate. On the other hand, maybe the point is that increased use of encryption for botnet C&C demands new methods to handle many of the previously described use cases.
KM> Well, from an incident investigation standpoint, many advanced operators have found detection methods that don't hinder their capabilities when traffic is encrypted. Maybe I need to make this text more clear as I think I may have written it, not positive.
I think I'm going to leave the text alone as I think it reads as a positive example, one which techniques could be applied to other areas. The DoS section talks a little about use of fingerprinting traffic already, which is one common technique.
5.7. Further work
Although incident response work will continue, new methods to prevent
system compromise through security automation and continuous
monitoring [SACM] may provide alternate approaches where system
security is maintained as a preventative measure.
KR> Not clear how the unknowns relate to the purpose of this document. Being sarcastic for a minute, I'm interpreting this as "Any cleartext metadata just *might* be used in the future for some kind of enterprise security monitoring!"
KM> Hmm, it's meant to say endpoints (which you control) should be used and technology like what is expected out of SACM will help with automating this. We are open to text suggestions.
6.1. IP Flow Information Export
...
The collection of IPFIX data itself, of course, provides a point of
centralization for potentially business- and privacy-critical
information. The IPFIX File Format specification [RFC5655]
recommends encryption for this data at rest, and the IP Flow
Anonymization specification [RFC6235] defines a metadata format for
describing the anonymization functions applied to an IPFIX dataset,
if anonymization is employed for data sharing of IPFIX information
between enterprises or network operators.
KR> I don't understand how IPFIX relates to the purpose of this document. Each of the IEs should be covered by one of the functional categories described earlier in the document.
KM> Leaving as-is for now. Was added per Benoit and Brian Trammell for completeness on network management and encryption.
6.4. Content Length, BitRate and Pacing
Although block ciphers utilise padding, this makes a
negligible difference. Bitrate and pacing are generally application
specific, and do not change much when the content is encrypted.
Multiplexed formats (such as HTTP/2 and QUIC) may however incorporate
several application streams over one connection, which makes the
bitrate/pacing no longer application-specific.
KR> Are the four items listed here all the application-level flow information available to the network for encrypted flows? I think I need some more comprehensive top-down analysis to be confident of that. It seems like this could be folded into the metadata taxonomy, if you decide to go that route.
This section is also weird in that it doesn't describe a problem caused by encrypted flows. There's a lack of parallelism in structure between the other sections and this one.
KM> Good point, but not my area and am open to suggestions. Is it just anothe example where things are ok when encrypted? I think a few of those are a good thing.
7. Impact on Mobility Network Optimizations and New Services
This section considers the effects of transport level encryption on
existing forms of mobile network optimization techniques, as well as
potential new services. The material in this section assumes
familiarity with mobile network concepts, specifications, and
architectures.
KR> Good warning, but the entire section is very specific to a particular set of technologies, while the rest of the document is much more general. This feels like a case of "writing to what you know", which is fine in principle, but it feels out of place in this document to use so much jargon and to discuss metrics in the context of a single technology when many of these KPIs have equivalents outside of 3GPP.
c. Performance-enhancing proxy with low RTT determines the
responsiveness of TCP flow control, and enables faster adaptation
in a delay & capacity varying network due to user mobility. Low
RTT permits use of a smaller send window, which makes the flow
control loop more responsive to changing mobile network
conditions.
KR> Again, as with section 2.2.4, this section provides the clearest and most convincing arguments for the need for middlebox cooperation on flows.