> Date: 2005-02-16 01:14
> From: John C Klensin <john-ietf@xxxxxxx>
First let me say that in principle separating MSAs from MTAs
does not seem unreasonable (on the other hand, I don't believe
the distinction is clear from a protocol view), and that I can't
forsee any huge amount of damage that could arise from RFC 2476
or the draft revision. Nevertheless, I am uneasy about some of
the content and some of the statements about 2476 vs. ESMTP, and
I have some procedural uncertainties about migration to Draft
Standard status given lack of clarity about the distinction
noted above and the unclear state of implementation conformance.
> Bruce, if I understand your concern (I may not), let's back up a
> bit and look at a bit of history and the usage case. ÂThe notion
> of a separate submission protocol arose in part out of a problem
> that became clear when we were doing 2821. ÂThere were a whole
> series of things that we would really prefer that MTAs not do
> --things that violated the "don't mess with the message"
> meta-principle for MTAs but were necessary because clients
> weren't able to supply all of the information needed for a valid
> 2822/822 message, or weren't able to supply it in a form that
> would be security-acceptable. ÂSo, to face the reality of the
> fact that these MTA-fixes were being applied (and were
> necessary), 2821 contains a loophole for submission servers to
> fix messages up.
One bit of uneasiness is that I'm not convinced that there
really is something that SMTP clients can't supply which is
vital to the message format [aside from cases of "the user
is computer-illiterate and hasn't set the time zone" or some
such thing]. As it stands, the items required by the message
format which 2476 permits an MSA to alter are:
1. mandatory From header field
2. mandatory Date field
All other message header fields are optional [RFC 2822], and
tampering with the SMTP envelope does not affect the message
format content (at least not directly; at final delivery the
SMTP envelope return path gets inserted in a Return-Path trace
header field). Now I can understand that administrators might
like to enforce policy w.r.t. use of host names under a domain,
etc., but that's a very different matter from "needed but can't
be supplied" (paraphrasing). If the issue is really about
administrative fiddling with message content to centrally enforce
policy (rather than have policy enacted at the client -- which
certainly does not appear to be impossible), then say so instead
of using purported client inability as a justification. [then
we can move on to discussion of the real issue, ACAP, DHCP
parameters and extensions, etc.]
The two specific mandatory fields listed above aren't particularly
strong cases in favor of message munging; a case can be made that
the From field should be optional (draft-lilly-from-optional),
and the Date field plays absolutely no role in protocol other
than end-to-end, user-to-user information (moreover the semantics
are specified as being the time of message *creation*, not of
message *submission* for transport).
> That loophole is, however, somewhat bad news, since it doesn't
> involve explicit client authorization for message-tampering and
> there is no real way to permit such authorization even if one
> wanted to do it
How to authorize message-tampering via ESMTP in 3 easy steps
(some detail omitted):
1. server offers TAMPER extension in response to EHLO
2. client indicates acceptance of TAMPER option (separate
command or argument; as promised, I have omitted details :-)
or not, as the case may be (if the client does nothing,
tampering is not authorized).
3. server tampers if authorized; does not tamper if not
authorized.
That seems almost too easy; what have I missed (aside from the
details)?
> "Message Submission" ("Submit" below) is
> obviously different: we allowed for extensions to be used with
> Submit that are not specified for SMTP (even though no one has
> done that -- the mechanism has been shown to work with
> extensions that are also specified for SMTP)
I'm a bit baffled by that as I can see no ESMTP extension
specified or referenced by RFC 2476 or the draft which does
not also apply to SMTP. The converse is true (but irrelevant);
ETRN, for example, can be used with SMTP but is forbidden by
RFC 2476 with an MSA.
It's also unclear what "mechanism" you refer to...
It seems like the hypothetical "TAMPER" extension (if not
permitted for non-submission SMTP) would do what is described
above (and shortly below) as desired...
> and the mere fact
> of using the Submit protocol says "this is a submission agent,
> not a general MTA, and hence gets to use those features".
The description as a separate protocol is also a bit baffling,
as 2476 and the draft state "The protocol used is ESMTP", i.e.
the same as specified in 2821. Moreover, if I connect to an
MSA on port 25 (which is explicitly permitted), there is
nothing in the greeting reply code or EHLO extended response
that indicates that a server is an MSA (conversely, as noted
above, if ETRN is offered, clearly the server is not (supposed
to be) an MSA). So where exactly is the supposed indication
in the protocol that "says 'this is a submission agent'"?
What I seem to be reading is that there really isn't any
indication in the protocol; the administrator of an SMTP
server waves an imaginary magic wand and utters (very quietly)
the incantation "this is a submission agent", unbeknownst to
the client or the message originator, and if that happens,
then any of the message munging that would be a no-no without
the incantation is now magically OK. Please tell me what
subtlety I missed that makes that not so. [No disrespect to
you, your co-author, or the submit group intended -- I'm
really baffled by the apparent magic (or smoke-and-mirrors
if you prefer) that seems to be implied.]
> But, if it is run on port 25, the question
> immediately arises as to how you tell it from SMTP, since the
> two are deliberately quite similar in their handshakes.
"similar" (as opposed to "identical" implies some significant
difference. I can find none to point to that specifically
identifies submission. Even with no provision for client
control (aside from QUIT), there could be a difference
simply by defining an EHLO keyword (let's see 1869 - letters,
digits, hyphens, no maximum length) e.g.
this-is-a-submission-agent-not-a-general-MTA-and-hence-gets-to-mangle-content
and requiring its use for submission agents and prohibiting it
for MTAs. However, I see nothing even as simple as that (i.e.
just a machine-parseable announcement w/o corresponding feedback)
that truly defines "submit" as a (trivially) *different*
protocol from ESMTP. I submit (no pun intended) that lacking
such a differentiating protocol difference claims that the
protocol is different are misguided. Mind you, with such a
trivial difference, I would call it a minor variant on a
single protocol rather than a different protocol.
> The
> answer is "administrative or policy decision, reflected in
> configurations".
But that doesn't tell *me* (as the client connecting to that
server) whether it is "ESMTP" or "Submit" (which we're told *is*
ESMTP). Sure, the administrator of the server (supposedly) knows,
but so what? If I (as the client) can't tell which (variant of)
protocol the server is running, then I can't make sense of the
response codes, especially where the semantics differ (as with
5.6.2).
> > My understanding is that implementations are permitted to have
> > configuration options that allow sites to deploy them in ways
> > that violate the RFCs, as long as they can be operated in a
> > conforming manner. ÂFor example, it is not uncommon for POP
> > implementations to have options to have timeouts shorter than
> > mandated, or to enter UPDATE state on abort.
>
> This has always been the rule, at least since we decided that
> specifications, to advance to draft, must exhibit implementation
> of all of the features.
I don't want to get too bogged down, but I'm concerned about
a specific subset of violations, where (as in this case):
1. there is an RFC 2119 "MUST" or "MUST NOT" provision. Recall that
a 2119 MUST [NOT] is only supposed to be used where a violation
would cause harm or where the feature is necessary for
interoperation (RFC 2119 section 6).
2. implementations can be operated in such a way as to violate
said provision.
Note that I'm not referring to a "SHOULD". My conclusion is that
the implementation in question can be configured in such a way
as to cause harm (e.g. initiate DoS attacks (where "attack" may
include behavior on the part of the implementation which is not
intentional on the part of the administrator) and/or result in
failure of interoperation, possibly including excessive numbers
of retransmissions, etc.), or that there is an error in the
specification (inappropriate use of 2119 imperatives). I can't
imagine that we'd want to encourage the former, and in the latter
case, the error in the specification should be corrected.
> We don't do code review to verify
> conformance and spot possibly-non-conforming capabilities.
There is supposed to be "testing of the interoperation of [...]
implementations" (RFC 2026 section 4.1.2). Code review is not
specifically required, and might not be possible for proprietary
implementations. Nor is it necessary for interoperation testing;
just see how the implementations behave. Black-box testing is
hardly a novel concept.
> The
> rule is that the implementations must be able to demonstrate
> that they can be configured in a way that conforms to the spec.
The full specification, presumably, not just parts of it.
> > From my understanding of the requirements, there are multiple
> > implementations which comply. ÂThis is based on two points:
> > (1) implementations can be compliant even if they have options
> > which permit non-conformance; and (2) there must be at least
> > two implementations for each feature, but it is not required
> > that any individual implementation meet every SHOULD. ÂI
> > believe Message Submission meets the criteria on the basis of
> > either of these points; the fact that it meets it on both is
> > nice but not required.
>
> Under other circumstances, I might actually quibble about (2),
> but (1) is the important point.
As I mentioned, I don't want to get bogged down on the first point;
I hope I have made my concerns clear above.
The second point is also important, and relates to the rationale
behind the RFC 2026 sect. 4.1.2 "requirement for at least two
independent and interoperable implementations [which] applies to
all of the options and features of the specification". Now I
can think of two reasons for such a requirement:
1. to ensure interoperability, which is the point of having
standards in the first place. In support of the interpretation
I note that 2026 goes on to state that a Draft standard is
considered to be a final specification and that it is reasonable
to deploy (conforming) implementations in disruption-sensitive
environments. One certainly would not want to deploy
implementations in a disruption-sensitive environment if
serious (i.e. warranting RFC 2119 imperatives) interoperability
problems were known, nor would one wish to encourage
implementation by vendors if the specification were unstable.
2. to winnow out unnecessary or unused features, which is implied
by the remainder of the second paragraph of 2026 4.1.2.
Again, as before, my primary concern is with mandatory features
(MUST/MUST NOT) rather than with less consequential matters.
The interoperability report lists many features and many
implementations; many feature/implementation pairs have unknown
status. In the interest of brevity, I'll consider a hypothetical
feature set of 3 features (numbered 1-3) and three implementations
indicated by letters A, B, and C. Implementation A supports
features 1 and 3, B supports 2 and 3, C supports 1 and 2. While
one can list two implementations in this hypothetical example which
support each of the features separately, one cannot list any two
implementations which are interoperable with respect to "all of
the options and features". Implementation A will not interoperate
with implementation B because the former lacks support for feature
2 and the latter lacks support for feature 1. Likewise for (lack
of) interoperability between A and C and between B and C. What
are the implications for the two possible reasons for the 4.1.2
requirements enumerated above?
Clearly interoperability doesn't exist w.r.t. all of the features.
It's not clear what the implications are w.r.t. necessity or
utility of features; clearly one implementer can be found who
believes that any given feature is either unnecessary or otherwise
not worth implementing. However, one can find at least two
implementers who support any individual feature. Most important,
though, is the fact that none of the implementers in this
hypothetical example support "all of the options and features of
the specification"; consequently one cannot enumerate two
implementations which do so.
Returning to the case at hand, the question w.r.t. the 2026 4.1.2
implementation requirement, as I see it, is: "can one list two
implementations which support all of the mandatory features of the
specification?". If there are more than two which support all of
the features, that's a bonus. But listing a dozen implementations,
each of which supports a couple of features (but not all features)
doesn't answer that question, and doesn't fill me with confidence
that there are any fully interoperable implementations (w.r.t. all
features) or that unnecessary features have been trimmed from the
specification.
[re. extended response code discrepancies]
> To the extent to which this is an issue, it is an issue with RFC
> 3463 and not with this draft.
I disagree. RFC 3463 (nee 1893) defined the extended response
codes and was specified as a reference. There are defined
response codes to cover the situation(s) addressed by 2476
and the draft (though those are not used), and at least one
code is used in a way which conflicts with the (original)
definition.
> A real problem would arise only
> in the "both protocols, same machine, port 25" case described
> above.
I believe it also applies to the case "I have no way to determine
whether or not the server is a 'submission' server; it responded
with 5.6.2 -- is that supposed to mean 'conversion required and
prohibited' or 'bad domain or address', and what do I (as the
client) present to the user (who might not understand English)?".
> But I agree with Randy -- the right action is to delete
> the specific codes from this specification and leave that to
> 1893, 3463, and their successors.
I believe there's an identity crisis that needs to be resolved;
if the protocol is just some minor variant of ESMTP, as implied
by the statement in section 3.1, then
a. the response codes should be consistent with ESMTP except in
specific enumerated cases where some difference is warranted;
it may suffice to state explicitly that codes are the same
except for enumerated differences (giving 2821 as a normative
reference).
b. we should refer to it as a variant of ESMTP rather than as some
wholly distinct protocol
Conversely, if it is in fact intended to be a wholly distinct
protocol from ESMTP, section 3.1 should be reworded to accurately
portray that and specific responses should be enumerated for each
command -- including positive responses and temporary failure
responses.
I don't think sweeping the problem under the rug by simply not
mentioning response codes is appropriate for a document that is
intended to be "considered to be a final specification" and
"well-understood".
> For example, assume that one has a
> submission client on a private network using 1918 space.
OK (I'm typing on one).
> It
> submits the message to an MSA which lies at the NAT boundary
> between that network and the public Internet.
Nope. The MSA in this case is on the inside of the firewall
which provides NAT. I wouldn't want to run an SMTP server
(submit or otherwise) *on* a firewall given the history of
security vulnerabilities in various implementations of SMTP,
not to mention the possibilities for DoS attacks. Nor do I
trust firewall appliance vendors to implement plain vanilla
SMTP correctly (I know for a fact from experience that some
fail to do so), much less a particular variant of SMTP. Because
the MSA is on the inside, it also has a 1918 IP address (in
my specific case, it happens to be the same machine as the
client, which it knows as 127.0.0.1).
Now I suppose there might be another MSA on the outside of
the firewall (the next step uses ESMTP with AUTH on port 25,
but I have have no way of knowing what incantations its
administrator might have muttered :-), but neither 2476 nor
the draft mentions anything about multiple MSAs in the transport
path. [Patrik Faltstrom's diagram at
http://www.ripe.net/ripe/meetings/ripe-47/mailflows.pdf
might be illustrative of how complex such paths can be]
> To meet the
> requirements of 2821, the MSA (or submission MTA) should modify
> or supplement addresses and internal names so that they reflect
> public names and addresses.
In the SMTP envelope? Sure; no problem. I wouldn't expect
domain literals (the 1918 issue) there in the first place,
though.
In the message content? Big problems.
1. Verboten for Received fields already in the message. You
want to treat Received fields specially? Now you need a
full-blown message parser.
2. No need for change to address fields (message is unreplyable
if mailbox(es) in from or address(es) in Reply-To are
broken, and the client has a better chance of getting those
right (closer to the user) than some remote MSA. The Sender
field is irrelevant, and in any case need not be replyable.
To and Cc fields likewise are best set by the client and
left untouched by transport. Modification would also be a
layer violation; the header fields are specifically for
user-to-user, end-to-end communication, not for transport.
About the only thing that might be OK is handling Bcc
fields, but that really should be handled by the client in
conjunction with the SMTP "envelope" transaction(s).
3. What happens when somebody invents a new address field?
The client that generates a field clearly knows about it.
But who is going to update all of the SMTP (oops, "submit")
servers around the world to start mangling that field, too?
4. What happens when somebody (Hi, Carl) invents a field that
uses domain-name-like constructs but in an odd arrangement
(RFC 3865)?
5. The only RFC 2821 requirement[s] that I see that is relevant
specifically applies only to gateways...
> That moves it into gateway
> territory,
Circular argument detected; "I modify the message" therefore
"I am a gateway" therefore "RFC 2821 permits/requires me to modify
the message". I don't buy it.
> and 2821 was written to permit that case (doesn't
> make that I, or the DRUMS WG, liked the case very much, but it
> reflects reality).
The dislike (speaking for myself) has to do with unnecessary
and therfore undesirable layering violations. A *real* gateway,
where there is a drastic change in network conventions, such that
necessitates (reversible) transformations for end-to-end
functionality to operate, is another matter [even so, there's
still those nasty "what happens when..." #3 & #4 above].
> >>> ÂIt is not the intent of either this draft or RFC 2476 to
> >>> Âsay that MSAs are always gateways; rather, the intent in
> >>> Âboth is to recognize the reality that organizations
> >>> Âsometimes see a need to examine and potentially modify
> >>> Âmessages submitted using their servers,
"Desire" != "need". (truth in advertising, please) If there
were no layering violation involved, I would ask what
interoperability or harm-prevention issues constitute such
a "need". Given the layering violation, I would expect any
such issues to be very clear and subject to intense scrutiny.
[hint: "we want to hide host names at the edge because we're
too lazy to properly configure clients and we truly believe
that obscurity equals security" doesn't pass the sniff test]
At minimum, the consequences in terms of network damage or
serious interoperability problems associated with not mangling
content would have to be quite serious to justify the layering
violation of fiddling around with ser-level content.
> >>> Âand to make a clear
> >>> Âdistinction between this case, and the
> >>> Âexamination/modification of messages being relayed.
Given the inability for a client to detect and/or control
the situation on behalf of the user initiating the message
(as detailed above), I don't see that there is "a clear
distinction". The message is being relayed by exactly the
same store-and-forward mechanism used by MTAs, and provides
exactly the same (non-)information and (lack of) control
to/by the message originator regarding examination/modification.
> Bruce, you are assuming, I think, that every MSA must be
> conformant to the MTA requirements of 2821 (gateway or not).
> That was never intended to be the case. ÂIndeed, that is part of
> the reason why this is a separate protocol (i.e., the "clear
> distinction" to which Randy refers to above).
See my "identity crisis" remarks above.
_______________________________________________
Ietf@xxxxxxxx
https://www1.ietf.org/mailman/listinfo/ietf