Donald, Thanks for the obviously careful review of a long and complicated document. Because the response below proposes several changes to the document and to encourage them to check my explanations, I've explicitly copied the WG mailing list. Before I go further, I want to note that many of my comments below are personal opinions or editor's explanations. I've tried to reflect WG positions when I am confident I know what they are but I have identified explanations and comments when I don't know explicitly as personal opinions. A key part of your review is, I think, based partially on a significant misunderstanding due to information you almost certainly did not have and that the community, and the IESG, should have in considering the review. If nothing else, that background is important to understanding some of my inline comments. My fault or that of the WG Chairs -- some mention of this probably should have been suggested to Murray for inclusion in the Last Call announcement: When EMAILCORE was chartered, and even going back to the first draft charter in September 2020, the explicit plan was for it to produce three documents: The first two were to be Internet Standard replacements for RFC 5321 and 5322. They were constrained, by both the charter and by the rules for Internet Standard, to the scope and coverage of their predecessors. The third document (which is mentioned at the end of Section 1.2 and that is still in progress in the WG as draft-ietf-emailcore-as) was to be an Applicability Statement that addressed other work and related issues in the email world. The result of those constraints and decisions was that a discussion of, e.g., DMARK and DKIM was out of scope for the current document and very much in scope for that A/S. There are additional issues about they, and related work, should be handled there, more on that inline below. It is probably worth my saying here what Section 1.2 of the current document implies and that I have said explicitly to the WG on more than one occasion. I'm in the odd position of being author/editor of a document that I have really come to dislike even while feeling that we've gotten ourselves into a situation where all of the plausible alternatives are worse. That does not mean I'm not open to carefully thought out clarifications or other changes but, as explained below, I think we need to tread carefully. If the IESG's conclusion is that this document (or some small variation on it) should be approved and published while many in the community hold their noses about various part of it, I will certainly be among the nose-holders. On to specifics. --On Friday, October 11, 2024 23:56 -0400 Donald Eastlake <d3e3e3@xxxxxxxxx> wrote: > I have reviewed this document as part of the security directorate's > ongoing effort to review all IETF documents being processed by the > IESG. These comments were written primarily for the benefit of the > security area directors. Document editors and WG chairs should > treat these comments just like any other last call comments. > > The summary of the review is Has Issues. > > This document is an obsoleting revision of RFC 5321 on the Simple > Mail Transfer Protocol (SMTP), incorporating Errata, etc. It is a > mix of specification, operational, and historic information. > Changes from RFC 5321 are summarized in Appendix F. I did not > review the Acknowledgements or References sections or Appendices C > or E. > > > Security > -------- > > This document has an extensive Security Considerations section > which covers most points. But I have the following suggestions: > > Section 7: Could be clearer on what the threat model is. I agree, but going much further than the current text goes would probably take us down a rather deep rabbit hole (or several of them). The WG therefore decide to kick the rabbits down the road to the A/S. The question that obviously raises is whether the Last Call on this document should be held until some of it can be reviewed in parallel with a version of the A/S that the WG thinks is ready for publication. I would prefer to avoid that because it would create a near-invitation for going back and forth about what should be in the A/S and what details or explicit references should be slipped back and forth between the A/S and this document. FWIW, the WG spent many months on similar issues involving the relationship between the current document and draft-emailcore-rfc5322bis, which has gone through IETF LC and has been approved by the IESG. Holding this document (referred to below as rfc5321bis) would also be a good argument for holding rfc5322bis. I think it is safe to say that neither the WG nor our long-suffering AD would be happy about such holds. My personal concern is that it might result in further attrition among those active in the WG, possibly reducing the breath of perspectives and quality of the work, especially on the A/S, and consequently considerably extend the time it will take to get all three documents out. There is a specific comment on the threat model below. Speaking personally, but I hope for the WG and its participants, I would very much welcome your participation, and that of others who are traditionally more focused on the security issues, in the development of those parts of the A/S (again, see below). > Section 7.1, particularly re 1st paragraph: Shouldn't there be some > mention of DMARC (RFC 7489) and DKIM (RFC 6376) and perhaps other > things that don't come to my mind immediately? No. See above. And those two documents in particular have a difficult relationship with part of the threat model. Some people would argue that most (or nearly all) of the Internet's email goes directly from the administrative domain of the sender to the administrative domain of the recipient. A few would even argue that the numbers are large enough that the hop-by-hop model --one has been part of mail transport since about the time the community gave up to transporting mail over FTP -- should be deprecated. Perhaps unfortunately, counterexamples abound because even simple mail forwarding and list expansion introduce intermediary systems. Those intermediary systems -- whether mail relays or something more complicated -- have the potential to alter any part of the mail headers they receive. Whether such alterations are deliberate or accidental results of other processing, there are many examples of, e.g., DKIM/DMARC not being trustworthy beyond the most recent hop and, in a multihop environment, therefore fairly useless in authenticating the sender or the identity of the received message to the original one. The problem runs even deeper: running SMTP over TLS provides encryption protection in transit between hosts but, if any intermediate host is compromised, the message is (at least as far as any protection from TLS) sitting on that host in the clear. I hope I'm not just passing the buck here, but I believe that documentation of these threats are more properly something that should be done in Security Area documents than somehow discussed in depth as part of SMTP or even in the proposed A/S. > Section 7.1, particularly re 2nd paragraph: I think it would be > useful to mention something like the following: > 1. Transport authentication between SMTP systems could improve the > authenticity of the Received line added by a server but does not > protect those lines against modification, in violation of this > document, by subsequent SMTP systems. > 2. Nation State intelligence agencies and others on occasion, have > been known, even in the case of extremely high bandwidth traffic > mixing many streams, for example between large data centers, as > well as lower bandwidth connections, to capture and hold immense > quantities of raw traffic for potential later analysis, so there > may be good reason to encrypt transmissions between SMTP systems. See Section 6 of the draft A/S. I personally don't think that goes far enough and have said so in correspondence within the WG; your suggestions add to that belief or might suggest an additional document. > Non-Security > ------------ > > Section 6.1, 6.2: I do not quite understand saying that something is > mandatory/required and then talking about where it is impossible or > impractical to do that thing. Section 6.1 says, concerning a host > holding an email message, "It MUST NOT lose the message for > frivolous reasons, such as because the host later crashes ...". > "frivolous" does not seem very well defined for a mandatory > implementation requirement. The only examples given later of > non-frivolous reasons seem to be spam or "hostile" (infected?) > messages. Maybe this should be more like "In so far as practical, > messages MUST NOT be accidentally lost. For example, messages and > their processing state should be committed to non-volatile storage > until responsibility is taken for that message by the next host in > the delivery path." or something like that. In Section 6.2, how > does it make sense to say that something is required and > impractical? Yeah. This problem goes back to RFC 2821, i.e., it has been with us since decisions made in the last century. Indeed, that particular text goes back more than another decade: it is copied, almost exactly, from RFC 1123 (and I didn't write that paragraph then either). The ideal solution, at least in or before RFC 2821 was published in 2001, would have been to completely restructure and rewrite this rather complicated combination of previous documents. See Section 1.2 for an explanation of that problem. We could re-think and tune some localized text now, but it is not clear that it would make much difference. Some of the obvious fixes would be substantive (e.g., if something is not really mandatory/required, probably any associated or implicit MUSTs should be changed to SHOULDs and then explained better. This time around, the WG has been very reluctant to make changes like that and has decided to minimize change especially when there was no significant evidence of real harm or serious confusion in the last 16 years (or much longer). Again, and at least in retrospect, I think we should have completely rewritten the thing during DRUMS in 1995-2001. We (the WG at the time) decided to not do that, in part because the risk of introducing errors that would effect existing deployed implementations. Now, even if we wanted to do so, I don't believe that there is either sufficient energy or resources to do that rewrite, nor am I confident that a subsequent review that would catch all discrepancies. The WG very briefly discussed the "full rewrite" option and reached the same conclusion. I admire Rich Salz's effort to produce updated versions of RFC 2026 and 2814 with all of their updates folded in, but the effort has unearthed some tough issues. Those documents are far less complex than SMTP has become and involve "only" IETF procedures and not deployed implementations that affect many millions of Internet users. > Minor > ----- > > Global: RFC 821 is obsolete. It was published 42 years ago and > obsoleted 21 years ago. Is there really a need for references to it > in this document Procedural/nitpicky explanation: Due to a twitch in our vocabulary and how aspects of RFC 2026 have been implemented, "obsoleted" is a tricky term. Yes, RFC 2821 listed RFC 821 as being obsoleted but the RFC index and metadata still lists it as STD 10 and is the only Internet Standard among core email documents (unless you count RFC 6409). I have believed, at least on and off, for a long time that any standards Track or BCP document that is obsoleted should have its status changed to "Obsolete" or something like that. But that is not what the RFC Editor does and, when I have raised this with a few generations of RFC Editors (but probably not to RSWG), they expressed the view that it would require a change to RFC 2026. I could certainly see getting rid of the NCP, NITS, and X.25 appendices (the latter if someone could prove it is really dead worldwide). But even that would require more, possibly error-attracting, fussing with the text than anyone in the WG was inclined to propose. That does suggest one addition to the document. In Section 1.2, the paragraph starting "It obsoletes" should perhaps contain an additional sentence explicitly removing the "Internet Standard" status from RFC 821. If that is needed, draft-emailcore-rfc5322bis should be similarly patched. Do you (and others) think such a fix would be desirable? More substantive answer: the present document contains significant text copied from RFC 821 but without the context being copied. The references to 821 could be removed but only at the expense of rewriting sections to include more of that context. Similar comments might be made about some of the other very old documents being referenced although 821 is the oldest. It is probably the one that is most sensitive to citations vis-a-vs the later ones. > Section 1.2, last paragraph: Future commitments always seem a bit > risky. Suggest replacing "forthcoming" with "planned". Normally, I would agree. But, not only is the A/S well along but describing it simply as "planned" runs directly into the discussion above about the importance of discussing, e.g., privacy, integrity, or authentication protocols associated with mail. See there, including the discussions about holding this document until the A/S is ready .. in this case for concurrent publication. > Section 2.3.5, last paragraph (before the final two bullet points): > The first sentence of this paragraph is of questionable grammar > and, in any case, extremely hard to parse. Assuming I have parsed > it correctly, how about "Unless further restricted in this > document, domain names used in SMTP are names that can be resolved > to MX or address (A or AAAA) RRs (see also Section 5), or to CNAME > RRs that can be resolved to an MX or address RR." Good catch. This sentence resulted from too much patching (my notes show at least four partial rewrites since its predecessor first went into the document) and should have been caught and rewritten earlier. Let me consider your proposed new text, but I will rewrite this somehow in the working version. Unless instructed to the contrary, that version will appear before the IESG starts its deliberations. If it seems controversial, I'll follow up on this part of your review. > Section 2.3.12, first sentence: Appears to have missing text and/or > have grammatical errors and is hard to parse. How about something > like : "This document distinguishes between two concepts: (1) an > "SMTP session (aka "mail session") that starts when a connection is > made between client and server and ends when than connection > terminates", and (2) a "mail transaction that is started and > terminated by particular SMTP commands." Another good catch with the original being more complicated by a wording (near-typographical) error. I think even your proposed fix may be too complicated and obscure (although far better than the original). How about: 'This document distinguishes between an "SMTP session" and a "mail transaction". An SMTP session, often called a "mail session", starts when a connection is made between client and server, ending when that connection is terminated. A "mail transaction" is started and terminated by particular commands, most often MAIL and RSET or QUIT (the latter also instructs the server to close the SMTP session).' > Section 3.5.1/4.1.1.3: I saw the text in Section 1.3 about this, > but it still seems dubious to include real domain names like > foo.com and xyz.com in examples. Is it really going to cause > confusion to go with example.com or the like? (In any case, if you > are using "foo" or "bar" or "foobar", you should consider adding > RFC 3092 to the references.) For the reasons given in 1.3, I'm loathe to make changes to the examples, but, if you or others think it would be helpful, I could add, as a last sentence of that section, something like: 'In particular, some of those examples use variations on "foo", a convention explained in RFC 3092.' Does that work for you? Does anyone else have a problem with it? > Section 4.1.3: Please use IPv4 addresses reserved for > documentation. There are plenty available, particularly any in > 192.0.2/24, 198.51.100.0/24, and 203.0.113.0/24. See > https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ip > v4-special-registry.xhtml Seems entirely reasonable and trivial to fix. Done. > Section 4.1.3: Isn't a "1*" prefix, as in 1*dcontent, superfluous? My understanding of ABNF is that "*dcontent" means "any number of those including zero", while "1*dcontent" means there must be at least one of those. If ABNF experts speak up and say otherwise, I could, I suppose, change it to something like Standardized-tag ":" dcontent [ *dcontent ] but that seems at least equally likely to confuse readers. > Section 4.1.4, etc.: Why isn't there a state diagram? Instead, we > have somewhat verbose and redundant prose that is basically trying > to provide a state diagram and transition rules. I think the simple answer is that a state diagram would be extremely difficult to construct and error-prone, especially given the added complexities introduced by optional SMTP extensions. The verbosity is a result of the problem discussed in Sections 1.2 and 1.3 -- the document was created by merging text from several others and there has been great reluctance (first extensively discussed as what become RFC 2821 was coming together) to try to smooth things out lest errors be introduced. This is a case in which, if there are strong feelings that a state diagram would be helpful _and_ the WG is willing to review it, I'd be willing to give it a shot, or, better, review a draft prepared by someone else. But, absent that feedback, I recommend leaving it alone. > Section 4.1.4, page 48, 5th paragraph: Says server can verify > domain name argument in EHLO command. Can it also do that for HELO? > And what should it do if the verification fails? You are talking about the paragraph starting "An SMTP server MAY verify..." and talking about checking for an address record, right? If it tries and fails, the implementation has two choices: to reject the command as unacceptable (see two paragraphs earlier and the discussion of codes) or to conclude that the problem isn't worth the energy, reply with a 2yz code, and move on. That may sound strange, but there are some perfectly reasonable exception cases. For example, suppose we had EHLO big-cluster.example.com or even EHLO example.com but that a lookup yielded only MX records because the actual hosts either didn't have public DNS names at all or were cluster1.example.com cluster2.example.com One would certainly hope the command would not be rejected. As to HELO, please look at the second paragraph of Section 2.2.1, especially the last sentence. If you think HELO should be called out specifically in the portion of Section 4.1.4 referenced above, please propose a rule about when it should or should not be identified and discussed when ELHO is mentioned. If you feel those cases need further discussion, I urge you to get on the emailcore mailing list and propose text for the Applicability Statement. > Section 4.1.4, page 49, 3rd paragraph: This "MAIL MUST NOT be sent" > text seems confusing at a minimum. To most readers MAIL is "being > sent" with every RCPT command, with the DATA command, and with > every new line of mail content. Please re-word it so it's something > about "initiating" the sending of mail or the like. This is much like several cases where the WG decided to not change the text because the same wording (in this case, with the exception of removing discussion of SEND, SOML, and SAML) has been present since RFC 2821 and no actual issues with interoperability or even confusion have been demonstrated in more than 23 years. I'm open to suggested text and discussion in the WG, but I fear that any improvements would be subject to confusion on other points. As I reread it, I could see changing "MAIL MUST NOT be sent" to "message content MUST NOT be sent". Would that help? (Question addressed to the WG and community, not just Donald). > Section 4.2, page 50, right after the ABNF: the "where ..." > construct is normally used to provide more information for > something appearing in the immediately preceding > ABNF/diagram/whatever but "Greeting" does not occur in the ABNF > which is confusing. Reading further, apparently the actual quoted > word "Greeting" is not meant, which is also confusing. If it's any > generic greeting message, maybe just use lower case unquoted > greeting. But "Greeting" does appear in the ABNF. The first two text lines of the ABNF in that section are, separated by a blank line, 'In ABNF, server responses are: Greeting = ( "220 " (Domain / address-literal)' Am I missing something? > Section 4.2, page 51, first complete paragraph: For the IETF, what > is the difference between a Standard and a Standards-Track > specification? It seems like bad practice to put IANA assignment > criterion in the middle of body text like this. To the first question, IIR, back when we had a fantasy about the (then) three-level Standards process being actively used, those levels were "Proposed Standard", "Draft Standard", and "Standard", with an expectation that the distinction would be taken seriously. The preference for "Internet Standard" to refer to the latter came much later. I'd like instructions from the WG or some sort of community consensus, but I am open to (i) Leaving the text alone (ii) Putting "Internet" in front, i.e., making that "new Internet Standards or Standards-Track specifications" (iii) Dropping "Standards or". However, for (iii), note that many in the IETF, including some IESG decisions, have argued or assumed that BCPs are really standards-track documents. Because of that, the current text is possibly deliberately ambiguous and we should be careful what we wish for because an alternative to it could require a very careful, and possibly contentious, revision. Second, I don't see this text as specifying IANA assignment criteria, especially since there is no registry for SMTP reply codes, only the discussion in Section 4.2 and any updates that might be applied in the future. If you believe there should be such a registry, please make that case, but note that the interrelationships covered by the subsections in Section 4.2 would be quite hard to capture in a conventional IANA registry. In that regard, also note that the "rare and significant" comment in Section 4.2 is intended to discourage new codes in favor of enhanced reply codes and/or new text with existing codes. > Section 4.2.2: OK, but what are these "Functional Groups"? Some > sort of subheadings (not necessarily part of the hierarchical > numbering) are needed. These groupings, and the use of that terminology without a further breakdown or description or subheadings, come straight out of RFC 821. That is more than 40 years with no evidence that it has been the source of a problem. It might be that the difference in formatting between RFCs 821 (and 2821) and their successors is helpful. The formatting in the current document is identical to that in RFC 5321 (2821 followed the 821 model but was probably prepared using xml2rfc version 1 (i.e., the RFC 2629 version) and translated to nroff). I have figured out how to make xml2rfc restore that earlier layout but, if we want to do that, the WG is going to need to very carefully review the new groupings since there are codes that do not appear in RFC 821 or even in RFC 2821 and the ordering of some of the existing codes was changed (even between 2821 and 5321). > Section 4.3.2, for EHLO 450 says "see note above" but it seems > unclear what note how far above. The page break in this version of the draft is the problem. If you scroll up to the bottom of page 60, you will see the note. Given alternate rendering forms for RFCs, etc., I've tentatively changed the note to read 'see note immediately above under "CONNECTION ESTABLISHMENT"'. Does that work for you and everyone else? > Section 4.5.3.2: It seems confusing that the first paragraph talks > about "each buffer of the data transfer" but there is only one mail > data buffer at the server for a particular mail message and the > subject of 4.5.3.2.5 uses "block" while the contents of 4.5.3.2.5 > uses "chunk" and "TCP SEND call"s. I need WG advice as to what, if anything, to do about this. And I hope someone will correct me if the explanation below is not quite right. While I'm fairly certain that no one anticipated it when RFC 821 was being written, there are server implementations out there that need to transfer the incoming mail message somewhere else -- relaying it or putting it into a mailbox or non-SMTP delivery queue. They will process received commands through DATA, but will then (or soon thereafter) open the next connection in sequence, go through EHLO, MAIL, RCPT, and DATA commands, and then possibly start sending out message content while the content from the original message is still arriving. In addition to reducing the time between message sending and delivery, this allows those implementation to report most errors from the destination site while the incoming connection is still open (i.e., without later constructing bounce messages that can pose their own problems in the contemporary Internet, including security-related ones). If that isn't clear, ask and I'll try to draw a picture. > Section 8.1.1.1, last paragraph: The existing IANA model is > intended to minimize IANA "judgement" and, in general, make IANA's > job mechanical. Since the idea is to avoid name conflicts, this > document should request one table with the assignment model > indicated in a column or the like. A general comment about the many suggestions about Section 8 in this review: Perhaps, in my writing and discussions with the WG over the almost five years we've been struggling with this document (including a few drafts in the year before the WG was chartered), I'm still suffering from some unreasonable nostalgia for an IANA that operated somewhat independent of the IETF/IESG and actually made decisions (even some the IETF didn't like). At the same time, I've got considerable respect for the people who are there now and for their knowledge and insights about registry organization, parallels with other registries, and so on. Yes, the IETF can, these days, tell them exactly what to do and expect them to do it unless they clearly explain why it is impossible. But, at least IMO, that does not mean we need to assume they are not competent to make educated judgments. That section has been carefully reviewed with IANA and the text mentioned above is not one of their reported concerns. The IANA comments (see the datatracker entry if you are interested) point out that that there are extensive interactions with other registries and cross references. There are also, IIR, other registries that are split depending on how things got there, e.g., separating provisional and permanent registrations. One thing I'm sure of --and another motivation for that language -- is that figuring out exactly how registries should be organized is not a core competence of the WG. So, while my personal instinct matches yours, I think this document, at this point, should say as-is for now. My personal hope is that the IESG and IANA can work out whatever is best to do during IESG review or before publication and this text can then be adjusted to reflect what is being done or has been done rather than giving advice or talking about the future. > Section 8.3, first line: Should be phrased as a request to IANA > such as "IANA is requested to reorganize the Mail Parameters > Registry Group as follows:" All of Section 8 is, formally, requests to IANA. While that change would not be harmful, see above. I hope the final RFC can simply reflect what has been done. > Section 8.3, item (1) and Item (4): "should be" -> "is". But they aren't. These are more requests (or, if you prefer, instructions to) IANA. Either "is" is appropriate only after the changes are made. The alternative is that we should anticipate those changes being made and not be talking about requests at all. > Section 8.3, item (2): Should not tell IANA to "consider" anything. > Either ask IANA to do it or don't ask IANA to do it. This is all > going to get massaged in connection with IESG/IANA review anyway. See above. It is not clear that the exact organization of registries is within scope for the WG. The text was written on the assumption that IANA might have had reasons (even long-forgotten instructions) for organizing and labeling things the way they are. Some of these registries and their contents and organization date from very long ago, were not specified by the IETF, and, if there are notes or memories about particular choices of language and/or organization, neither I nor the WG have been given access to them. But the important thing, IMO, is the "massaged" part (consistent with my comments above). I'd prefer to focus on getting the document approved, working with IANA to get the registries organized in a reasonable way and all of the associated loose ends tied up, and then adjust the document to reflect what happened rather than spend time modifying the document over terminology, requests, etc. > And what IANA cares most about is clarity and lack of ambiguity. Yes, and, as mentioned above, IANA has already done a Last Call review of this document and they didn't raise any of the concerns about Section 8 that you are identifying. > Section 8.3, item (3): It is generally not IANA's job to review > registry names to judge whether they are "appropriate" which seems > like a technical question rather than a clerical question. Perhaps > IANA could chime in on whether a Registry name was excessively long > or confusing similar to another registry or the like, but if you > want to change the registry name, just request IANA to do that. > (Actually, with the change above, all these points are part of a > request to IANA.) Yes, and see above although I would argue that, when there are either many interrelated registries or ones with complex structures, it is reasonable to expect IANA to have opinions about consistency of naming, structure, etc. > Section 8.3, items (...): Changes in line with the above. See above. > Section 8.3, items (5) and (6): suggest combining these so it is > clear what "Those bullets ..." refers to. I think combining them would create other problems (and, again, IANA has already reviewed this and not objected). How would you feel about changing "Those bullet point fields" at the beginning of (6) to "The bulleted items identified immediately above"? I've tentatively made that change to the working draft. > Section 8.3, item (9): Appears to have an open technical question > that should be resolved. Whoops. No idea how that slipped through except that the WG has sometimes been moving too slowly or intermittently. I've made a note in the working draft. WG?? > Appendix D: I think this one paragraph of material should be > somewhere in Section 3.7. This was Appendix E of RFCs 5321 and 2821, with rather clear decisions made at RFC 2821 time to put it there. I'm reluctant to move it given the "has not caused identified problems in 23 years" principle, at least without either evidence of harm, clear WG consensus, or both. > Nits > ---- >... The above is quite long, it is late and I want to get the above off to give you, the WG, and anyone on the Last Call list who is interested, time to start going through it. I will come back to the Nits and "Matters of Personal Taste" lists in a separate reply. -- last-call mailing list -- last-call@xxxxxxxx To unsubscribe send an email to last-call-leave@xxxxxxxx
