Hi. What a surprise to spend some time working on something else and come back to 24 messages in the first 24 hours starting with Donald's posting Sunday evening and many more since then. I've been working on this in parallel with the incoming messages and have said some of what is below in those replies, but want to try to put parts of the puzzle together in the note. References to ticket numbers below are to https://github.com/ietf-wg-emailcore/emailcore/issues Dave Crocker and John Levine have said much of what I would have said, so this note is something of a summary as well as some comments about where I personally (and as editor) stand right now. I also appreciate Rob Sayre's observations about the role of the SMTP protocol itself. The handling of all of these issues is up to the WG and then to the IESG, but the comments below represent some sort of default if the WG does not instruct me otherwise (in some cases, there are explicitly no defaults, only a question for the WG). First, unless the WG and I hear differently from Murray (as responsible AD), the intent to publish this document as an Internet Standard very significantly constrains the number and type of changes that can be made from RFC 5321. That "limited change" view is strongly reinforced by the EMAILCORE charter and some on-list discussions since I started to reply to Donald's note within 24 hours after he posted them. As I have understood and interpreted those rules over the years, some of the changes suggested would send us straight back to Proposed Standard. I'm confident there is no desire in the WG --and I hope no significant desire in the community-- to go there. Second, as far as retroactive application of the list in RFC 3552 is concerned (and in addition to Dave's comment about the chronological relationship between 3552 and 5321), I note with interest that RFC 4871 and then RFC 6376 were also published after 3552. AFAICT, neither mentions the ability of a hostile relay/intermediate MTA system to tamper with the headers, change "Received:" header fields, signatures, etc., rather than, e.g., just rearranging them or adding new ones. RFCs 8301, 8463, 8553, and 8616, which update 6376, do not appear to change that situation. The potential for intermediate MTA systems to tamper with mail headers in hostile ways has been known since, IIR, even before RFC 821 was published, so, if Security Considerations sections are required to be as comprehensive as some of the notes in this thread seem to imply, RFC 6376 should probably be demoted to Proposed Standard pending more adequate documentation. Third, I would personally welcome a complete security analysis of email -- not only transport via SMTP but looking at Message Submission, LMTP, IMAP and POP, and the various remedies and workaround for the problems that are discussed. That should include an analysis of the problems that S/MIME and OpenPGP (and SPF, DKIM, and friends) don't solve and when those problems are important. At least as important, it should address whether remedies such as transport over TLS and various server and authentication methods for (human) senders, servers, etc., really work in all cases, where they don't, and what can be done to remedy the deficiencies in the remedies. Partially because of the number of protocols that such an analysis would need to consider and partially because I think the conclusion might be "SMTP can't be fixed, need to start over and figure out if an alternate solution could possibly be deployed" it seems clear that we should not hold draft-ietf-emailcore-rfc5321bis waiting for it, much less plan to incorporate it into the current draft. With only a bit less conviction I'd make almost the same argument about trying to incorporate it into draft-ietf-emailcore-as -- much of it may be really out of scope there too. A conclusion that it is time to discard SMTP, and maybe even significant parts of the mail header model described in draft-ietf-emailcore-rfc5322bis, would, of course, not be a new conclusion: Suzanne Sluizer, the co-author of RFC 780 (RFC 821's primary immediate predecessor) was saying that loudly in various forums before we started on the extension mechanism documented in RFC 1425 and eventually incorporated into RFC 2821 and hence 5321 and the present draft. Much more recently, we've seen proposals for email in specialized environments (e.g., MULE in RFC 8494) and newer ones such as mail over the Mathematical Mesh (see draft-hallambaker-mesh-*). I don't think a comprehensive analysis of email security issues would be complete without at least some consideration of both the current protocol collection and, unless whatever problems are identified can be completely fixed, at least some of the proposed replacements. >From that perspective, suggestions like "why don't we at least mention FOO" make it very hard to draw the line about various values of "FOO" (starting with "DKIM") and how they are mentioned. The mention of Multipart/Signed, Multipart/Encrypted, OpenPGP, and S/MIME in the first paragraph of Section 7.1, is not an exception to that line-drawing exercise because they are cited as specific examples of methods that work end to end and that are not dependent on primary message headers or header fields. A different, more specific response to "why not at least mention DKIM?" is that doing so, especially in that context, would not only require the name, but an explanation. I suppose we could include specific mention of DKIM in the second paragraph of 7.1 ("Various protocol extensions...") because it matches that description, but without a more through explanation, mentioning it there would probably require a complete analysis and that takes up back to either that complete security analysis or needing a stopping rule about which the community could reach consensus. My conclusion, and that of the WG so far (this was discussed) is that the best stopping rule is to not go in that direction in the base SMTP document. Based on the recent discussions about requirements for Security Considerations, I think "certainly not without a comprehensive security analysis in the DKIM document" belongs somewhere there too. In partial response to Steffen's question about "why RFC 9051 can talk about STARTTLS and "Implicit TLS", but 5321 cannot", IMAP is a direct client-server connection protocol, not a hop-by-hop one. Comments below on points not covered above. --On Sunday, October 27, 2024 21:28 -0400 Donald Eastlake <d3e3e3@xxxxxxxxx> wrote: > Hi John, > > My apologies for my delay in response. In responding, I am looking > at the -32 version while my comments originally referred to -31. > > On Wed, Oct 16, 2024, at 2:22â AM John C Klensin > <john-ietf@xxxxxxx> wrote: >> ... >> >> Because the response below proposes several changes to the document >> and to encourage them to check my explanations, I've explicitly >> copied the WG mailing list. > > Sure, I should have copied the WG on my initial message. > >> ... >> >> When EMAILCORE was chartered, and even going back to the first >> draft charter in September 2020, the explicit plan was for it to >> produce three documents: The first two were to be Internet Standard >> replacements for RFC 5321 and 5322. They were constrained, by both >> the charter and by the rules for Internet Standard, to the scope >> and coverage of their predecessors. The third document (which is >> mentioned at the end of Section 1.2 and that is still in progress >> in the WG as draft-ietf-emailcore-as) was to be an Applicability >> Statement that addressed other work and related issues in the email >> world. The result of those constraints and decisions was that a >> discussion of, e.g., DMARK and DKIM was out of scope for the >> current > > I did not suggest a discussion of DMARK or DKIM. I suggested that > they be mentioned. See above. >> document and very much in scope for that A/S. There are additional >> issues about they, and related work, should be handled there, more >> on that inline below. >> >> ... >> >> On to specifics. >> >> --On Friday, October 11, 2024 23:56 -0400 Donald Eastlake >> <d3e3e3@xxxxxxxxx> wrote: >> >> > ... >> > >> > The summary of the review is Has Issues. >> > >> > This document is an obsoleting revision of RFC 5321 on the Simple >> > Mail Transfer Protocol (SMTP), incorporating Errata, etc. It is a >> > mix of specification, operational, and historic information. >> > Changes from RFC 5321 are summarized in Appendix F. I did not >> > review the Acknowledgements or References sections or Appendices >> > C or E. >> > >> > Security >> > -------- >> > >> > This document has an extensive Security Considerations section >> > which covers most points. But I have the following suggestions: >> > >> > Section 7: Could be clearer on what the threat model is. >> >> I agree, but going much further than the current text goes would >> probably take us down a rather deep rabbit hole (or several of >> them). The WG therefore decide to kick the rabbits down the road >> to the A/S. > > If some of the Security Considerations are actually deferred to the > Applicability Statement, then this document should clearly say that > is what is being done. It should say that in the Security > Considerations section, perhaps between the 7. and the 7.1 headers. See on-list comments about references to I-Ds in progress but whose content is just not stable yet. Also note the comments about a comprehensive security analysis for email above. That said, if the WG changes its mind or the IESG interprets the community as insisting, I have no problem holding my nose and putting the forward pointer in. A placeholder has been inserted in the working draft. >> The question that obviously raises is whether the Last Call on this >> document should be held until some of it can be reviewed in >> parallel with a version of the A/S that the WG thinks is ready for >> publication. I would prefer to avoid that because it would create a >> near-invitation for going back and forth about what should be in >> the A/S and what details or explicit references should be slipped >> back and forth between the A/S and this document. FWIW, the WG >> spent many months on similar issues involving the relationship >> between the current document and draft-emailcore-rfc5322bis, which >> has gone through IETF LC and has been approved by the IESG. >> Holding this document (referred to below as rfc5321bis) would also >> be a good argument for holding rfc5322bis. I think it is safe to >> say that neither the WG nor our long-suffering AD would be happy >> about such holds. My personal concern is that it might result in >> further attrition among those active in the WG, possibly reducing >> the breath of perspectives and quality of the work, especially on >> the A/S, and consequently considerably extend the time it will >> take to get all three documents out. > > It seems unusual for much of the Security Considerations for an IETF > protocol specification to be in a separate Applicability Statement > document. Maybe the Applicability Statement should be renamed to > something more inclusive. If the threat model is in part deferred to > the other document, this document should clearly say so in the > Security Considerations section. See above. FWIW, I don't believe the threat model has been deferred, only a discussion of possible (or partial) remedies and mitigation measures and their weaknesses and implications. But that may be just a difference in terminology about "threat model". >> ... >> >> > Section 7.1, particularly re 1st paragraph: Shouldn't there be >> > some mention of DMARC (RFC 7489) and DKIM (RFC 6376) and perhaps >> > other things that don't come to my mind immediately? >> >> No. See above. > > As above, I asked for a mention, not a discussion of these. Again, see above. > This document is extraordinarily focused on what are, to my mind, > 2nd order application-level security concerns and gives essentially > no weight to the obvious primary threat of observation of plain text > email in flight. The concerns covered are real concerns and I am not > suggesting taking anything out, but it is a symptomatic security > flaw of this document that the character string "encrypt" occurs > nowhere in the main body of the document. That you can spoof mail, > leak information in trace lines and through some commands, and the > few other things mentioned are not the only thing insecure about > SMTP. It's like this document was written before pervasive > surveillance was a thing (RFC 7258). Given that it is based on RFC 5321 and uses much of the same text as that document, parts of it obviously were written before RFC 7258 even though one might debate when surveillance of traffic on the network because "pervasive". > Suggest changing the beginning of Security 7.1 from "SMTP mail is > inherently insecure ..." to "The authenticity of SMTP mail is not > secure..." since that's what Section 7.1 is about. I think I said something like that during the list discussion, but I'm not sure that a change in that sentence is the right answer. Might be, just not sure. Tickets #111 and #112. >> And those two documents in particular have a >> difficult relationship with part of the threat model. Some people >> would argue that most (or nearly all) of the Internet's email goes >> directly from the administrative domain of the sender to the >> administrative domain of the recipient. A few would even argue that >> the numbers are large enough that the hop-by-hop model --one has >> been part of mail transport since about the time the community >> gave up to transporting mail over FTP -- should be deprecated. >> Perhaps > > I like the SMTPhop-by-SMTPhop model. I don't see why you are > dragging this issue into the discussion. This document is clearly > based on the SMTPhop-by-SMTPhop model and I don't see how I have > suggested any change in that model. Your paragraph above seems to > have almost nothing to do with my comments. I apologize, it should not have been directed at you and was intended for a broader audience. It was strongly influenced by other notes that seemed to imply that, if everyone used STARTTLS, there would be no security problems with email. >> unfortunately, counterexamples abound because even simple mail >> forwarding and list expansion introduce intermediary systems. Those >> intermediary systems -- whether mail relays or something more >> complicated -- have the potential to alter any part of the mail >> headers they receive. Whether such alterations are deliberate or >> accidental results of other processing, there are many examples of, >> e.g., DKIM/DMARC not being trustworthy beyond the most recent hop >> and, in a multihop environment, therefore fairly useless in >> authenticating the sender or the identity of the received message >> to the original one. > > The Security Considerations for DKIM/DMARC are issues for the > Security Considerations of the RFCs for those protocols. This > document should mention them and point to their Security > Considerations in their RFCs. There is little need to discuss their > Security Considerations in this document, just mention them and > point to them. See my introductory comments. The difficulty is that first we mention DKIM/DMARC, which are not even SMTP extensions -- arguably, if they should be mentioned/pointed to anywhere in the core documents, it should be in 5322bis or the Message Submission documents. Then we mention STARTLS. Then AUTH. Then ENHANCEDSERVICECODES. Then maybe CHUNKING and/or CHECKPOINT. Then SMTPUTF8 (which really is important). Soon one has a catalog of registered extensions (see <https://www.iana.org/assignments/mail-parameters/mail-parameters.xhtml#mail-parameters-2>) and another of stuff people use with Internet Email. None of that, as I and others have repeatedly pointed out, is _part_ of SMTP. Including it, even as a matter of mentions or a table, would be a long list and, because the list, and the RFCs that define any given one (or group) of them changes, a maintenance problem for what we'd like to be a very stable document. >> The problem runs even deeper: running SMTP over TLS provides >> encryption protection in transit between hosts but, if any >> intermediate host is compromised, the message is (at least as far >> as any protection from TLS) sitting on that host in the clear. > > Of course, that's obvious. It is a general characteristic of link > security being unable to provide end-to-end security and a general > characteristic of "store and forward" systems (even if the > implementation is actually pipelined). I continue to be surprised by the number of people who have commented on the Last Call list that don't seem to understand that. > In a unified Security Considerations section for SMTP, it would be > reasonable to classify most of those Considerations into end-to-end > considerations and SMTPhop-by-SMTPhop considerations. I'm not asking > for detailed Security Consideration in SMTP documents for any of > such end-to-end or SMTPhop-by-SMTPhop protocols. Their Security > Considerations should be in their RFCs. You mention PGP and S/MIME > but don't discuss them. I have not seen from you any good reason to > not, in an analogous way, mention DKIM and DMARC without discussing > them. Precisely because PGP and S/MIME work, reliably, end-to-end. Again see the introductory comments above. As I have said elsewhere, if the mention of PGP and S/MIME are the problem here, I think the WG should consider taking them out even though it would probably weaken the document somewhat. >> I hope I'm not just passing the buck here, but I believe that >> documentation of these threats are more properly something that >> should be done in Security Area documents than somehow discussed in >> depth as part of SMTP or even in the proposed A/S. > I'm not an AD and the boundaries between Areas in the IETF are quite > flexible. But I think that generally the Security Area does not > consider it to be the Security Area's job to do threat analysis for > a particular application protocol that is in another area. The difficulty, as you are more or less pointed out, is that the Security Area has created protocols that create the impression of more security when used with SMTP than is actually the case, at least partially due to what appear to be incomplete understanding to SMTP's properties and the operational environments in which it is used. So we may be looking at a situation in which the two Areas have not been paying adequate attention to each other's work in protocol development and specification. >> > Section 7.1, particularly re 2nd paragraph: I think it would be >> > useful to mention something like the following: >> > 1. Transport authentication between SMTP systems could improve >> > the authenticity of the Received line added by a server but does >> > not protect those lines against modification, in violation of >> > this document, by subsequent SMTP systems. >> > 2. Nation State intelligence agencies and others on occasion, >> > have been known, even in the case of extremely high bandwidth >> > traffic mixing many streams, for example between large data >> > centers, as well as lower bandwidth connections, to capture and >> > hold immense quantities of raw traffic for potential later >> > analysis, so there may be good reason to encrypt transmissions >> > between SMTP systems. >> >> See Section 6 of the draft A/S. I personally don't think that goes >> far enough and have said so in correspondence within the WG; your >> suggestions add to that belief or might suggest an additional >> document. > > Feel free to use any text I suggest in any of the documents. See Ticket #113. >> > Non-Security >> > ------------ >> > >> > Section 6.1, 6.2: I do not quite understand saying that >> > something is mandatory/required and then talking about where it >> > is impossible or impractical to do that thing. Section 6.1 says, >> > concerning a host holding an email message, "It MUST NOT lose >> > the message for frivolous reasons, such as because the host >> > later crashes ...". "frivolous" does not seem very well defined >> > for a mandatory implementation requirement. The only examples >> > given later of non-frivolous reasons seem to be spam or >> > "hostile" (infected?) messages. Maybe this should be more like >> > "In so far as practical, messages MUST NOT be accidentally lost. >> > For example, messages and their processing state should be >> > committed to non-volatile storage until responsibility is taken >> > for that message by the next host in the delivery path." or >> > something like that. In Section 6.2, how does it make sense to >> > say that something is required and impractical? >> >> Yeah. This problem goes back to RFC 2821, i.e., it has been with us > > Perhaps I am sacrilegious, but I don't care how far back it goes. > That something was poorly or contradictorily stated in obsoleted > predecessor documents and there has not been a disaster is not a > reason to refrain from simple obvious improvements to the text in > this document. I'll look at it again, and ask the WG to do so, but my guess is that part of the problem is that there are no "simple obvious improvements" that don't risk causing problems elsewhere. Ticket #114. >> since decisions made in the last century. Indeed, that particular >> text goes back more than another decade: it is copied, almost >> exactly, from RFC 1123 (and I didn't write that paragraph then >> either). The ideal solution, at least in or before RFC 2821 was >> published in 2001, would have been to completely restructure and >> rewrite this rather complicated combination of previous documents. >> See Section 1.2 for an explanation of that problem. We could >> re-think and tune some localized text now, but it is not clear that >> it would make much difference. Some of the obvious fixes would be >> substantive (e.g., if something is not really mandatory/required, >> probably any associated or implicit MUSTs should be changed to >> SHOULDs and then explained better. This time around, the WG has >> been very reluctant to make changes like that and has decided to >> minimize change especially when there was no significant evidence >> of real harm or serious confusion in the last 16 years (or much >> longer). >> >> Again, and at least in retrospect, I think we should have >> completely rewritten the thing during DRUMS in 1995-2001. We (the >> WG at the time) decided to not do that, in part because the risk >> of introducing errors that would effect existing deployed >> implementations. Now, > > I do not think that changing the documentation actually reaches out > and changes the deployed successfully running code to introduce > errors. > >> even if we wanted to do so, I don't believe that there is either >> sufficient energy or resources to do that rewrite, nor am I >> confident that a subsequent review that would catch all >> discrepancies. The WG very briefly discussed the "full rewrite" >> option and reached the same conclusion. > > I have not suggested a complete rewrite. I have not suggested even > one change of a "MUST" to a "SHOULD". I am above just suggesting > improved wording in Section 6.1 and complaining about the conflict > in Section 6.2 > > 6.2 could be trivially fixed by simply replacing "requires that > messages that can be delivered should be delivered" with "benefits > when messages that can be delivered are delivered". I do note that > this "requires" statement does not use a capitalized keyword and is > softened by the lower case "should". I did not notice any > self-contradiction as egregious as this elsewhere in the document. Also included in Ticket #114. >> ... >> >> > Minor >> > ----- >> > >> > Global: RFC 821 is obsolete. It was published 42 years ago and >> > obsoleted 21 years ago. Is there really a need for references to >> > it in this document? >> >> Procedural/nitpicky explanation: Due to a twitch in our vocabulary >> and how aspects of RFC 2026 have been implemented, "obsoleted" is a >> tricky term. Yes, RFC 2821 listed RFC 821 as being obsoleted but >> the > > I do not think "obsoleted" is "tricky" for the purposes of this > document. I see the purpose of a Standards Track protocol document > as enabling developers to produce interoperable implementations. If > the IETF says "X obsoletes Y" then they are correct if you can do > such implementations based on X. If you still have to read Y to > produce interoperable implementations, then the declaration of > obsolescence was wrong. See discussion below and the comment, essentially on this subject, in Section 1.2 of draft-ietf-emailcore-rfc5321bis-32 and -33. Ticket #115. >> RFC index and metadata still lists it as STD 10 and is the only >> Internet Standard among core email documents (unless you count RFC >> 6409). I have believed, at least on and off, for a long time that >> any standards Track or BCP document that is obsoleted should have >> its status changed to "Obsolete" or something like that. But that >> is not what the RFC Editor does and, when I have raised this with >> a few generations of RFC Editors (but probably not to RSWG), they >> expressed the view that it would require a change to RFC 2026. I >> could certainly see getting rid of the NCP, NITS, and X.25 >> appendices (the latter if someone could prove it is really dead >> worldwide). But even > > I have looked at the X.25 appendix to RFC 821. It is only one > sentence and all it says is that if your lower level transport is > X.25, you should use TCP over it. seems pretty vacuous to me. I am tempted to say "take it up with Postel". But I won't. >> that would require more, possibly error-attracting, fussing with >> the text than anyone in the WG was inclined to propose. >> >> That does suggest one addition to the document. In Section 1.2, the >> paragraph starting "It obsoletes" should perhaps contain an >> additional sentence explicitly removing the "Internet Standard" >> status from RFC 821. If that is needed, draft-emailcore-rfc5322bis >> should be similarly patched. Do you (and others) think such a fix >> would be desirable? > > I see this has been added. Yes. And the note updated a bit in -33. >> More substantive answer: the present document contains significant >> text copied from RFC 821 but without the context being copied. The >> references to 821 could be removed but only at the expense of >> rewriting sections to include more of that context. Similar >> comments might be made about some of the other very old documents >> being referenced although 821 is the oldest. It is probably the >> one that is most sensitive to citations vis-a-vs the later ones. > > In my opinion, the purpose of a Standards Track protocol RFC is to > provide the information developers need to produce interoperable > implementations. If you are saying that there is material in RFC 821 > that developers must read to produce interoperable implementations, > then you should just plain confess that the chain of obsoletes going > back from this document is wrong and you should tell developers up > front in this document that they have to study RFC 821 as well as > this document in order to produce an interoperable implementation. > If you are saying that there are historical aspects or ultra fine > gradations of something or other that might be conveyed by reading > RFC 821 but don't affect implementation, then you should say that > and you should re-assure developers that this document is > sufficient to develop interoperable implementations of SMTP and > that they do NOT need to study RFC 821 despite the dozens and > dozens of references to RFC 821 in this document. References which, > in my opinion, add nothing for a developer and which give the > impression that RFC 821 should be checked on most things. > > Every time I came across the numerous cases where this document says > ~as in RFC 821~ or the like, the thought always popped into my head: > "Why would I, as a consumer of this protocol specification, care > what RFC 821 said?" Either this document is the protocol > specification for SMTP or it isn't. The difficulty is that, every time we have considered taking that material out (starting with RFC 2821), someone says "legacy implementations" and/or "backwards compatibility". The observation that, if someone asks "what is the internet standard for email transport", the answer is likely to come back "RFC 821" does not help, of course. The most obvious example is that, when we introduced the extension model in 1993, that WG came to the conclusion that the only way to prevent disruption was to use a different session-opening comment, EHLO rather than 821's HELO. Well, it is now 30+ years later. The number of implementations that are sending or receiving non-trivial traffic across the public Internet that don't support EHLO and default to it is very likely zero or at least close to it. But, especially if one looks at IoT-ish applications, as discussed on-list, the number of things around is probably not zero, just as the number of things that can't support STARTTLS is probably not zero, the number who cannot handle non-ASCII addresses is not zero, etc. So we can't really depreciate the old stuff in any effective way -- doing so would cause interoperability problems. Now, if I were giving advice to someone doing a brand-new implementation of SMTP, it would include "don't even consider sending HELO, but be prepared to receive it sometimes and don't go nuts about that". Should the document or the Applicability Statement say that clearly? Don't know but I'd say probably the latter if at all. Why then, talk about 821 at all? Because, if one is going to run a server and feels a need to support legacy, 821-only, clients -- that is different from accepting HELO and turning a bunch of stuff off-- one probably needs the details. So why not incorporate all of that obsolete material in the document, probably in an appendix: Because pulling it together in a neat way would be hard and a lot of work (starting with point by point discussions of whether one wanted 821, or 821+974, or 821+974+1123), because the WG really does not want a 150+ page document if we can avoid it (100 or 110 is bad enough), and, most important, because the question interacts with reasons why we did not do a complete rewrite, rather than a merge, when RFC 2821 was assembled. If we had, the 821 material would probably selectively be there and, if it wasn't, it could probably be easily added back in now. But we didn't, it is too late to change that decision, and, as someone else more or less said, what we face now is really a question of whether this draft, plus or minus localized and non-structural changes, is better or worse than just sticking with RFC 5321. > Part of the problem is that this document is a tangle of protocol > specification, operational considerations, historical information, > and general motivational essays. I like historical information. A > number of my RFCs have Historical Notes in them, but always as an > aside, not scattered through the text. There is a section (1.2) > designated for such material. Some might say it should be an > Appendix, but I am fine with having 1.2 up front. I understand that > disentangling the specifications and operational guidance would be > hard at this point and am not pushing for that. But ideally > historic stuff would not clutter up the protocol specification but > should be concentrated in one section to make the protocol > specification crisper. See above. And, for someone trying to build or maintain a system that has to interact with legacy systems, the definition of "historical information" is far less precise than the above implies. >> > Section 1.2, last paragraph: Future commitments always seem a bit >> > risky. Suggest replacing "forthcoming" with "planned". >> >> Normally, I would agree. But, not only is the A/S well along but >> describing it simply as "planned" runs directly into the discussion >> above about the importance of discussing, e.g., privacy, integrity, >> or authentication protocols associated with mail. See there, >> including the discussions about holding this document until the A/S >> is ready .. in this case for concurrent publication. > > As long as the AD and IESG are willing to go along with this > somewhat peculiar relationship between this document and the so > called "Applicability Statement" document. Agreed. But note that other comments have strongly suggested normative references to it, which is (or should be using your logic and mine) an even harder pill to swallow. >> > Section 2.3.5, last paragraph (before the final two bullet >> > points): The first sentence of this paragraph is of questionable >> > grammar and, in any case, extremely hard to parse. Assuming I >> > have parsed it correctly, how about "Unless further restricted >> > in this document, domain names used in SMTP are names that can >> > be resolved to MX or address (A or AAAA) RRs (see also Section >> > 5), or to CNAME RRs that can be resolved to an MX or address RR." >> >> Good catch. This sentence resulted from too much patching (my notes >> show at least four partial rewrites since its predecessor first >> went into the document) and should have been caught and rewritten >> earlier. Let me consider your proposed new text, but I will >> rewrite this somehow in the working version. Unless instructed to >> the contrary, that version will appear before the IESG starts its >> deliberations. If it seems controversial, I'll follow up on this >> part of your review. > > Looks good in version -32. thx. >> > Section 2.3.12, first sentence: Appears to have missing text >> > and/or have grammatical errors and is hard to parse. How about >> > something like : "This document distinguishes between two >> > concepts: (1) an "SMTP session (aka "mail session") that starts >> > when a connection is made between client and server and ends >> > when than connection terminates", and (2) a "mail transaction >> > that is started and terminated by particular SMTP commands." >> >> Another good catch with the original being more complicated by a >> wording (near-typographical) error. I think even your proposed fix >> may be too complicated and obscure (although far better than the >> original). How about: >> >> 'This document distinguishes between an "SMTP session" and a >> "mail transaction". An SMTP session, often called a "mail >> session", starts when a connection is made between client and >> server, ending when that connection is terminated.  A "mail >> transaction" is started and terminated by particular >> commands, most often MAIL and RSET or QUIT (the latter also >> instructs the server to close the SMTP session).' > > I'm quite happy with your change in version -32. two in a row! :-) >> > Section 3.5.1/4.1.1.3: I saw the text in Section 1.3 about this, >> > but it still seems dubious to include real domain names like >> > foo.com and xyz.com in examples. Is it really going to cause >> > confusion to go with example.com or the like? (In any case, if >> > you are using "foo" or "bar" or "foobar", you should consider >> > adding RFC 3092 to the references.) >> >> For the reasons given in 1.3, I'm loathe to make changes to the >> examples, but, if you or others think it would be helpful, I could >> add, as a last sentence of that section, something like: >> >> 'In particular, some of those examples use variations on >> "foo", a convention explained in RFC 3092.' >> >> Does that work for you? Does anyone else have a problem with it? > > Fine on foo. I guess on the non-reserved example domain names, we > will see what the AD/IESG say. Ack >> > ... >> ... >> >> > Section 4.1.3: Isn't a "1*" prefix, as in 1*dcontent, >> > superfluous? >> >> My understanding of ABNF is that "*dcontent" means "any number of >> those including zero", while "1*dcontent" means there must be at >> least one of those.  If ABNF experts speak up and say otherwise, >> I could, I suppose, change it to something like >>   Standardized-tag ":" dcontent  [ *dcontent ] >> but that seems at least equally likely to confuse readers. > > Sorry, don't know what I was thinking. The ABNF is fine. >> > Section 4.1.4, etc.: Why isn't there a state diagram? Instead, we >> > have somewhat verbose and redundant prose that is basically >> > trying to provide a state diagram and transition rules. >> >> I think the simple answer is that a state diagram would be >> extremely difficult to construct and error-prone, especially given >> the added complexities introduced by optional SMTP extensions. The >> verbosity > > Sure, I meant a state diagram or table(s) for this document covering > the basics specified in this document, not extensions specified > elsewhere. I would normally guess that if it is "extremely > difficult" to construct a correct state machine, as you say, then > the protocol isn't understood. An implementer necessarily has to > directly or indirectly figure out all the states, events, and > transition rules they are implementing whether they implement that > in an organized tabular fashion or in spaghetti code. In general, > it seems to me better to have organized table(s)/diagram(s). But if > there isn't enough energy to make the improvement, then I agree we > have gotten along with it as is. I'm not going to open a ticket on this but, if anyone else feels strongly about it, feel free. >> is a result of the problem discussed in Sections 1.2 and 1.3 -- the >> document was created by merging text from several others and there >> has been great reluctance (first extensively discussed as what >> become RFC 2821 was coming together) to try to smooth things out >> lest errors be introduced. This is a case in which, if there are >> strong feelings that a state diagram would be helpful _and_ the WG >> is willing to review it, I'd be willing to give it a shot, or, >> better, review a draft prepared by someone else. But, absent that >> feedback, I recommend leaving it alone. >> >> > Section 4.1.4, page 48, 5th paragraph: Says server can verify >> > domain name argument in EHLO command. Can it also do that for >> > HELO? And what should it do if the verification fails? >> >> You are talking about the paragraph starting "An SMTP server MAY >> verify..." and talking about checking for an address record, right? >> If it tries and fails, the implementation has two choices: to >> reject the command as unacceptable (see two paragraphs earlier and >> the discussion of codes) or to conclude that the problem isn't >> worth the energy, reply with a 2yz code, and move on. That may >> sound strange, but there are some perfectly reasonable exception >> cases. For example, suppose we had >> EHLO big-cluster.example.com or even >> EHLO example.com >> but that a lookup yielded only MX records because the actual hosts >> either didn't have public DNS names at all or were >> cluster1.example.com >> cluster2.example.com >> >> One would certainly hope the command would not be rejected. >> >> As to HELO, please look at the second paragraph of Section 2.2.1, >> especially the last sentence. If you think HELO should be called >> out specifically in the portion of Section 4.1.4 referenced above, >> please propose a rule about when it should or should not be >> identified and discussed when ELHO is mentioned. > > You always seem to think that a request to mention (or identify) > something is a request to add an extended discussion to the > document. Not so. I didn't see anything about an extended discussion there although the comments above about EHLO and HELO support might imply that. The only question is whether we need to go through the document, find every instance of EHLO, and figure out if HELO should be mentioned (or identified) there. That is almost certain to turn out to require some effort to get right even if no discussion is added or complicated. The extended discussion would only be in the WG about who gets to do the work. But see below. > Seems simple enough to me. There are ~83 occurrences of EHLO in the > document. There are plenty of places where it is clear that EHLO is > mentioned in relation to extensions, so HELO is irrelevant. And > there are a number of places where "EHLO (or HELO)" is already > used. For the remaining sections where EHLO is mentioned, I suggest > replacing the first and only the first occurrence of "EHLO" in each > of the following numbered sections with "EHLO (or HELO)": > 3.3 > 4.1.1.5 > 4.1.4 > 4.2 > 4.2.4.1 > 4.2.4.2 > 4.4.1 and > 4.4.3 Ok. Ticket #116. >> ... >> > Section 4.1.4, page 49, 3rd paragraph: This "MAIL MUST NOT be >> > sent" text seems confusing at a minimum. To most readers MAIL is >> > "being sent" with every RCPT command, with the DATA command, and >> > with every new line of mail content. Please re-word it so it's >> > something about "initiating" the sending of mail or the like. >> >> This is much like several cases where the WG decided to not change >> the text because the same wording (in this case, with the exception >> of removing discussion of SEND, SOML, and SAML) has been present >> since RFC 2821 and no actual issues with interoperability or even >> confusion have been demonstrated in more than 23 years.  I'm >> open to suggested text and discussion in the WG, but I fear that >> any improvements would be subject to confusion on other points.  >> As I reread it, I could see changing "MAIL MUST NOT be sent" to >> "message content MUST NOT be sent".  Would that help?  >> (Question addressed to the WG and community, not just Donald). > > I think we are talking somewhat at cross purposes here. I simply > misread the text. I read it as if it said "Mail MUST NOT be sent > ...". Perhaps just change to "The MAIL command MUST NOT be sent > ...". Aha. Ticket #101 but that fix works for me. Tentatively changed and explained in the working draft. >> > Section 4.2, page 50, right after the ABNF: the "where ..." >> > construct is normally used to provide more information for >> > something appearing in the immediately preceding >> > ABNF/diagram/whatever but "Greeting" does not occur in the ABNF >> > which is confusing. Reading further, apparently the actual quoted >> > word "Greeting" is not meant, which is also confusing. If it's >> > any generic greeting message, maybe just use lower case unquoted >> > greeting. >> >> But "Greeting" does appear in the ABNF.  The first two text >> lines of the ABNF in that section are, separated by a blank line, >> >> 'In ABNF, server responses are: >> Greeting = ( "220 " (Domain / address-literal)' >> >> Am I missing something? > > Sorry, my mistake. >> > Section 4.2, page 51, first complete paragraph: For the IETF, >> > what is the difference between a Standard and a Standards-Track >> > specification? It seems like bad practice to put IANA assignment >> > criterion in the middle of body text like this. >> >> To the first question, IIR, back when we had a fantasy about the >> (then) three-level Standards process being actively used, those >> levels were "Proposed Standard", "Draft Standard", and "Standard", >> with an expectation that the distinction would be taken seriously. >> The preference for "Internet Standard" to refer to the latter came >> much later. >> >> I'd like instructions from the WG or some sort of >> community consensus, but I am open to >> (i) Leaving the text alone >> (ii) Putting "Internet" in front, i.e., making that "new >> Internet Standards or Standards-Track specifications" >> (iii) Dropping "Standards or". >> >> However, for (iii), note that many in the IETF, including some IESG >> decisions, have argued or assumed that BCPs are really >> standards-track documents. Because of that, the current text is > > RFC 8126 says that the "Standards Action" assignment policy is the > issuance of a Standards Track or Best Current Practice RFC in the > IETF Stream. Thus, at least for regular IANA Considerations > purposes, a BCP is not a Standards Track document but its issuance > is a Standards Action. Which makes sense since, while the Standards > Track has only two steps as opposed to the three it used to have, > there is still a track to progress through but there is no such > track for BCP. My point, however, was that any IETF "Standard" is > clearly included in "Standards Track" documents. So, the current > text of "... new codes may be added as the result of new Standards > or Standards-Track specifications." is strictly redundant and > "Standards orÓ" should be stricken. Ack. Working draft updated; Ticket #117. >> possibly deliberately ambiguous and we should be careful what we >> wish for because an alternative to it could require a very >> careful, and possibly contentious, revision. > > I see no reason why the assignment policy for SMTP reply codes > should be contentious. That sort of thing (reply codes, error > codes, etc. that may have to be interpreted by an implementation of > an older version of a protocol) is normally Standards Action (e.g., > DNS RCODEs). But if desired it could be tightened to "Standards > Action other than BCP". > >> Second, I don't see this text as specifying IANA assignment >> criteria, especially since there is no registry for SMTP reply >> codes, only the discussion in Section 4.2 and any updates that >> might be applied in the future. If you believe there should be >> such a registry, please > > An assignment policy is an assignment policy. Its existence should > not be buried in the middle of a large specification if you want > people to notice it and pay attention. > >> make that case, but note that the interrelationships covered by the >> subsections in Section 4.2 would be quite hard to capture in a >> conventional IANA registry. In that regard, also note that the >> "rare > > If there are believed to be unusual constraints on or procedures for > the allocation of code points for an IANA Registry, you shouldn't > include an essay as a note with the registry, you just include a > pointer to the appropriate RFC section. > >> and significant" comment in Section 4.2 is intended to discourage >> new codes in favor of enhanced reply codes and/or new text with >> existing codes. > > The document should create an IANA registry for SMTP reply code > points. This is straightforward. I have attached some suggested XML. Ticket #118. >> > Section 4.2.2: OK, but what are these "Functional Groups"? Some >> > sort of subheadings (not necessarily part of the hierarchical >> > numbering) are needed. >> >> These groupings, and the use of that terminology without a further >> breakdown or description or subheadings, come straight out of RFC >> 821. That is more than 40 years with no evidence that it has been >> the source of a problem. It might be that the difference in >> formatting between RFCs 821 (and 2821) and their successors is >> helpful. The formatting in the current document is identical to >> that in RFC 5321 (2821 followed the 821 model but was probably >> prepared using xml2rfc version 1 (i.e., the RFC 2629 version) and >> translated to nroff). I have figured out how to make xml2rfc >> restore that earlier layout but, if we want to do that, the WG is >> going to need to very carefully review the new groupings since >> there are codes that do not appear in RFC 821 or even in RFC 2821 >> and the ordering of some of the existing codes was changed (even >> between 2821 and 5321). > > Thanks for at least re-establishing the groupings. This probably > raises the utility of this section to just above zero. If the groups > were actually labeled with their functional theme, it would be even > better. Note Ticket #102. >> > Section 4.3.2, for EHLO 450 says "see note above" but it seems >> > unclear what note how far above. >> >> The page break in this version of the draft is the problem. If you >> scroll up to the bottom of page 60, you will see the note. Given >> alternate rendering forms for RFCs, etc., I've tentatively changed >> the note to read 'see note immediately above under "CONNECTION >> ESTABLISHMENT"'. Does that work for you and everyone else? > > It's better. > Maybe it's just me but the problem is that when I read "see note > above", I think of a free-standing piece of text and look above for > "Note:" or the like at/near the left margin. Failing to find that, I > might look for a "Note " embedded in other text. Failing both of > those, I give up. I think the best thing would be to duplicate the > parenthetical annotation. If not that, then maybe "450 (see > parenthetical annotation to 450 immediately above)". Open a ticket on this if you like. >> > Section 4.5.3.2: It seems confusing that the first paragraph >> > talks about "each buffer of the data transfer" but there is only >> > one mail data buffer at the server for a particular mail message >> > and the subject of 4.5.3.2.5 uses "block" while the contents of >> > 4.5.3.2.5 uses "chunk" and "TCP SEND call"s. >> >> I need WG advice as to what, if anything, to do about this. And I >> hope someone will correct me if the explanation below is not quite >> right. >> >> While I'm fairly certain that no one anticipated it when RFC 821 >> was being written, there are server implementations out there that >> need to transfer the incoming mail message somewhere else -- >> relaying it or putting it into a mailbox or non-SMTP delivery >> queue. They will process received commands through DATA, but will >> then (or soon thereafter) open the next connection in sequence, go >> through EHLO, MAIL, RCPT, and DATA commands, and then possibly >> start sending out message content while the content from the >> original message is still arriving. In addition to reducing the >> time between message sending and delivery, this allows those >> implementations to report most errors from the destination site >> while the incoming connection is still open (i.e., without later >> constructing bounce messages that can pose their own problems in >> the contemporary Internet, including security-related ones). If >> that isn't clear, ask and I'll try to draw a picture. > > It's great that there are pipelined mail servers. All I'm bothered > by is the word "buffer" because I see this use as somewhat > conflicting with other uses of "buffer" in the document. Why can't > it say for "each chunk of" or for "each block of" or anything other > than for "each buffer of"? Noting that we have a specific SMTP extension for "chunking", I see some risk of creating even more confusion. Open a ticket if you like. >> > Section 8.1.1.1, last paragraph: The existing IANA model is >> > intended to minimize IANA "judgement" and, in general, make >> > IANA's job mechanical. Since the idea is to avoid name >> > conflicts, this document should request one table with the >> > assignment model indicated in a column or the like. >> >> A general comment about the many suggestions about Section 8 in >> this review: >> Perhaps, in my writing and discussions with the WG over the almost >> five years we've been struggling with this document (including a >> few drafts in the year before the WG was chartered), I'm still >> suffering from some unreasonable nostalgia for an IANA that >> operated somewhat independent of the IETF/IESG and actually made >> decisions (even some the IETF didn't like). At the same time, I've >> got considerable respect for the people who are there now and for >> their knowledge and insights about registry organization, >> parallels with other registries, and so on. Yes, the IETF can, >> these days, tell them exactly what to do and expect them to do it >> unless they clearly explain why it is impossible. But at least >> IMO, that does not mean we need to assume they are not competent >> to make educated judgments. > > I don't recall ever seeing a draft which explicitly orders IANA to > do something in so many words. They are almost always politely > worded as requests, which gives IANA plenty of room to push back if > they perceive some difficulty or wish to provide some advice. But > it is also normally the case that, except for the very unusual case > of something like an RFC about IANA Considerations, like RFC 8126, > the WG/author/AD/... understand the technical aspects of what is > being documented much better than IANA does. Noted. See below. >> That section has been carefully reviewed with IANA and the text >> mentioned above is not one of their reported concerns. The IANA >> comments (see the datatracker entry if you are interested) point >> out that that there are extensive interactions with other >> registries and cross references. There are also, IIR, other >> registries that are split depending on how things got there, e.g., >> separating provisional and permanent registrations. One thing I'm >> sure of --and another motivation for that language -- is that >> figuring out exactly how registries should be organized is not a >> core competence of the WG. So, while my personal instinct matches >> yours, I think this document, at this point, should say as-is for >> now. > > Ok. >> My personal hope is that the IESG and IANA can work out whatever is >> best to do during IESG review or before publication and this text >> can then be adjusted to reflect what is being done or has been done >> rather than giving advice or talking about the future. > In my experience, IANA and the RFC Editor cooperate to re-write the > IANA Considerations section, if it requests any IANA actions, to > show what has been done. It is rare for the IESG to spend any > cycles to "work out" something with IANA. Our experience seems to differ somewhat, especially since IANA has become increasingly dependent on the IESG to initiate or review proposed registry changes, etc. >> > Section 8.3, first line: Should be phrased as a request to IANA >> > such as "IANA is requested to reorganize the Mail Parameters >> > Registry Group as follows:" >> >> All of Section 8 is, formally, requests to IANA. While that change > > Then perhaps text should be inserted between the 8. header and the > 8.1 header. Maybe "The following subsections present this document's > requests to IANA." See above about this having been checked and discussed with IANA at some length. >> would not be harmful, see above. I hope the final RFC can simply >> reflect what has been done. > > In my experience, the IANA Considerations section of the final RFC > gets re-written, completely re-written if necessary, by IANA and the > RFC Editor, so it *always* reflects what has been done. >> > Section 8.3, item (1) and Item (4): "should be" -> "is". >> >> But they aren't. These are more requests (or, if you prefer, >> instructions to) IANA. Either "is" is appropriate only after the >> changes are made. The alternative is that we should anticipate >> those changes being made and not be talking about requests at all. > > In general, drafts should be worded so they are true when issued as > an RFC. That's why drafts don't say "It is proposed ..." > throughout. It is entirely reasonable to say "The following is a > request to IANA: ... x is changed to y ..." or the like. > > But since this is all going to get re-written by IANA anyway, I > should have classified this as a nit, not an issue. It's not > important enough to argue about. Agreed. Or, if it is, this is not the right place. >> > Section 8.3, item (2): Should not tell IANA to "consider" >> > anything. Either ask IANA to do it or don't ask IANA to do it. >> > This is all going to get massaged in connection with IESG/IANA >> > review anyway. >> >> See above. It is not clear that the exact organization of >> registries is within scope for the WG. The text was written on the >> assumption that IANA might have had reasons (even long-forgotten >> instructions) for organizing and labeling things the way they are. >> Some of these registries and their contents and organization date >> from very long ago, were not specified by the IETF, and, if there >> are notes or memories about particular choices of language and/or >> organization, neither I nor the WG have been given access to them. >> But the > > This is the first time I have heard the theory that IANA might be > the custodian of Ancient Occult Knowledge and Manuscripts that they > conceal but use to make wise decisions if decisions are specifically > deferred to them. No comment. >> important thing, IMO, is the "massaged" part (consistent with my >> comments above). I'd prefer to focus on getting the document >> approved, working with IANA to get the registries organized in a >> reasonable way and all of the associated loose ends tied up, and >> then adjust the document to reflect what happened rather than >> spend time modifying the document over terminology, requests, etc. > > Ok. >> > And what IANA cares most about is clarity and lack of ambiguity. >> >> Yes, and, as mentioned above, IANA has already done a Last Call >> review of this document and they didn't raise any of the concerns >> about Section 8 that you are identifying. > > IANA will review it again after changes to resolve IESG ballot > comments... Of course. >> > Section 8.3, item (3): It is generally not IANA's job to review >> > registry names to judge whether they are "appropriate" which >> > seems like a technical question rather than a clerical question. >> > Perhaps IANA could chime in on whether a Registry name was >> > excessively long or confusing similar to another registry or the >> > like, but if you want to change the registry name, just request >> > IANA to do that. (Actually, with the change above, all these >> > points are part of a request to IANA.) >> >> Yes, and see above although I would argue that, when there are >> either many interrelated registries or ones with complex >> structures, it is reasonable to expect IANA to have opinions about >> consistency of naming, structure, etc. > > Ok. >> > Section 8.3, items (...): Changes in line with the above. >> >> See above. >> >> > Section 8.3, items (5) and (6): suggest combining these so it is >> > clear what "Those bullets ..." refers to. >> >> I think combining them would create other problems (and, again, >> IANA has already reviewed this and not objected). How would you >> feel about changing "Those bullet point fields" at the beginning >> of (6) to "The bulleted items identified immediately above"? I've >> tentatively made that change to the working draft. > > Ok. >> > Section 8.3, item (9): Appears to have an open technical question >> > that should be resolved. >> >> Whoops. No idea how that slipped through except that the WG has >> sometimes been moving too slowly or intermittently. I've made a >> note in the working draft. WG?? See Ticket #103, including proposed resolution >> > Appendix D: I think this one paragraph of material should be >> > somewhere in Section 3.7. >> >> This was Appendix E of RFCs 5321 and 2821, with rather clear >> decisions made at RFC 2821 time to put it there. I'm reluctant to >> move it given the "has not caused identified problems in 23 years" >> principle, at least without either evidence of harm, clear WG >> consensus, or both. > > I have re-read Section 3.7 and Appendix D. It continues to be > obvious to me that the material in Appendix D belongs in Section > 3.7 and that moving it there would improve the document. Can't you > at least add a reference to Appendix D in Section 3.7? Ticket #119. Thank you! john -- last-call mailing list -- last-call@xxxxxxxx To unsubscribe send an email to last-call-leave@xxxxxxxx
