On Wed, 17 Mar 2004, Dean Anderson wrote: > On Wed, 17 Mar 2004, Robert G. Brown wrote: > > > a) Preparse the header so that the entire delivery path is the primary > > content of the message, with the message itself added (header intact) as > > an attachment. > > This is true now. Many people don't know how to parse the headers, but it > obeys a specific syntax and is machine parseable. Of course. To both -- many people don't know how to parse the headers. I'd estimate some 499,950,000 of the half a billion users of mail. So the CAPABILITY of parsing the headers exists, but not even AV vendors (who should know how and know better) do it. > > b) Return the complaint as mail to postmaster of the originating > > network with a special error code that would allow it to be counted and > > digested easily. One doesn't want to create a new kind of DoS attack on > > postmaster, and making it easy and automatic to return a complaint COUNT > > of some sort on otherwise identical content can help prevent that while > > making it easier for SP admins to deal with mounting complaint levels. > > This is possible now, and SpamCop does this. The problem with SpamCop is > that they alter the message, making it useless. SpamCop complaints are > routinely deleted as a result. I usually forward them on to customers > with an FYI that we don't consider such to be a valid complaint, but they > may want to be aware of what someone said. So what you are saying is, that this CAN be done and even is being done, but it isn't done correctly and consistently and isn't widely available. I agree. In fact, I think that this is potential work for the IETF -- define a way for it to be done correctly and consistently (which wouldn't be too difficult, I imagine) via a protocol. Then SpamCop could fix their tool, SpamAssassin could add a compliant interface, Joe's Spam Seal (not yet written) could be written to comply with the protocol and abuse@ or postmaster@ addresses could actually AUTOMATE a response process based on consistent completely reported problems. This lowers their cost and makes it easier and more likely that they'll take effective action. As you say, if you get insufficient information, there is little that you can do even with the best of will as the manager of an ISP with too little time and too many things to fill it. You probably don't have time or energy to engage in the "please resend your complaint with the full header and message attached" game (known to administrators everywhere, and not just in regard to mail:-). > > c) Work out what to do about relays and proxies, again to prevent > > man-in-the-middle DoS more than anything else. One site should not be > > able to generate mail that it "forwards" with fictitious headers and > > evil content so that it appears to come from some hapless but innocent > > network. > > Relays don't add ficticious headers, nor do they add evil content. They > may place their (true) headers on top of ficticious headers. They cannot > verify that the headers given are accurate. This is true of all relays, > open or closed. > > Proxies are another story. I don't know of any genuine reasons to run open > proxies, though closed proxies (web caches) are clearly useful. I don't > know of anyone claiming they are useful. But neither does that mean they > have no use. However, as a service provider, I would say this much: If a > customer found a useful reason to have an open proxy, then I would only > insist that they keep logs of its use. This is easy to place in a contract > or AUP. Again, precisely. So what this means is that if I set up a mail server and send a lot of spam from it all with a header that is forged UPstream from my host, I obscure my host as its true point of origin. I can pick on a hapless host somewhere far away and make it look like the spam originates from THEIR network, and one would have to compare traffic logs on the two hosts and/or intermediary hops to determine which of us is lying. This makes for an interesting "attack" -- send egregious spam purporting to come from somebody you hate or some network you are in competition with, in a messages with multiple relays. Somebody with a big view of the hand might finally figure out what was happening, but proving it would be, well, "difficult" doesn't begin to describe it, given that all the log data on both sides could be easily forged. Again, there might be work for the IETF here that could help IF this becomes a problem or as a preemptive measure now. There are ways to do this -- signing a key with a local secret and adding it to the message, for example -- that even if you didn't have a universal PKI for all participating hosts would make forging headers for openly fraudulent reasons easy to catch. You can forge the upstream route, but you cannot forge the upstream keys, and the last common host whose key can unlock their MTA-added tag is left holding the responsibilty bag. > > d) Take steps to ensure that SPs run a postmaster address, and maybe > > There is already a BCP for this. Rarely is this a problem. In the US. Try hitting the postmaster of 7.197.76.17, and good luck communicating with the postmaster of the brazilian relay in my previous reply. (This is picking nits -- actually I agree that what can be done largely has been done, but it would be lovely to be able to extend overseas with a little more ease, to get a bit poetic...;-) > > come up with things like Jeff Chase was proposing to continuously > > measure their responsiveness to spam/virus-class bounce messages and > > globally blacklist the worst (least responsive) offending SPs. Etc. > > How would you define "responsiveness"? Does it mean getting an > auto-responder message? Does it mean getting a message saying something > had been done? You cannot give out certain information about customers. > Basically, all you can do is send an auto-responder message and a notice > that the ticket was closed. That doesn't indicate what happened, nor does > it indicate who the spammer was, or if the ISP agreed they were a spammer. > Sometimes the action is obvious if, say, a website disappears. But how do > you know they took action against a dialup customer? You can't. No, I think that responsiveness has to be measured by the level of spam reported as originating within the site -- results. I don't think this is that difficult, actually. Vernon pointed out that once earthlink got serious about controlling spam, the reduction in traffic was very apparent. If the IETF has any role here (and it may not) it might be to codify the process of blacklisting ISPs that aren't serious as they are revealed. You've pointed out several times that abuse of blacklists is pretty easy as things stand. It shouldn't be. And people like you who have bad experiences with the previous non-process should be the ones who end up agreeing that whatever schema that might be proposed is fair and equitable and not likely to punish ISPs who are doing their best. > > Right now enabling SPs are insulated from any kind of RFC-based > > responses or complaints about spam because MUA's and MTA's have no > > predefined protocol for generating such a response in a constructive > > way. > > ??? I'm not sure what you mean. By the time you -read- a spam, it is > delivered, and the SMTP protocol has long since finished. Spam isn't the > only kind of abuse that an ISP responds to. The abuse@<isp> works pretty > well, since you can forward the complained-of message. There are some > things I'd like MUA's to do, such as include the whole headers(some MUA's > do, some MUA's don't), but that isn't an RFC issue, either. Why not? SPAM is a major networking problem. Reporting spam and viruses (at the MTA or MUA level, as filters can work at either one) is only going to be effective if it is easy to report spam and if the reports are consistent and contain the information needed to combat spam. Most MUA's cannot help users to report, and users definitely cannot help themselves. If MUA writers tried to add a reporting capability now, they'd botch it by coming up with a dozen different formats and effectively precluding the creation of automated tools on the other end to facilitate processing the complaints at minimal cost. Perhaps not an RFC, but a standard is most definitely needed, and it needs to be both articulated and discussed in an open RFC-like process. And there are already RFC's that address this, e.g. RFC 2635. It is just that this RFC reads like spam for the beginner (intended for users, a sort of anti-spam howto) and doesn't spell out anything like a uniform protocol for reporting spam TO the abuse@ address it suggests MTAs maintain. Lacking this (and due to the fact that nobody reads RFCs but techie nerds; others don't even know what RFC stands for) the document isn't horribly useful to a software designer wanting to actually design TOOLS that might be portable and consistent. > > Most complaints/bounces that are automatically generated by > > antivirus software or reported by humans (I've read plenty of both:-( > > are hopeless and de facto useless without several rounds of > > communications, and sometimes not even then: the humans don't even know > > what a mail header IS and often have no way of knowing or suspecting > > that the From address is bogus or sending in the real header so it can > > be parsed by the SP postmaster. Antivirus software developers should > > know better (damn it!) but even THEY don't bother to parse the header or > > include the header in the stupid bounces they generate, or validate > > any sort of correspondance between originating host and From address. > > Yes, it would be good to send the entire headers. > > But users should know that the from: header can be forged. They should > also know some other things about email and the internet, such as don't > open attachments you aren't expecting. If you get an unexpected > attachment, ask the person if they sent it. This is an education problem. Users SHOULDN'T have to know ANYTHING but that if they fill in an address here and write a message and click this little button it goes where you want it to. The fact that they do is a failure in design. It may be a failure we cannot fix, but it is a failure. Kids send mail. Old ladies send mail. Stupid people can send mail. The process should be robust and idiot proof AT THE TOP LEVEL, although that is indeed not an IETF issue, it is a software design issue. The IETF issues are to places where by specifying particular ways of handling certain problems according to a clearly spelled out protocol, they enable software designers to write software that FIX these flaws that is consistent and interoperable. Reporting spam would clearly be much more effective if it were done in a standardized way. If standardized report formats existed, software designers would have a clear roadmap for adding reporting features to MTAs that my kids, that old ladies, that stupid people could use without even knowing that a "mail header" exists. It enables other software designers to write tools for the other end to preparse traffic to abuse@ addresses into efficient, condensed reports so that instead of getting 1000 different messages reporting one spam incident (which yes, one would be tempted to pipe into /dev/null) one gets ONE message reporting 1000 different complaints of ONE spam incident (with many targets). Get it to where ISP admins can whack those moles efficiently and easily, and they might just have at the mole-heads in their subnets with a will and a way. rgb > > --Dean > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@xxxxxxxxxxxx