> From: "Robert G. Brown" > ... > Or, mark for later accept/reject decisioning AFTER the SMTP server per > se, in the filter pipeline between the server and the mail spool of the > addressee. Spam assassin does the right thing already (and this is > exactly what it does). ***NO***! Except when run as a milter or otherwise during the SMTP transaction, SpamAssassin does the WRONG thing. As run almost everywhere, after the SMTP transaction, SpamAssassin can only iether silently discard spam or generate new spam by sending bounces to innocent people. > > A better positition is that everything should be logged, particularly > > including discarded mail, ... > Logging a message you reject is nearly a waste of time. Based on my experience, people running ISPs or other large mail system strongly disagree with your position. Besides, I intentionally wrote about logging ***discarded** mail. Many institutions do turn off logging of greylisted messages, reduce the default per-message logging limit of 32K Bytes, or delete log files far sooner than the 14 default in the DCC source. Still, logging is seen as vital to be able to answer questions about which messages were filtered and why, including being able to say "that message was never sent" or "substantially identical copies of that message were sent to 310 other users here and 433,797 users elsewhere; it was spam." > In order to > recover the message (as you note, nobody ever looks at the logs, which > are VERY LARGE for a busy mailer and beyond human capacity to scan), I said nothing about humans scanning everything. Besides, giving users the sense that they can see what's happening with spam filtering on their mail and control it is a requirement for getting users to accept filtering. > >.. > This is where, and why, I take issue with filtering and discarding at > the level of the SMTP server, unless the accept/reject decision can be > made with 100% precision (no false positives, no false negatives, and it > may not be good even then because MY idea of the correct basis for the > decision may not be the same as YOURS). What you describe is a broken version of what I advocate, if you consistently look at your personal log of rejected mail. Your version is broken because a "reject decision" after the SMTP transaction must at least sometimes result in sending spam to innocent people. > ... > It's not that filtering based on non-header-linked aspects of content is > or isn't a good idea in some cases. It is that it has no business being > in the specification of TCP. ... > pure chance have a byte sequence like SEX that caused it to be rejected > >.. Nice straw man. I've never heard anyone with a taint of technical clues talk about looking for "SEX" in raw TCP segments. > ... > For nearly all filtering programs, it is too easy to create a message > that is filtered but shouldn't be. ... You evidently lack experience with the filters used by commercial institutions. Commersical SMPT servers cannot tolerate false positives (legitimate rejected/total legitimate) of more than 0.01%, and even that's pushing it. The design requirement for filtering mail on which money depends is that false positives must not be much worse than the underlying SMTP error rate due to problems such as full disks and broken DNS servers--not to mention mail recipients too quick to delete. A lot of current talk about false positives is self-serving nonsense from such as the Direct Marketing Association. Manual spam filtering also has false positives. A human suffering a common spam load of 100 spam/day has trouble not deleting legitimate mail. My filters are rejecting about 300 spam/day sent in my direction, 12227 in the last 40 days. Mechanical filtering even with a significant false positive rate can reduce the overall false positive rate. > ... > SMTP was designed to permit reasonably RELIABLE (simple) transport of It would be good to skip the networking 101 tutorials. Those of us who don't know all of that about TCP, SMTP, CSMA/CD, etc. often thanks to decades of personal experience should apply elsewhere to learn the basics. It is may hard to imagine how old farts like me see such tutorials, but please try. I've been receiving email as "vjs" since 1968. > It seems to me to be highly unacceptable to attempt to insert > content-based accept/reject decisioning in at this PROTOCOL level in the > delivery process. That use of "level" confuses me. It does not seem to conform to ISO OSI architecture. > ... > reliable transport mechanism for important messages. Filtering it "for" > me according to ANY CONTENT-BASED RULESET risks discarding at least some > messages that are not correctly classified when they are rejected. > Important messages can be lost. Bad things can result. Who is > responsible when this occurs? Who do I get to sue? You don't get too sue anyone, because a reasonably designed system lets you choose to do all of your spam filtering manually or in your personal MUA. The only caveat is that your account should be terminated for network abuse if whatever mechanisms you choose bounce very much spam to innocents. That many filtering systems are junk including not allowing per-user preferences is as illuminating as the fact that spam flogging spam filters is common. All I can see from either is that hard problems attract charletans and would be gurus. > ... > It is perfectly reasonable for you to add content filters that YOU > control ABOVE this transport layer. If you want to hire a secretary to > open all of your mail for you and sort it and reject all the > advertisements, you can. .... You are welcome to require manual filtering for your mail, but you have no standing to dictate to others at other institutions. "Transport layer" seems like a misuse of the term. It certainly conveys nothing to me. > With that all said, there are tools that ALREADY provide the kind of > content level filtering mentioned above. The better ones do not > themselves discard or bounce any mail to any user. People with experience in this field never say never. > They simply SCORE > THE CONTENT with regard to its likelihood of being spam (or a virus) on > the basis of a whole battery of tests. Scores that exceed a given > threshold can easily and automatically be rejected or binned for a > ... All of that can be done during the SMTP transaction just as effectively as after. Rejecting spam after the SMTP transaction should be and in many quarters already is seen as network abuse requiring at least a stern warning. Vernon Schryver vjs@xxxxxxxxxxxx