> >surely we in the IETF should be able to do better than to have our mail > >servers filter incoming mail based on completely irrelevant criteria > >like whether a PTR lookup succeeds! > Spam filtering is sort of like chemotherapy, the difference between > the good and the bad is pretty small, and the trick is to find > measures that will kill the disease without killing the patient. It's > entirely a matter of statistics, not fundamental design. And sort of not like it at all. For one thing, the mutation rate in the spam world is much higher. And that brings us to the real issue here: Whether or not a given technique, which was very effective indeed at one point, is effective enough now to continue using. > I can assure you that in the outside world, the amount of legitimate > mail that comes from no-PTR hosts these days is infinitesimal. And in my experience the amount of spam coming from such sources, while much greater, is now distinctly in the minority in terms of all incoming spam. More specifically, since rather than rejecting mail coming from IPs with no PTR entry I instead use this as a weighted factor in computing an overall spam score, I was able to examine today's spam to see to what extent this check is helping or hurting. Turns out it's doing neither - I could eliminate it entirely and my false negative rate would not go up by even a single message. And right now the weight is sufficiently low that it isn't causing any false positives either. This defense definitely was very effective once upon a time, but the world has adapted. In this particular case what seems to have happened is that because of various PTR record checks a lot of ISPs now unconditionally assign PTR records to every address, e.g., cpe-76-171-209-109.socal.res.rr.com and 173.sub-75-217-87.myvzw.com were associated with the last two dynamic IPs I've used. A few years ago it was common to get assigned IP with no PTR records at all, nowdays much less so. And this has now led to a new issue: We now have various systems out there that are now attempting to parse PTR results order to try and figure out the difference between static business addresses and dynamic residential addresses. And as you might expect, they often get it wrong. Now, of course one advantage of this check is that it can be done before you even accept the message. But the tradeoff then is that this is giving it infinite weight. > It's one of the filtering rules with the lowest false positive rates. > Other than temporary glitches like the 6-to-4 one, the only place > where I see problems is from auld pharts like us whose mail systems > have been working just fine since the 1980s, and who out of a weird > sort of principle refuse to make changes to bring them in line with > modern practice, even changes that are compatible with equally ancient > STD documents. > So, yeah, spam stinks, but it's not going away, and arguments that you > shouldn't use a technique today because it didn't work in 1998 don't > cut it. Nor does continuing to use a technique that has outlived it's usefulness. IMO the utility of this particular check peaked a few years ago and we're now on the down-slope in terms of utility. And in the specific case of the IETF, where we want to encourage people to use our servers to experiment with IPv6 and where there are bound to be a lot of PTR record issues, I think an absolute block on the basis of no PTR record is totally inappropriate unless it can be shown to be singularly effective in dealing with spam and that our spam would increase dramatically without the rule. I doubt very much this can be demonstrated. OTOH, using it as a component in computing an overall spam score may still make sense, especially if the use of IPv6 could also be factored in. Ned _______________________________________________ Ietf@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf