Hi all, Unfortunately, freedesktop.org's job of sending mail to a huge number of people whilst pretending to be other people, has just got even harder than it was. fd.o can no longer (at least for the moment) send mail with the From addresses of DMARC/DKIM/SPF-protected sender domains. When we try to do so, large providers reject the mail, despite DMARC records explicitly specifying that the mail should be accepted. Worse than that, not only does the immediate delivery attempt fail, but it punishes our sender reputation enough that _other_ mail delivery fails: after we hit a DMARC-related failure, large providers often refuse delivery attempts for mail from non-DMARC-protected domains. As a result, if the sender domain has a DMARC record, we rewrite the >From address to be the list's address, preserving the original author in Reply-To. I'm chasing this up through a few channels, but in the meantime, please be aware that the From address on mails may no longer be accurate. If you are attempting to apply patches with git-am, please check that the commit author is not 'Person Name via listname-devel <listname-devel@xxxxxxxxxxxxxxxxxxxxx>'. If you are replying privately to a list mail, please be _very_ careful that you are replying to the original sender (in Reply-To) and not the list itself. Cheers, Daniel ---------- Forwarded message --------- From: Daniel Stone <daniel@xxxxxxxxxxxxx> Date: Mon, 11 Feb 2019 at 23:38 Subject: PSA: Mailman changes, From addresses no longer accurate To: <freedesktop@xxxxxxxxxxxxxxxxxxxxx>, <sitewranglers@xxxxxxxxxxxxxxxxxxxxx> Hi all, We have hit another step change in aggressive anti-spam techniques from major mail providers. Over the past few days, we saw a huge spike in the number of mails we were failing to deliver to GMail and outlook.com in particular. It looks like it is now no longer acceptable for us to break DMARC/DKIM/SPF. These are DNS-based extensions to SMTP, which allow domains to publish policies as to who should be allowed to send email on their behalf. SPF provides source filtering, so e.g. freedesktop.org could specify that no-one should accept mail with a From: *@freedesktop.org unless it came from gabe.freedesktop.org. Mailman completely breaks this: if I specified a filter only allowing Google to send mail for @fooishbar.org, then anyone enforcing SPF would reject receipt of this mail, as it would arrive from fd.o with my From address. DKIM allows domains to publish a public key in DNS, inserting a header into mails sent from their domain cryptographically signing the value of named headers. Mailman breaks this too: changing the Sender header (such that bounces get processed by Mailman, rather than sending a deluge of out-of-office and mailbox-over-quota messages to whoever posts to the list) can break most DKIM signatures. Mailman adding the unsubscribe footer also breaks this; we could make it not add the footer, but in order to do so we'd have to convince ourselves that we were not decreasing our GDPR compliance. DMARC ties the two together, allowing domains to specify whether or not DKIM/SPF should be mandatory, and if they fail, what action should be taken. Despite some domains specifying a fail action of 'none' (receiving MTA to send an advisory report to a named email address, but still allow the email), it seems that popular services still interpret 'none' as 'reject'. As a result, Google in particular is dropping some number of our mails on the floor. This does _not_ just apply to mails which fail DMARC/DKIM/SPF: every mail we send that fails these filters (which is a lot - e.g. Intel and NVIDIA both have SPF) decreases our sender reputation with GMail and causes it to reject legitimate mails. I've reached out to Google through a couple of channels to see what we can do to increase our delivery rate to them. In the meantime, if your mail is hosted by Google, or Outlook, and you think you're missing mails - you probably are. Mailman has also now been reconfigured such that if it spots a potential DMARC violation, it rewrites the From address to be @lists.freedesktop.org, keeping the original author in Reply-To. It also strips DKIM headers. This seems to be about the best we can do, unless and until the major mail service providers offer us some alternate way to send mail. If you are replying privately to someone, you should check very carefully that you are replying to them and not to the list. Unfortunately we don't have a good answer in the long run. The latest published RFC at https://tools.ietf.org/html/rfc6377 suggests that there are no good solutions. If anyone is blessed with an abundance of time and familiar with the current email landscape, I would love to talk to you and get your help to work on this. Unfortunately we don't have the manpower to actually properly monitor email; it can often take a collapse in successful-delivery rates for us to notice. Ultimately, I suspect the best solution is to move most of our discussions to dedicated fora like GitLab issues, or something like Discourse. Fundamentally, the thing we're trying to do (send email to thousands of people at a time using a fake From address) is ... kind of the opposite of what the 2019 Internet wants us to do. Every few months the major providers drop more of our mail as they become more aggressive with spam, and every few months their userbase increases by a non-trivial amount. We've done a lot of work on our email infrastructure, and are doing our best to be a responsible citizen within the constraint of having to launder mail and forge identity on an industrial scale, but it's coming to the point where it just may not be possible to run such a service at such a scale anymore. This is before even considering our other issues with Mailman 2.x: no centralised identity management (mailing your passwords out every month ... ?!), difficulty of GDPR compliance (editing archives requires hand-editing every single HTML index, as there is no non-destructive archive rebuild), the flat-out bugs (e.g. the mesa-dev archives are usually missing half the messages), and the fact it's been abandoned upstream in favour of Mailman 3.x, which is not obviously better, nor is there a clear upgrade path to. Of course we do not have any plans to stop providing email any time soon, but it might be worth thinking about what you can do to reduce your dependency on email lists. At the current rate of degradation, it might be non-viable quicker than you'd think. Maybe this is unduly gloomy, but the entire internet's direction of travel has been away from services like Mailman, and its velocity is only increasing. Cheers, Daniel _______________________________________________ Intel-gfx mailing list Intel-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/intel-gfx
