On Mon, Feb 9, 2009 at 8:53 PM, Carlos Horowicz <carlos.horowicz@xxxxxxxxx> wrote: > On Mon, Feb 9, 2009 at 9:02 AM, Bron Gondwana <brong@xxxxxxxxxxx> wrote: >> On Mon, Feb 09, 2009 at 10:24:38AM +0000, Ian Eiloart wrote: >>> >>> >>> --On 7 February 2009 02:36:36 -0200 Carlos Horowicz >>> <carlos.horowicz@xxxxxxxxx> wrote: >>> >>>> I'm wondering what to do in a live system with may be hundreds of >>>> thousands of these strange e-mails already in users´ mailboxes, >> >> My god, that many? Nuke it from orbit. Only way to be sure. >> >> (alternative plan, grep for the bogus files, unlink them and >> reconstruct the mailboxes - cheaper and less radioactive) > > I have already done that (find ... -exec grep ...) but it took days to > scan entire volumes, and in this case I already knew which pattern to > look for. > >> >>>> Should imapd be patched so that it just ignores the repetitions , both >>>> when building cyrus.cache and when it returns the headers to a client >>>> ? or should imapd really modify the original e-mail by stripping >>>> unnecessary/illegal headers and store a cleaned-up version ? >>> >>> It shouldn't be modifying messages. It should handle such messages >>> elegantly. Ignoring repetitions (beyond a threshold of repeats) seems the >>> most sensible option. However, failing to report them to a client could >>> cause confusion, so a threshold should be reasonably high. Of course some >>> headers are supposed to have multiple instances... >> >> Ditto with that. This patch ignores repetitions beyond a threshold. It >> turns out that ignoring specific headers separately is hard[tm], but >> just stopping parsing them after a count is way-easy. >> >>> Alerting the system administrator to the existence of such bogus messages >>> seems like a good idea, too. Perhaps through the logging system. >> >> Yeah, good point. Allow me to add that. >> >>> If you don't want a particular message in the system, then it should not >>> be accepted by LMTP or by any IMAP message creation mechanism. >> >> Ditto. Gosh. That makes 3 tunables. The gods of tunable >> non-proliferation will want my head for doing this: >> >> maxcacheheaders_warn = 500 >> maxcacheheaders_skip = 1000 (same as the current patch) >> maxcacheheaders_reject = 2000 >> >> Sound like reasonable defaults? I'm tempted to make the _reject be '0' >> (don't reject) by default. > > > Is this patch intended to be also part of reconstruct ? 'cause this > would catch many other situations, like restores, spool rebuilds, and > even scanning imap spools. I was thinking of reconstruct just for the sake of warning the admin. for instanace in syslog(LOG_ERR, "ERROR: message has %d header lines, not caching any more", max_skip); how can the message be identified so the admin can cat upon ? > >> >> Bron. >> > > Carlos > ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
