Fwd: Huge header detection

[Bron Gondwana]

On Mon, Feb 09, 2009 at 10:24:38AM +0000, Ian Eiloart wrote:
>
>
> --On 7 February 2009 02:36:36 -0200 Carlos Horowicz  
> <carlos.horowicz at gmail.com> wrote:
>
>> I'm wondering what to do in a live system with may be hundreds of
>> thousands of these strange e-mails already in users´ mailboxes,

My god, that many?  Nuke it from orbit.  Only way to be sure.

(alternative plan, grep for the bogus files, unlink them and
reconstruct the mailboxes - cheaper and less radioactive)

>> Should imapd be patched so that it just ignores the repetitions , both
>> when building cyrus.cache and when it returns the headers to a client
>> ? or should imapd really modify the original e-mail by stripping
>> unnecessary/illegal headers and store a cleaned-up version ?
>
> It shouldn't be modifying messages. It should handle such messages  
> elegantly. Ignoring repetitions (beyond a threshold of repeats) seems the 
> most sensible option. However, failing to report them to a client could  
> cause confusion, so a threshold should be reasonably high. Of course some 
> headers are supposed to have multiple instances...

Ditto with that.  This patch ignores repetitions beyond a threshold.  It
turns out that ignoring specific headers separately is hard[tm], but
just stopping parsing them after a count is way-easy.

> Alerting the system administrator to the existence of such bogus messages 
> seems like a good idea, too. Perhaps through the logging system.

Yeah, good point.  Allow me to add that.

> If you don't want a particular message in the system, then it should not 
> be accepted by LMTP or by any IMAP message creation mechanism.

Ditto.  Gosh.  That makes 3 tunables.  The gods of tunable
non-proliferation will want my head for doing this:

maxcacheheaders_warn = 500
maxcacheheaders_skip = 1000 (same as the current patch)
maxcacheheaders_reject = 2000

Sound like reasonable defaults?  I'm tempted to make the _reject be '0'
(don't reject) by default.

Bron.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Max-header-lines-1000-by-default-to-avoid-supersi.patch
Type: text/x-diff
Size: 1336 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090209/c88bc59d/attachment.bin