Message contains NUL characters ...

[Greg A. Woods]

[ On Thursday, May 12, 2005 at 09:38:03 (+0200), John Fawcett wrote: ]
> Subject: Re: Message contains NUL characters ...
>
> I meant they are not readable in cyrus: at least not without my
> patch :-)

Well to be even more pedantic, they're not storable by Cyrus.

Were they to be delivered to some other system they may well be stored
and readable and valid.  :-)

> I think we all agree that nuls don't belong in email. There are various
> options for what to do about it. The current situation is the worst:
> MTA lets them through, cyrus rejects.
>
> The current behaviour of cyrus (reject) produces bounces that may go
> to the wrong people.

Neither of those are problems with Cyrus.  They are problems with broken
and/or mis-configured MTAs that some folks have unfortunately chosen to
use with Cyrus, perhaps through being mis-informed that the chosen MTA
should work well with Cyrus.  As you've discovered that is simply not
true of Postfix "out-of-the-box".


> Rejecting at the MTA is one option as is stripping in cyrus or the MTA.

Stripping "arbitrary" characters in either the MTA, or worse in Cyrus,
is, as I said, the moral equivalent of destructively stripping the 8'th
bit off all bytes in order to "convert" the message to ASCII.

I.e. it would be very wrong.

Conversion or rejection are the only valid choices.


> I've made proposed patches for all these options (configurable).

That configurability is the worst part -- software which is intended to
be _operated_ by non-expert users should not give them so much rope that
they can do very obviously wrong things, especially not in such a way
that the users they affect cannot know what is happening or why.

The right thing to do is to fix the (arguably broken) MTA so that it
always does the right thing and so that it can only do the right thing.

Postfix design, as in this example, was originally misguided into
believing that it would be a good thing to try to be a fully binary
transparent e-mail transport.  However experience has shown that this
could only work in an ideal world where every other e-mail tool involved
in sending, transmitting, and reading e-mail were also fully binary
transparent while at the same time being intelligent enough to know how
to automatically and properly classify the content it handles.  Witness
the evolution of a full MIME handling engine inside Postfix, as well as
recent options such as the "strict_7bit_headers" and
"strict_8bitmime_body" configuration flags.

For all intents and purposes a really simple and obvious fix to modern
Postfix would be to treat NUL as if it were an "8-bit" character
(i.e. so that it will be rejected or converted as necessary) and to make
sure that any Postfix install used with Cyrus has "strict_8bitmime"
turned on and that "disable_mime_output_conversion" is kept turned off
(and make sure that the Cyrus LMTP daemon does not advertise 8BITMIME
and that Postfix does right MIME conversions when sending to LMTP).

-- 
						Greg A. Woods

H:+1 416 218-0098  W:+1 416 489-5852 x122  VE3TCP  RoboHack <woods at robohack.ca>
Planix, Inc. <woods at planix.com>          Secrets of the Weird <woods at weird.com>
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html