Message contains NUL characters ...

[John Fawcett]

Greg A. Woods wrote:
> [ On Thursday, May 12, 2005 at 00:30:46 (+0200), John Fawcett wrote: ]
> 
>>Subject: Re: Message contains NUL characters ...
>>
>>nothing actually breaks by removing the nuls. The messges weren't
>>readable before and may not be readable afterwards.
> 
> 
> You cannot know that they were not readable, and/or that they were not
> signed with the NULs in place.
I meant they are not readable in cyrus: at least not without my
patch :-)
> 
> As I said, finding a NUL embedded anywhere in the data is usually an
> indication of either a serious bug on the sending side, or at least an
> attempt to send opaque binary content of some type (which should have
> been encoded somewhow and thus also indicates either a serious bug on
> the sending side; or else it indicates malicious intent on the sending
> side).
> 
> The opaque binary data containing NULs could well be quite valid and
> useful and usable from both the sender's and recipient's point of view
> -- they may simply be expecting the e-mail transport to be binary
> transparent and are not being helped by proper mailer software that
> would have properly encoded their content for them.  Of course when the
> transport involves SMTP (and IMAP), it is not necessarily transparent.
I think we all agree that nuls don't belong in email. There are various
options for what to do about it. The current situation is the worst:
MTA lets them through, cyrus rejects.

> However if you silently and destructively remove the NULs from the
> stream without giving any indication to anyone of what you've done then
> you've definitely "broken" the message.  This is morally equivalent to
> blindly and destructively stripping off the 8th bit in order to
> "convert" (i.e. pretend to convert) a message to ASCII.
> 
> So, the only safe and sane thing to do when a NUL is found anywhere in
> SMTP data is to reject the whole message immediately with a fatal error
> response indicating the problem so that the sender is informed as
> quickly as possible that their content cannot be sent as-is.  (The
> sending MTA should have properly MIME-encoded the message if it didn't
> want to bounce it, or have it rejected.)
The current behaviour of cyrus (reject) produces bounces that may go
to the wrong people.
To be honest I throw away so many spam messages today without every
reading them that the idea of someone wanting to send nul characters
and needing to be advised as quickly as possible of the problem so
as to be able to resend them to me correctly is just not an issue.
I would apply the same logic as for the spam: if someone sends me
something and doesn't get a reply, it's up to them to contact me.
They cannot assume I read the message. It may have been filtered out
for whatever reason.
> 
> (Someone should make sure the Cyrus install guide strongly recommends to
> properly configure the MTA of the user's choice so that such it rejects
> invalid content, and to choose an MTA that can do so!)
> 
Rejecting at the MTA is one option as is stripping in cyrus or the MTA.
I've made proposed patches for all these options (configurable).
Then people can chose (if the patches are accepted of course),
otherwise they may have more limited options. Only one of
these is needed to solve my original problem. Given that I use postfix
my preference is also for the postfix enhancement.

John

---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html