How to make cyrus not change non US-ASCII characters to "X"

[Christian Schulte]

Alessandro Oliveira wrote:

> Is there any possibility to put this as an autoconf option?
>
> maybe something like: "./configure --enable-strict-headers | 
> --disable-strict-headers" would do the trick,
> or even better, a runtime configuration option !!!
>
> what are the REAL consequences of having 8bit characters in the 
> Subject header ?
>
> Thanks,
>
> Alessandro Oliveira
>
>>> A> Date: Wed, 20 Nov 2002 09:30:31 -0200
>>> A> To: info-cyrus <info-cyrus at lists.andrew.cmu.edu>
>>>
>>> A> Hi,
>>>
>>> A> I'd like to know if there is a way to avoid this cyrus behavior, my
>>> A> users are getting very angry with it.
>>>
>>> A> Thanks for any help,
>>>
>>> Yes, users do. And notice about strict 7-bit ASCII within headers
>>> does not help.
>>>
>>> Same problem using Cyrillic. I have not found anything else and
>>> used a simple hack. Locate the following 2 files in the source tree: 
>>> lmtpengine.c and message.c
>>>
>>> change them the following way ( by inserting `#if 0' and `#endif' ) 
>>> and re-compile the entire software.
>>>
>>> This eliminates the substitution unconditionally.
>>>
>>>
>>> lmtpengine.c:
>>>
>>> .....
>>>                /* ignore this whitespace, but we'll copy all the 
>>> rest in */
>>>                break;
>>>            } else {
>>>
>>> // HACK -- Violet
>>> #if 0
>>>                if (c >= 0x80) {
>>>                    if (reject8bit) {
>>>                        /* We have been configured to reject all mail 
>>> of this
>>>                           form. */
>>>                        r = IMAP_MESSAGE_CONTAINS8BIT;
>>>                        goto ph_error;
>>>                    } else {
>>>                        /* We have been configured to munge all mail 
>>> of this
>>>                           form. */
>>>                        c = 'X';
>>>                    }
>>>                }
>>> #endif
>>>                /* just an ordinary character */
>>>                body[off++] = c;
>>> .....
>>>
>>> and message.c:
>>>
>>> .....
>>>            else {
>>>                sawcr = 0;
>>>                blankline = 0;
>>> // HACK -- Violet
>>> #if 0
>>>                if (inheader && *p >= 0x80) {
>>>                    if (reject8bit) {
>>>                        /* We have been configured to reject all mail 
>>> of this
>>>                           form. */
>>>                        if (!r) r = IMAP_MESSAGE_CONTAINS8BIT;
>>>                    } else {
>>>                        /* We have been configured to munge all mail 
>>> of this
>>>                           form. */
>>>                        *p = 'X';
>>>                    }
>>>                }
>>> #endif
>>>            }
>>>        }
>>>
>>>        fwrite(buf, 1, n, to);
>>>    }
>>> .....
>>>
>> In 2.2 cvs you do this in message.c and spool.c
>
>
>
I thought that cyrus does not only replace non US-ASCII characters in 
the mail-header. If a mail with a specified enconding of US-ASCII gets 
send through LMTP all non US-ASCII characters in the mail body get 
replaced to "X" also! My users were not interested in this behaviour if 
that would only affect the header. The problem is that in e.g. germany 
all umlauts like ä,ü,ö etc. get replaced in the body if specified as 
US-ASCII. Of course the mail body specifies a wrong character encoding 
if sent as US-ASCII but containing umlauts but even a big 
free-mail-provider (www.web.de) in germany with millions of users sends 
his newsletter in US-ASCII but with umlauts! I got it done with these 
small source-changes but this will not change user from sending mail in 
wrong encodings :-(