From simon.matter at invoca.ch  Wed Apr  1 02:15:41 2009
From: simon.matter at invoca.ch (Simon Matter)
Date: Wed, 1 Apr 2009 08:15:41 +0200 (CEST)
Subject: upgrade from 2.2.x to 2.3.x
In-Reply-To: <1238537042.3680.26.camel@lin-workstation.azapple.com>
References: <1238438096.5672.170.camel@lin-workstation.azapple.com>
	<ba084055279a3026546616efbeb35d04.squirrel@webmail.bi.corp.invoca.ch>
	<1238537042.3680.26.camel@lin-workstation.azapple.com>
Message-ID: <5fb3e4bf57b262137e814a25d61bd150.squirrel@webmail.bi.corp.invoca.ch>

> On Tue, 2009-03-31 at 09:38 +0200, Simon Matter wrote:
>> > Planning on upgrading my server from...
>> >
>> > db4-4.2.52-7.3.el4
>> > cyrus-imapd-2.2.12-9.RHEL4
>> >
>> > to
>> >
>> > db4-4.3.29-9.fc6
>> > cyrus-imapd-2.3.14-1
>>
>> Looks like you are going from RHEL4 to RHEL5?
>>
>> >
>> > obviously, I can move the mail files but I'm also concerned with the
>> > various seen.db, mailboxes.db, annotations.db
>> >
>> > Do I just copy them over and use berkeley utilities to update them?
>>
>> Looks like you use my rpms on RHEL5 right? Then you may want to first
>> update cyrus-imapd on RHEL4 to cyrus-imapd-2.3.14-1. Then start and stop
>> cyrus-imapd on RHEL4 and transfer all cyrus data to RHEL5 and start it
>> up.
>> The RPM handles this by converting all berkeley DB's to skiplist in
>> shutdown and converting back as needed on startup. (I think the 2.2.12
>> rpm
>> already does this on RHEL4 but I'm not sure so just stop it and check
>> the
>> db files.)
> ----
> yes - yes - yes
>
> Yes, from CentOS 4 to CentOS 5
> Yes, from CentOS 4 cyrus-imapd package to Simon's package on CentOS 5
> Yes, shut it down, did 'rsync -ar' everything from /var/spool/imap

If you happen to run single instance store, make this 'rsync -aH' to
preserve the hardlinks.

> and /var/lib/imap, /etc/imapd.conf and /etc/cyrus.conf and generated new
> certs.
>
> No - it all failed at startup. Did chown cyrus:mail on both directories

That's interesting because I don't think permissions have changed between
both packages.

Simon

> recursively and it started up and seems to be running fine. Did a
> 'reconstruct -fr user.*' just to be safe...
>
> Thanks for the src packages and thanks for the reply but I didn't bother
> installing 2.3.x on my RHEL 4 and it doesn't seem to have mattered. I
> think I've almost done everything necessary in migrating my server to
> where I can shut down the old server now.
>
> Craig
>



From eddy.beliveau at hec.ca  Wed Apr  1 07:53:43 2009
From: eddy.beliveau at hec.ca (Eddy Beliveau)
Date: Wed, 01 Apr 2009 07:53:43 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <49D281FF.90305@kickflop.net>
References: <49D2403C.50908@kickflop.net>	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
Message-ID: <49D355C7.7010304@hec.ca>

-------- Message original --------
Sujet : Re: Weekly/Monthly record-keeping / maintenance?
De : Jeff Blaine <jblaine at kickflop.net>
Pour : Andreas Winkelmann <ml at awinkelmann.de>
Copie ? : info-cyrus at lists.andrew.cmu.edu
Date : 2009-03-31 16:50
> Andreas Winkelmann wrote:
>   
>> Am Dienstag 31 M?rz 2009 18:09:32 schrieb Jeff Blaine:
>>
>>     
>>> Every year or so, a user of ours reports a discrepancy
>>> between on-disk usage for their spool compared to what
>>> 'FETCH' is reporting (as implemented via imapdu.pl)
>>>       
>> What means "on-disk usage for spool" ? 
>>     
>
> What I mean, is, for one example -- a user is currently
> reporting that 'FETCH' (via the imapdu command) is showing
> 142 messages totalling 640KB in a folder that is actually
> completely empty on disk (except for cyrus.* files).
>
> reconstruct -r user.hername did not change what is reported
> via 'FETCH'.  It did update the cyrus.* files though.
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>
>   
Hi!

How about

reconstruct -r -f user.hername

Eddy

-- 
Eddy Beliveau
HEC Montreal
Montreal (Quebec)
Canada

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090401/3986a15f/attachment.html 

From craigwhite at azapple.com  Wed Apr  1 09:54:36 2009
From: craigwhite at azapple.com (Craig White)
Date: Wed, 01 Apr 2009 06:54:36 -0700
Subject: upgrade from 2.2.x to 2.3.x
In-Reply-To: <5fb3e4bf57b262137e814a25d61bd150.squirrel@webmail.bi.corp.invoca.ch>
References: <1238438096.5672.170.camel@lin-workstation.azapple.com>
	<ba084055279a3026546616efbeb35d04.squirrel@webmail.bi.corp.invoca.ch>
	<1238537042.3680.26.camel@lin-workstation.azapple.com>
	<5fb3e4bf57b262137e814a25d61bd150.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <1238594076.699.1.camel@lin-workstation.azapple.com>

On Wed, 2009-04-01 at 08:15 +0200, Simon Matter wrote:
> > Yes, from CentOS 4 to CentOS 5
> > Yes, from CentOS 4 cyrus-imapd package to Simon's package on CentOS 5
> > Yes, shut it down, did 'rsync -ar' everything from /var/spool/imap
> 
> If you happen to run single instance store, make this 'rsync -aH' to
> preserve the hardlinks.
----
ooops... don't know if this mattered but I didn't do that.
----
> 
> > and /var/lib/imap, /etc/imapd.conf and /etc/cyrus.conf and generated new
> > certs.
> >
> > No - it all failed at startup. Did chown cyrus:mail on both directories
> 
> That's interesting because I don't think permissions have changed between
> both packages.
----
and all the files/folders appeared to be owned by cyrus:mail but it
seems to have made a difference.

Craig


From jross at broad.mit.edu  Wed Apr  1 11:06:34 2009
From: jross at broad.mit.edu (Jesse Ross)
Date: Wed, 1 Apr 2009 11:06:34 -0400
Subject: cyrus won't recognize new partition
In-Reply-To: <200903312234.14517.ml@awinkelmann.de>
References: <a93d0eff0903300923i2c573770u971e986130d8434d@mail.gmail.com>
	<200903312234.14517.ml@awinkelmann.de>
Message-ID: <a93d0eff0904010806v56943cbey8c7beeabfa051b24@mail.gmail.com>

On Tue, Mar 31, 2009 at 4:34 PM, Andreas Winkelmann <ml at awinkelmann.de>wrote:

> Am Montag 30 M?rz 2009 18:23:57 schrieb Jesse Ross:
>
> > partition-imap06: /var/spool/imap06
> > partition-imap07: /var/spool/imap07
>
> Please check for invalid characters around the second line (imap07) in your
> imapd.conf. Maybe show a hexdump of the config file.
>


All the characters in that section are valid - here's the output from
hexdump -C:

00000270  6d 61 70 30 34 0a 70 61  72 74 69 74 69 6f 6e 2d
|map04.partition-|
00000280  69 6d 61 70 30 35 3a 20  2f 76 61 72 2f 73 70 6f  |imap05:
/var/spo|
00000290  6f 6c 2f 69 6d 61 70 30  35 0a 70 61 72 74 69 74
|ol/imap05.partit|
000002a0  69 6f 6e 2d 69 6d 61 70  30 36 3a 20 2f 76 61 72  |ion-imap06:
/var|
000002b0  2f 73 70 6f 6f 6c 2f 69  6d 61 70 30 36 0a 70 61  |/spool/
imap06.pa|
000002c0  72 74 69 74 69 6f 6e 2d  69 6d 61 70 30 37 3a 20  |rtition-imap07:
|
000002d0  2f 76 61 72 2f 73 70 6f  6f 6c 2f 69 6d 61 70 30
|/var/spool/imap0|
000002e0  37 0a 0a 23 20 55 73 65  20 74 68 65 20 55 4e 49  |7..# Use the
UNI|
000002f0  58 20 73 65 70 61 72 61  74 6f 72 20 63 68 61 72  |X separator
char|

thanks,
jesse
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090401/303f78c3/attachment.html 

From jblaine at kickflop.net  Wed Apr  1 11:10:42 2009
From: jblaine at kickflop.net (Jeff Blaine)
Date: Wed, 01 Apr 2009 11:10:42 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <49D355C7.7010304@hec.ca>
References: <49D2403C.50908@kickflop.net>	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net> <49D355C7.7010304@hec.ca>
Message-ID: <49D383F2.2010501@kickflop.net>

> How about
> 
> reconstruct -r -f user.hername

Nope.  FETCH/imapdu.pl reports 142 messages still.

From nodens2099 at gmail.com  Wed Apr  1 14:26:27 2009
From: nodens2099 at gmail.com (nodens2099)
Date: Wed, 01 Apr 2009 20:26:27 +0200
Subject: sendmail, lmtp, cyrusv2d, shared folders and case
Message-ID: <49D3B1D3.3040905@gmail.com>

Hi,

I'm trying to send mail directly to a shared folder, via the
plus-addressing syntax.

My user entry redirect to +Folder/SubFolder at domain.tld .

cyrusv2d mailer is called via the mailertable.

The ACL are ok.

It would work with a folder in lower case, but not if the folder
includes upper case characters.

I thought it was ok to have upper case after the + in the local part of
the address ?

I tried setting
"lmtp_downcase_rcpt: 0"
in imapd.conf, adding the u flag to the cyrusv2d mailer, but to no avail.

the +part is always converted to lower case.

initially I tried using the mrs_cyrus_mailertable approach but it was
even worse, as checking +folder/subfolder at domaine.tld with smmap returns
unknown user, even when the folder exists and has the "p" acl.

Any clue ?

I'd really like to avoid converting all existing folders to lower case
of possible, it would be painful.

Best regards,

-- 
Cl?ment Hermann (nodens)



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090401/d4cb601a/attachment.html 

From wes at umich.edu  Wed Apr  1 14:38:08 2009
From: wes at umich.edu (Wesley Craig)
Date: Wed, 1 Apr 2009 14:38:08 -0400
Subject: cyrus won't recognize new partition
In-Reply-To: <a93d0eff0903310733r9bc7a0ar6ee6dabb65e5912@mail.gmail.com>
References: <a93d0eff0903300923i2c573770u971e986130d8434d@mail.gmail.com>
	<20090330214659.GA8544@brong.net>
	<a93d0eff0903310733r9bc7a0ar6ee6dabb65e5912@mail.gmail.com>
Message-ID: <72CF3B00-C9AE-4A86-8DAC-D596B1B6F9D7@umich.edu>

On 31 Mar 2009, at 10:33, Jesse Ross wrote:
> Thanks for the suggestion.  The only thing that the mkimap script  
> does differently from the way I did it was to create a directory / 
> var/spool/imap07/sync./.  However, I still can't get cyrus to  
> recognize the partition!  The symptoms are the same; ownership of  
> directories set to cyrus, appropriate permissions as copied from  
> working partitions, configuration same as for the working partitions.

IMAP_PARTITION_UNKNOWN isn't associated with permission problems,  
typically.  For the create code path, you get it when there's a  
failure to convert the symbolic partition name to a path.

> Is there perhaps a way to get cyrus to log what it thinks the  
> problem with the partition is?

The number of cases that can result in IMAP_PARTITION_UNKNOWN is  
pretty small, I'd probably try inserting some logging into the  
process for those results.  It's either a bug or configuration  
oversight, I'm sure.

:wes

From wes at umich.edu  Wed Apr  1 14:46:24 2009
From: wes at umich.edu (Wesley Craig)
Date: Wed, 1 Apr 2009 14:46:24 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <49D281FF.90305@kickflop.net>
References: <49D2403C.50908@kickflop.net>
	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
Message-ID: <EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>

On 31 Mar 2009, at 16:50, Jeff Blaine wrote:
> What I mean, is, for one example -- a user is currently
> reporting that 'FETCH' (via the imapdu command) is showing
> 142 messages totalling 640KB in a folder that is actually
> completely empty on disk (except for cyrus.* files).

Fetch doesn't examine the messages in the mailbox if it doesn't need  
to.  Since you've reonstructed already, I wouldn't expect the seen DB  
is the problem.  Probably the quota.  Try rebuilding it.

:wes

From anfi at onet.eu  Wed Apr  1 14:47:42 2009
From: anfi at onet.eu (Andrzej Adam Filip)
Date: Wed, 01 Apr 2009 20:47:42 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <49D3B1D3.3040905@gmail.com> (nodens's message of "Wed\,
	01 Apr 2009 20\:26\:27 +0200")
References: <49D3B1D3.3040905@gmail.com>
Message-ID: <xsic81ch78@maria.ugly.jumpingcrab.com>

nodens2099 <nodens2099 at gmail.com> wrote:

> I'm trying to send mail directly to a shared folder, via the plus-addressing
> syntax.
>
> My user entry redirect to +Folder/SubFolder at domain.tld .
>
> cyrusv2d mailer is called via the mailertable.
>
> The ACL are ok.
>
> It would work with a folder in lower case, but not if the folder includes upper
> case characters.
>
> I thought it was ok to have upper case after the + in the local part of the
> address ?
>
> I tried setting
> "lmtp_downcase_rcpt: 0"
> in imapd.conf, adding the u flag to the cyrusv2d mailer, but to no avail.
>
> the +part is always converted to lower case.

Post your cyrusv2d definition from sendmail.cf.
  echo '=M' | sendmail -bt | grep cyrusv2d

Does it contain F=u flag?

By default sendmail's mailers convert user part of recipient address to
lowercase. F=u flag stops it. 
*BUT* it would apply to "normal recipients" *too*.

> initially I tried using the mrs_cyrus_mailertable approach but it was even
> worse, as checking +folder/subfolder at domaine.tld with smmap returns unknown
> user, even when the folder exists and has the "p" acl.

Does smmap return "unknown user" *even for lowercase folders*?

> Any clue ?
>
> I'd really like to avoid converting all existing folders to lower case of
> possible, it would be painful.

It merely a matter of choosing a simple way you like anyway :-)

BTW
Why have not you tried RTCyrus3 recipe?
[ It would require small "patching" too to meet your requirements ]


-- 
[pl>en: Andrew] Andrzej Adam Filip : anfi at onet.eu
To err is human,
To purr feline.
  -- Robert Byrne

From jblaine at kickflop.net  Wed Apr  1 15:59:11 2009
From: jblaine at kickflop.net (Jeff Blaine)
Date: Wed, 01 Apr 2009 15:59:11 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
References: <49D2403C.50908@kickflop.net>
	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
	<EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
Message-ID: <49D3C78F.5080908@kickflop.net>

Well, hmm...

% sudo -u cyrus /local/mail/cyrus/bin/quota -f
user.ahe: usage was 148771194, now 148756681
...
user.william: usage was 107244707, now 107116597
failed opening header for mailbox 'user.jay': System I/O error: %m
failed building quota list for '*': System I/O error: %m

Apr  1 15:56:12 our.host.org quota[9819]: [ID 136705 local6.error] 
IOERROR: opening /var/spool/imap/user/jay/cyrus.header: No such file or 
directory
Apr  1 15:56:12 our.host.org quota[9819]: [ID 357877 local6.error] 
failed opening header for mailbox 'user.jay': System I/O error: Bad file 
number
Apr  1 15:56:13 our.host.org quota[9819]: [ID 809228 local6.error] 
failed building quota list for '*': System I/O error: Bad file number

Wesley Craig wrote:
> On 31 Mar 2009, at 16:50, Jeff Blaine wrote:
>> What I mean, is, for one example -- a user is currently
>> reporting that 'FETCH' (via the imapdu command) is showing
>> 142 messages totalling 640KB in a folder that is actually
>> completely empty on disk (except for cyrus.* files).
> 
> Fetch doesn't examine the messages in the mailbox if it doesn't need 
> to.  Since you've reonstructed already, I wouldn't expect the seen DB is 
> the problem.  Probably the quota.  Try rebuilding it.
> 
> :wes
> 

From jblaine at kickflop.net  Wed Apr  1 16:03:04 2009
From: jblaine at kickflop.net (Jeff Blaine)
Date: Wed, 01 Apr 2009 16:03:04 -0400
Subject: 'stage.'  ???
Message-ID: <49D3C878.1090300@kickflop.net>

What's this all about?

imap:linus> pwd
/var/spool/imap
imap:linus> ls stage.
10014-1214461933-0  14035-1214433131-0  22926-1214436731-0 
3797-1214451132-0
10114-1214461933-0  14082-1214479931-0  22931-1214436731-0 
3802-1214451132-0
10119-1214461933-0  14084-1214419933-0  22936-1214436731-0 
3807-1214451132-0
10124-1214461934-0  14087-1214479931-0  23656-1214425930-0 
4047-1214429531-0
10129-1214461934-0  14092-1214479934-0  23704-1214425931-0 
4054-1214429531-0
10134-1214461934-0  14097-1214479934-0  2407-1214447531-0 
4060-1214429531-0
10139-1214461934-0  14102-1214479934-0  2474-1214447531-0 
4881-1214454731-0
10144-1214461935-0  14107-1214479934-0  2479-1214447531-0 
4948-1214454731-0
10149-1214461935-0  14112-1214479935-0  2484-1214447531-0 
4953-1214454731-0
10154-1214461935-0  14117-1214479935-0  2489-1214447531-0 
4958-1214454732-0
10359-1043588570    14123-1214479936-0  2494-1214447532-0 
4963-1214454732-0
1090-1214427292-0   14128-1214479936-0  27817-1214484256-0 
4968-1214454732-0
11435-1214472731-0  14133-1214479936-0  28549-1214485744-0 
4973-1214454733-0
11451-1214472731-0  15328-1214422330-0  28638-1214485728-0 
4978-1214454733-0
11456-1214472731-0  17973-1214465531-0  28931-1214486246-0 
4983-1214454733-0
11461-1214472732-0  18586-1214465531-0  29334-1214487131-0 
5871-1214458331-0
11466-1214472732-0  18605-1214465532-0  29402-1214487131-0 
6043-1214458331-0
11471-1214472732-0  18626-1214465532-0  29443-1214487131-0 
6048-1214458331-0
11476-1214472732-0  18637-1214465532-0  29482-1214487131-0 
6053-1214458331-0
11481-1214472733-0  18642-1214465533-0  29487-1214487132-0 
6058-1214458332-0
11486-1214472733-0  18653-1214465533-0  29492-1214487132-0 
6063-1214458332-0
11492-1214472733-0  18658-1214465533-0  29497-1214487132-0 
6068-1214458332-0
11497-1214472733-0  18663-1214465534-0  29502-1214487132-0 
6073-1214458332-0
1223-1214443931-0   18668-1214465534-0  29507-1214487133-0 
6078-1214458332-0
1228-1214443932-0   21077-1214482453-0  29512-1214487133-0 
6083-1214459200-0
12329-1214476331-0  21543-1214483531-0  29517-1214487133-0 
6855-1214469131-0
1233-1214443932-0   21605-1214483531-0  29522-1214487134-0 
6921-1214469131-0
1238-1214443932-0   21630-1214483531-0  29527-1214487134-0 
6926-1214469131-0
12662-1214476331-0  21635-1214483531-0  29532-1214487134-0 
6931-1214469132-0
12674-1214476331-0  21640-1214483532-0  29537-1214487135-0 
6936-1214469132-0
12679-1214476332-0  21645-1214483532-0  29542-1214487135-0 
6941-1214469132-0
12684-1214476332-0  21650-1214483532-0  29547-1214487135-0 
6946-1214469132-0
12689-1214476332-0  21655-1214483533-0  29850-1214440331-0 
6951-1214469132-0
12694-1214476333-0  21660-1214483533-0  3244-1214450204-0 
6956-1214469133-0
12699-1214476333-0  21665-1214483533-0  3442-1214450401-0 
6961-1214469133-0
12704-1214476333-0  21670-1214483533-0  3535-1214450979-0   704-1214443931-0
12709-1214476333-0  21675-1214483533-0  3685-1214429530-0   75-1214440331-0
12714-1214476333-0  21680-1214483534-0  3695-1214451131-0   772-1214426886-0
13446-1214478572-0  2201-1214447123-0   3772-1214451131-0 
7901-1214472209-0
13783-1214433131-0  22372-1214435402-0  3777-1214451131-0   80-1214440331-0
13966-1214479931-0  22736-1214425071-0  3782-1214451131-0 
8542-1231956337-0
14025-1214433131-0  22839-1214436731-0  3787-1214451131-0   86-1214440331-0
14030-1214433131-0  22921-1214436731-0  3792-1214451132-0   91-1214440332-0
imap:linus>

From brong at fastmail.fm  Wed Apr  1 18:13:19 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Thu, 2 Apr 2009 09:13:19 +1100
Subject: 'stage.'  ???
In-Reply-To: <49D3C878.1090300@kickflop.net>
References: <49D3C878.1090300@kickflop.net>
Message-ID: <20090401221319.GA6776@brong.net>

On Wed, Apr 01, 2009 at 04:03:04PM -0400, Jeff Blaine wrote:
> What's this all about?
> 
> imap:linus> pwd
> /var/spool/imap
> imap:linus> ls stage.
> 10014-1214461933-0  14035-1214433131-0  22926-1214436731-0 
> 3797-1214451132-0

Looks like you had a bunch of crashes back in June last year
and they left a bunch of stage files in place.  Usually lmtp
deliveries failing or something.  You can safely clean up
anything over a day or so old in there.

Cyrus "append" works by spooling the file into the stage.
directory, then parsing the file.  Finally it hardlinks (if
possible, otherwise copies) the file into place in the
actual mailbox directory.  Amongst other things, this is
good for single-instance store, where every copy is a hardlink
to the file in stage.

Finally it unlinks the copy in stage.

If anything goes wrong in between, the file in stage. is left
behind.

Bron ( I guess it would be sane to "rm -f stage./*" during
       the startup script, or just tmpreaper it occasionally )

From nodens2099 at gmail.com  Thu Apr  2 03:51:14 2009
From: nodens2099 at gmail.com (nodens2099)
Date: Thu, 02 Apr 2009 09:51:14 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <xsic81ch78@maria.ugly.jumpingcrab.com>
References: <49D3B1D3.3040905@gmail.com>
	<xsic81ch78@maria.ugly.jumpingcrab.com>
Message-ID: <49D46E72.6070803@gmail.com>

Andrzej Adam Filip a ?crit :
> nodens2099 <nodens2099 at gmail.com> wrote:
>
>   
>> I tried setting
>> "lmtp_downcase_rcpt: 0"
>> in imapd.conf, adding the u flag to the cyrusv2d mailer, but to no avail.
>>
>> the +part is always converted to lower case.
>>     
>
> Post your cyrusv2d definition from sendmail.cf.
>   echo '=M' | sendmail -bt | grep cyrusv2d
>
> Does it contain F=u flag?
>
>   
Yes, but only once I added the F=u flag to the m4 file
(MODIFY_MAILER_FLAGS(`cyrusv2d', `+u') in the sendmail.mc didn't work).

mailer 4 (cyrusv2d): P=[IPC] S=EnvFromSMTP/HdrFromL
R=EnvToSMTP/EnvToSMTP M=0 U=-1:-1 F=1DFMXlmnqsuz L=0 E=\r\n
T=DNS/RFC822/SMTP r=100 A=FILE /var/run/cyrus/socket/lmtp

Thanks !

> By default sendmail's mailers convert user part of recipient address to
> lowercase. F=u flag stops it. 
> *BUT* it would apply to "normal recipients" *too*.
>
>   
That whas my understanding as well.
but it's ok, since lmtp_downcase_rcpt: yes takes care of the username part.

>> initially I tried using the mrs_cyrus_mailertable approach but it was even
>> worse, as checking +folder/subfolder at domaine.tld with smmap returns unknown
>> user, even when the folder exists and has the "p" acl.
>>     
>
> Does smmap return "unknown user" *even for lowercase folders*?
>
>   
Nope, you're right. the real problem is on the mrs check, then. If I
create a lowerrcase folder along the mixed cased one, the mail is
received in the correct folder (that is, the mixed case).

>> Any clue ?
>>
>> I'd really like to avoid converting all existing folders to lower case of
>> possible, it would be painful.
>>     
>
> It merely a matter of choosing a simple way you like anyway :-)
>
> BTW
> Why have not you tried RTCyrus3 recipe?
> [ It would require small "patching" too to meet your requirements ]
>
>
>   
Actually, I was adding a new backend in an existing murder environment,
migrating existing users and public folders from MS Exchange server.
Since rtcyrusv2 "just works" in the existing environment, I didn't even
check if there was an update. I saw about RTCyrus3 only when searching a
solution to my public folder problem.

if it can resolve the mrs_cyrus_mailertable problem, and if it's not too
difficult to upgrade rtcyrus2 with it, I'm willing to give it a try.

Regards,

-- 
Cl?ment Hermann (nodens)

From anfi at onet.eu  Thu Apr  2 04:53:27 2009
From: anfi at onet.eu (Andrzej Adam Filip)
Date: Thu, 02 Apr 2009 10:53:27 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <49D46E72.6070803@gmail.com> (nodens's message of "Thu\,
	02 Apr 2009 09\:51\:14 +0200")
References: <49D3B1D3.3040905@gmail.com>
	<xsic81ch78@maria.ugly.jumpingcrab.com> <49D46E72.6070803@gmail.com>
Message-ID: <gim8nlqi78@shane.huge.strangled.net>

nodens2099 <nodens2099 at gmail.com> wrote:

> Andrzej Adam Filip a ?crit :
>> nodens2099 <nodens2099 at gmail.com> wrote:
>>
>>   
>>> I tried setting
>>> "lmtp_downcase_rcpt: 0"
>>> in imapd.conf, adding the u flag to the cyrusv2d mailer, but to no avail.
>>>
>>> the +part is always converted to lower case.
>>>     
>>
>> Post your cyrusv2d definition from sendmail.cf.
>>   echo '=M' | sendmail -bt | grep cyrusv2d
>>
>> Does it contain F=u flag?
>   
> Yes, but only once I added the F=u flag to the m4 file
> (MODIFY_MAILER_FLAGS(`cyrusv2d', `+u') in the sendmail.mc didn't work).
>
> mailer 4 (cyrusv2d): P=[IPC] S=EnvFromSMTP/HdrFromL
> R=EnvToSMTP/EnvToSMTP M=0 U=-1:-1 F=1DFMXlmnqsuz L=0 E=\r\n
> T=DNS/RFC822/SMTP r=100 A=FILE /var/run/cyrus/socket/lmtp
>
> Thanks !
>
>> By default sendmail's mailers convert user part of recipient address to
>> lowercase. F=u flag stops it. 
>> *BUT* it would apply to "normal recipients" *too*.
>   
> That whas my understanding as well.
> but it's ok, since lmtp_downcase_rcpt: yes takes care of the username part.
>
>>> initially I tried using the mrs_cyrus_mailertable approach but it was even
>>> worse, as checking +folder/subfolder at domaine.tld with smmap returns unknown
>>> user, even when the folder exists and has the "p" acl.     
>>
>> Does smmap return "unknown user" *even for lowercase folders*?
>>
> Nope, you're right. the real problem is on the mrs check, then. 

It can use only what smmap provides.
Are you ready to ask for changed in Cyrus' smmap to make it capable to
check Public folders availability?

mrs_cyrus_mailertable may be changed to check validity of mailbox folder
its own list of valid folders.

> If I create a lowerrcase folder along the mixed cased one, the mail is
> received in the correct folder (that is, the mixed case).
>
>>> Any clue ?
>>>
>>> I'd really like to avoid converting all existing folders to lower case of
>>> possible, it would be painful.
>>>     
>>
>> It merely a matter of choosing a simple way you like anyway :-)
>>
>> BTW
>> Why have not you tried RTCyrus3 recipe?
>> [ It would require small "patching" too to meet your requirements ]
>
> Actually, I was adding a new backend in an existing murder environment,
> migrating existing users and public folders from MS Exchange server.
> Since rtcyrusv2 "just works" in the existing environment, I didn't even
> check if there was an update. I saw about RTCyrus3 only when searching a
> solution to my public folder problem.
>
> if it can resolve the mrs_cyrus_mailertable problem, and if it's not too
> difficult to upgrade rtcyrus2 with it, I'm willing to give it a try.

It can be fixed but you should decide which path do you want do go:
a) Cyrus smmap can tell if public folders is accessible
b) sendmail uses its own list of valid publicly accessible Cyrus public folders

-- 
[pl>en: Andrew] Andrzej Adam Filip : anfi at onet.eu
She's the kind of girl who climbed the ladder of success wrong by wrong.
  -- Mae West

From nodens2099 at gmail.com  Thu Apr  2 05:20:23 2009
From: nodens2099 at gmail.com (nodens2099)
Date: Thu, 02 Apr 2009 11:20:23 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <blm8jqjl78@shane.huge.strangled.net>
References: <49D3B1D3.3040905@gmail.com>	<xsic81ch78@maria.ugly.jumpingcrab.com>
	<49D46E72.6070803@gmail.com> <blm8jqjl78@shane.huge.strangled.net>
Message-ID: <49D48357.1080208@gmail.com>

Andrzej Adam Filip a ?crit :
>>> Does smmap return "unknown user" *even for lowercase folders*?
>>>
>> Nope, you're right. the real problem is on the mrs check, then. 
> 
> It can use only what smmap provides.
> Are you ready to ask for changed in Cyrus' smmap to make it capable to
> check Public folders availability?
> 
> mrs_cyrus_mailertable may be changed to check validity of mailbox folder
> its own list of valid folders.
> 
This is already what I'm doing with ldap. There is no need to add
another database, which would have to be maintained as the public folder
list change, IMO.

I actually took a look at the smmapd code, and it uses the
lmtp_downcase_rcpt value to know whether the recipient should be
converted to downcase or not.
According to the code, it convert the user part only, and has a special
case for shared folder, so it should be ok.

Do you know a way to test smmapd manually ?


>> if it can resolve the mrs_cyrus_mailertable problem, and if it's not too
>> difficult to upgrade rtcyrus2 with it, I'm willing to give it a try.
> 
> It can be fixed but you should decide which path do you want do go:
> a) Cyrus smmap can tell if public folders is accessible
> b) sendmail uses its own list of valid publicly accessible Cyrus public folders
> 

the a) is definitely the way to go.

-- 
Cl?ment Hermann (nodens)

From jblaine at kickflop.net  Thu Apr  2 10:42:23 2009
From: jblaine at kickflop.net (Jeff Blaine)
Date: Thu, 02 Apr 2009 10:42:23 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <49D3C78F.5080908@kickflop.net>
References: <49D2403C.50908@kickflop.net>	<200903312243.18875.ml@awinkelmann.de>	<49D281FF.90305@kickflop.net>	<EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
	<49D3C78F.5080908@kickflop.net>
Message-ID: <49D4CECF.6040101@kickflop.net>

Okay, well, for what it's worth, I fixed all of the
problems prohibiting me from running a quota -f
to completion.

The problem: imapdu.pl is buggy

It fails to do the right thing with mailboxes containing
a space in the name.

...
1.46 MB 114 msgs INBOX.BMP
1.46 MB 114 msgs INBOX.Bio Stuff
0.00 bytes 0 msgs INBOX.Drafts
1.25 MB 36 msgs INBOX.HLT
1.25 MB 36 msgs INBOX.Information Retrieval
...

# $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $

I don't suppose 'murch', the author of the code reads
this list?

Jeff Blaine wrote:
> Well, hmm...
> 
> % sudo -u cyrus /local/mail/cyrus/bin/quota -f
> user.ahe: usage was 148771194, now 148756681
> ...
> user.william: usage was 107244707, now 107116597
> failed opening header for mailbox 'user.jay': System I/O error: %m
> failed building quota list for '*': System I/O error: %m
> 
> Apr  1 15:56:12 our.host.org quota[9819]: [ID 136705 local6.error] 
> IOERROR: opening /var/spool/imap/user/jay/cyrus.header: No such file or 
> directory
> Apr  1 15:56:12 our.host.org quota[9819]: [ID 357877 local6.error] 
> failed opening header for mailbox 'user.jay': System I/O error: Bad file 
> number
> Apr  1 15:56:13 our.host.org quota[9819]: [ID 809228 local6.error] 
> failed building quota list for '*': System I/O error: Bad file number
> 
> Wesley Craig wrote:
>> On 31 Mar 2009, at 16:50, Jeff Blaine wrote:
>>> What I mean, is, for one example -- a user is currently
>>> reporting that 'FETCH' (via the imapdu command) is showing
>>> 142 messages totalling 640KB in a folder that is actually
>>> completely empty on disk (except for cyrus.* files).
>> Fetch doesn't examine the messages in the mailbox if it doesn't need 
>> to.  Since you've reonstructed already, I wouldn't expect the seen DB is 
>> the problem.  Probably the quota.  Try rebuilding it.
>>
>> :wes
>>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> 

From anfi at onet.eu  Thu Apr  2 14:18:14 2009
From: anfi at onet.eu (Andrzej Adam Filip)
Date: Thu, 02 Apr 2009 20:18:14 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <49D48357.1080208@gmail.com> (nodens's message of "Thu\,
	02 Apr 2009 11\:20\:23 +0200")
References: <49D3B1D3.3040905@gmail.com>
	<xsic81ch78@maria.ugly.jumpingcrab.com> <49D46E72.6070803@gmail.com>
	<blm8jqjl78@shane.huge.strangled.net> <49D48357.1080208@gmail.com>
Message-ID: <pw4jqgxf78@shane.huge.strangled.net>

nodens2099 <nodens2099 at gmail.com> wrote:

> Andrzej Adam Filip a ?crit :
>>>> Does smmap return "unknown user" *even for lowercase folders*?
>>>>
>>> Nope, you're right. the real problem is on the mrs check, then. 
>> 
>> It can use only what smmap provides.
>> Are you ready to ask for changed in Cyrus' smmap to make it capable to
>> check Public folders availability?
>> 
>> mrs_cyrus_mailertable may be changed to check validity of mailbox folder
>> its own list of valid folders.
>> 
> This is already what I'm doing with ldap. There is no need to add
> another database, which would have to be maintained as the public folder
> list change, IMO.

OK - smmap map be needless for "LDAP centric" configuration.

> I actually took a look at the smmapd code, and it uses the
> lmtp_downcase_rcpt value to know whether the recipient should be
> converted to downcase or not.
> According to the code, it convert the user part only, and has a special
> case for shared folder, so it should be ok.
>
> Do you know a way to test smmapd manually ?

There are simple socket map client and server perl scripts in contrib
directory of sendmail distribution.

>>> if it can resolve the mrs_cyrus_mailertable problem, and if it's not too
>>> difficult to upgrade rtcyrus2 with it, I'm willing to give it a try.
>> 
>> It can be fixed but you should decide which path do you want do go:
>> a) Cyrus smmap can tell if public folders is accessible
>> b) sendmail uses its own list of valid publicly accessible Cyrus public folders
>
> the a) is definitely the way to go.

I "patch" only sendmail.cf :-)

-- 
[pl>en: Andrew] Andrzej Adam Filip : anfi at onet.eu
Woman inspires us to great things, and prevents us from achieving them.
  -- Dumas

From brong at fastmail.fm  Thu Apr  2 18:19:55 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Fri, 3 Apr 2009 09:19:55 +1100
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <49D4CECF.6040101@kickflop.net>
References: <49D2403C.50908@kickflop.net>
	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
	<EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
	<49D3C78F.5080908@kickflop.net> <49D4CECF.6040101@kickflop.net>
Message-ID: <20090402221955.GB6898@brong.net>

On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
> Okay, well, for what it's worth, I fixed all of the
> problems prohibiting me from running a quota -f
> to completion.
> 
> The problem: imapdu.pl is buggy

Doh.

> It fails to do the right thing with mailboxes containing
> a space in the name.

Yeah, that's not entirely a surprise.  Spaces in names
confuse lots of stuff.

> ...
> 1.46 MB 114 msgs INBOX.BMP
> 1.46 MB 114 msgs INBOX.Bio Stuff
> 0.00 bytes 0 msgs INBOX.Drafts
> 1.25 MB 36 msgs INBOX.HLT
> 1.25 MB 36 msgs INBOX.Information Retrieval
> ...
> 
> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
> 
> I don't suppose 'murch', the author of the code reads
> this list?

Yeah, he's around, though he doesn't always see stuff as quickly
if it's only sent to the list.  I've CC'd him.

That said, he's likely not the author - just the last person that
changed things.  Given that he did a giant sweeping copyright
update of nearly every file in the tree a couple of months ago...

  my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
  if ($rc eq 'OK') {
  } else {
      print "failed: $mb: $msg\n";
  }

Apart from being icky perl, that will fail to change mailboxes
because the EXAMINE command will have dodgy syntax.  I'm not
entirely sure why you're not seeing the 'failed' messages
though...  The "$mb" needs to be at least quoted - which is
why I generally use something like Mail::IMAPTalk that can
do correct protocol quoting.

Ahh - it probably does output the "failed" stuff further up.
 
Bron.

From murch at andrew.cmu.edu  Thu Apr  2 18:44:53 2009
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 02 Apr 2009 18:44:53 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <20090402221955.GB6898@brong.net>
References: <49D2403C.50908@kickflop.net>
	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
	<EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
	<49D3C78F.5080908@kickflop.net> <49D4CECF.6040101@kickflop.net>
	<20090402221955.GB6898@brong.net>
Message-ID: <49D53FE5.1010602@andrew.cmu.edu>

I can guarantee that I had nothing to do with writing this code.  Other 
than updating the copyright blurb as Bron noted, I'm fairly certain I've 
never even looked at it.  Since its only example code, use it at your 
own risk.  If you fix it or improve it, feel free to pass us the changes.


Bron Gondwana wrote:
> On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
>> Okay, well, for what it's worth, I fixed all of the
>> problems prohibiting me from running a quota -f
>> to completion.
>>
>> The problem: imapdu.pl is buggy
> 
> Doh.
> 
>> It fails to do the right thing with mailboxes containing
>> a space in the name.
> 
> Yeah, that's not entirely a surprise.  Spaces in names
> confuse lots of stuff.
> 
>> ...
>> 1.46 MB 114 msgs INBOX.BMP
>> 1.46 MB 114 msgs INBOX.Bio Stuff
>> 0.00 bytes 0 msgs INBOX.Drafts
>> 1.25 MB 36 msgs INBOX.HLT
>> 1.25 MB 36 msgs INBOX.Information Retrieval
>> ...
>>
>> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
>>
>> I don't suppose 'murch', the author of the code reads
>> this list?
> 
> Yeah, he's around, though he doesn't always see stuff as quickly
> if it's only sent to the list.  I've CC'd him.
> 
> That said, he's likely not the author - just the last person that
> changed things.  Given that he did a giant sweeping copyright
> update of nearly every file in the tree a couple of months ago...
> 
>   my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
>   if ($rc eq 'OK') {
>   } else {
>       print "failed: $mb: $msg\n";
>   }
> 
> Apart from being icky perl, that will fail to change mailboxes
> because the EXAMINE command will have dodgy syntax.  I'm not
> entirely sure why you're not seeing the 'failed' messages
> though...  The "$mb" needs to be at least quoted - which is
> why I generally use something like Mail::IMAPTalk that can
> do correct protocol quoting.
> 
> Ahh - it probably does output the "failed" stuff further up.
>  
> Bron.
> 

-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University

From jblaine at kickflop.net  Thu Apr  2 19:05:42 2009
From: jblaine at kickflop.net (Jeff Blaine)
Date: Thu, 02 Apr 2009 19:05:42 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <20090402221955.GB6898@brong.net>
References: <49D2403C.50908@kickflop.net>
	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
	<EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
	<49D3C78F.5080908@kickflop.net> <49D4CECF.6040101@kickflop.net>
	<20090402221955.GB6898@brong.net>
Message-ID: <49D544C6.5020106@kickflop.net>

I tried both

a) escaping spaces in $mb
b) quoting $mb

Neither worked.

That's about as far as I delve into Perl anymore, so
I'm happy to just retract my note to our users about
this tool.

Bron Gondwana wrote:
> On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
>> Okay, well, for what it's worth, I fixed all of the
>> problems prohibiting me from running a quota -f
>> to completion.
>>
>> The problem: imapdu.pl is buggy
> 
> Doh.
> 
>> It fails to do the right thing with mailboxes containing
>> a space in the name.
> 
> Yeah, that's not entirely a surprise.  Spaces in names
> confuse lots of stuff.
> 
>> ...
>> 1.46 MB 114 msgs INBOX.BMP
>> 1.46 MB 114 msgs INBOX.Bio Stuff
>> 0.00 bytes 0 msgs INBOX.Drafts
>> 1.25 MB 36 msgs INBOX.HLT
>> 1.25 MB 36 msgs INBOX.Information Retrieval
>> ...
>>
>> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
>>
>> I don't suppose 'murch', the author of the code reads
>> this list?
> 
> Yeah, he's around, though he doesn't always see stuff as quickly
> if it's only sent to the list.  I've CC'd him.
> 
> That said, he's likely not the author - just the last person that
> changed things.  Given that he did a giant sweeping copyright
> update of nearly every file in the tree a couple of months ago...
> 
>   my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
>   if ($rc eq 'OK') {
>   } else {
>       print "failed: $mb: $msg\n";
>   }
> 
> Apart from being icky perl, that will fail to change mailboxes
> because the EXAMINE command will have dodgy syntax.  I'm not
> entirely sure why you're not seeing the 'failed' messages
> though...  The "$mb" needs to be at least quoted - which is
> why I generally use something like Mail::IMAPTalk that can
> do correct protocol quoting.
> 
> Ahh - it probably does output the "failed" stuff further up.
>  
> Bron.
> 

From mills at cc.umanitoba.ca  Thu Apr  2 19:26:45 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Thu, 2 Apr 2009 18:26:45 -0500
Subject: Why do lmtpd processes accumulate?
Message-ID: <20090402232644.GA24262@cc.umanitoba.ca>

We have a fairly conventional Cyrus server with one front-end and one
back-end.  Recently, I've noticed that when the number of lmtpd
processes on the back-end server increases to the 400 range,
performance drops to a crawl, including local deliveries.  When I put
an upper limit of 128 or 64 to these processes on the front-end, which
requires a Cyrus restart, all of the local deliveries succeed in a
short time.  Performance also comes back to normal.

I can't tell if it's the restart that fixes the problem or if it's
the limit on lmtpd children.  I'm wondering, though, if the lmtpd
processes are all waiting on some Cyrus database, so that more of
them just makes it worse.  These are the databases, from imapd.conf:

    annotation_db:          skiplist
    duplicate_db:           berkeley-nosync
    mboxkey_db:             skiplist
    mboxlist_db:            skiplist
    quota_db:               quotalegacy
    seenstate_db:           skiplist
    subscription_db:        flat
    tlscache_db:            berkeley-nosync

I believe those are current recommendations.  Which ones might be
causing the problem?  Is there tuning that can be done on them?

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From murch at andrew.cmu.edu  Fri Apr  3 09:09:52 2009
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Fri, 03 Apr 2009 09:09:52 -0400
Subject: Why do lmtpd processes accumulate?
In-Reply-To: <20090402232644.GA24262@cc.umanitoba.ca>
References: <20090402232644.GA24262@cc.umanitoba.ca>
Message-ID: <49D60AA0.5060303@andrew.cmu.edu>

Have you done an strace on one of the lmtpd on the backend to see what 
it is doing?


Gary Mills wrote:
> We have a fairly conventional Cyrus server with one front-end and one
> back-end.  Recently, I've noticed that when the number of lmtpd
> processes on the back-end server increases to the 400 range,
> performance drops to a crawl, including local deliveries.  When I put
> an upper limit of 128 or 64 to these processes on the front-end, which
> requires a Cyrus restart, all of the local deliveries succeed in a
> short time.  Performance also comes back to normal.
> 
> I can't tell if it's the restart that fixes the problem or if it's
> the limit on lmtpd children.  I'm wondering, though, if the lmtpd
> processes are all waiting on some Cyrus database, so that more of
> them just makes it worse.  These are the databases, from imapd.conf:
> 
>     annotation_db:          skiplist
>     duplicate_db:           berkeley-nosync
>     mboxkey_db:             skiplist
>     mboxlist_db:            skiplist
>     quota_db:               quotalegacy
>     seenstate_db:           skiplist
>     subscription_db:        flat
>     tlscache_db:            berkeley-nosync
> 
> I believe those are current recommendations.  Which ones might be
> causing the problem?  Is there tuning that can be done on them?
> 

-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University

From nodens2099 at gmail.com  Fri Apr  3 10:47:11 2009
From: nodens2099 at gmail.com (nodens2099)
Date: Fri, 03 Apr 2009 16:47:11 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <pw4jqgxf78@shane.huge.strangled.net>
References: <49D3B1D3.3040905@gmail.com>	<xsic81ch78@maria.ugly.jumpingcrab.com>
	<49D46E72.6070803@gmail.com>	<blm8jqjl78@shane.huge.strangled.net>
	<49D48357.1080208@gmail.com> <pw4jqgxf78@shane.huge.strangled.net>
Message-ID: <49D6216F.1040709@gmail.com>

Andrzej Adam Filip a ?crit :
> nodens2099 <nodens2099 at gmail.com> wrote:
> 
>> Andrzej Adam Filip a ?crit :
>>>>> Does smmap return "unknown user" *even for lowercase folders*?
>>>>>
>>>> Nope, you're right. the real problem is on the mrs check, then. 
>>> It can use only what smmap provides.
>>> Are you ready to ask for changed in Cyrus' smmap to make it capable to
>>> check Public folders availability?
>>>
>>> mrs_cyrus_mailertable may be changed to check validity of mailbox folder
>>> its own list of valid folders.
>>>
>> This is already what I'm doing with ldap. There is no need to add
>> another database, which would have to be maintained as the public folder
>> list change, IMO.
> 
> OK - smmap map be needless for "LDAP centric" configuration.
> 
>> I actually took a look at the smmapd code, and it uses the
>> lmtp_downcase_rcpt value to know whether the recipient should be
>> converted to downcase or not.
>> According to the code, it convert the user part only, and has a special
>> case for shared folder, so it should be ok.
>>
>> Do you know a way to test smmapd manually ?
> 
> There are simple socket map client and server perl scripts in contrib
> directory of sendmail distribution.
> 

I made some more test. The problem is definitely in the cyrus map.

real mailbox name : Hosting/Abuse at domain.com

sendmail -d -bt
<cut meaningless debug info>
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> /map cyrus +Hosting/Abuse at domain.com
map_lookup: cyrus (+Hosting/Abuse at domain.com) no match (68)

(creating hosting/abuse at domain.com, that is the same folder but in
lowercase)
> /map cyrus +Hosting/Abuse at domain.com
map_lookup: cyrus (+Hosting/Abuse at domain.com)
map_rewrite(+hosting/abuse at domain.com), av =
        +Hosting/Abuse at domain.com
map_rewrite => +hosting/abuse at domain.com<OK>
returns +hosting/abuse at domain.com<OK> (0)


socketmapClient.pl :

./socketmapClient.pl unix:/var/run/cyrus/socket/smmap cyrus
"+Hosting/Abuse at domain.com"
+Hosting/Abuse at domain.com => OK +Hosting/Abuse at domain.com

So socketmap daemon works as expected.

Regards,

-- 
Cl?ment Hermann (nodens)



From jross at broad.mit.edu  Fri Apr  3 12:00:39 2009
From: jross at broad.mit.edu (Jesse Ross)
Date: Fri, 3 Apr 2009 12:00:39 -0400
Subject: cyrus won't recognize new partition
In-Reply-To: <72CF3B00-C9AE-4A86-8DAC-D596B1B6F9D7@umich.edu>
References: <a93d0eff0903300923i2c573770u971e986130d8434d@mail.gmail.com>
	<20090330214659.GA8544@brong.net>
	<a93d0eff0903310733r9bc7a0ar6ee6dabb65e5912@mail.gmail.com>
	<72CF3B00-C9AE-4A86-8DAC-D596B1B6F9D7@umich.edu>
Message-ID: <a93d0eff0904030900u47e428a0ufac2b0725e4389ca@mail.gmail.com>

Thanks to all for your help.  I finally figured out my problem, which was
just old-fashioned operator idiocy.  I have two imap processes running with
different configs - one without TLS, so that I can conveniently run cyradm
locally.  I had only made the config change in the public server, not the
one that cyradm connects to.

jesse


On Wed, Apr 1, 2009 at 2:38 PM, Wesley Craig <wes at umich.edu> wrote:

> On 31 Mar 2009, at 10:33, Jesse Ross wrote:
>
>> Thanks for the suggestion.  The only thing that the mkimap script does
>> differently from the way I did it was to create a directory
>> /var/spool/imap07/sync./.  However, I still can't get cyrus to recognize the
>> partition!  The symptoms are the same; ownership of directories set to
>> cyrus, appropriate permissions as copied from working partitions,
>> configuration same as for the working partitions.
>>
>
> IMAP_PARTITION_UNKNOWN isn't associated with permission problems,
> typically.  For the create code path, you get it when there's a failure to
> convert the symbolic partition name to a path.
>
>  Is there perhaps a way to get cyrus to log what it thinks the problem with
>> the partition is?
>>
>
> The number of cases that can result in IMAP_PARTITION_UNKNOWN is pretty
> small, I'd probably try inserting some logging into the process for those
> results.  It's either a bug or configuration oversight, I'm sure.
>
> :wes
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090403/527b65cd/attachment.html 

From andrzej.filip at gmail.com  Fri Apr  3 13:05:29 2009
From: andrzej.filip at gmail.com (Andrzej Adam Filip)
Date: Fri, 03 Apr 2009 19:05:29 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <49D6216F.1040709@gmail.com> (nodens's message of "Fri\,
	03 Apr 2009 16\:47\:11 +0200")
References: <49D3B1D3.3040905@gmail.com>
	<xsic81ch78@maria.ugly.jumpingcrab.com> <49D46E72.6070803@gmail.com>
	<blm8jqjl78@shane.huge.strangled.net> <49D48357.1080208@gmail.com>
	<pw4jqgxf78@shane.huge.strangled.net> <49D6216F.1040709@gmail.com>
Message-ID: <i5mstfrp78@harold.anfi.chickenkiller.com>

nodens2099 <nodens2099 at gmail.com> wrote:

> Andrzej Adam Filip a ?crit :
>> nodens2099 <nodens2099 at gmail.com> wrote:
>> 
>>> Andrzej Adam Filip a ?crit :
>>>>>> Does smmap return "unknown user" *even for lowercase folders*?
>>>>>>
>>>>> Nope, you're right. the real problem is on the mrs check, then. 
>>>> It can use only what smmap provides.
>>>> Are you ready to ask for changed in Cyrus' smmap to make it capable to
>>>> check Public folders availability?
>>>>
>>>> mrs_cyrus_mailertable may be changed to check validity of mailbox folder
>>>> its own list of valid folders.
>>>>
>>> This is already what I'm doing with ldap. There is no need to add
>>> another database, which would have to be maintained as the public folder
>>> list change, IMO.
>> 
>> OK - smmap map be needless for "LDAP centric" configuration.
>> 
>>> I actually took a look at the smmapd code, and it uses the
>>> lmtp_downcase_rcpt value to know whether the recipient should be
>>> converted to downcase or not.
>>> According to the code, it convert the user part only, and has a special
>>> case for shared folder, so it should be ok.
>>>
>>> Do you know a way to test smmapd manually ?
>> 
>> There are simple socket map client and server perl scripts in contrib
>> directory of sendmail distribution.
>> 
>
> I made some more test. The problem is definitely in the cyrus map.
>
> real mailbox name : Hosting/Abuse at domain.com
>
> sendmail -d -bt
> <cut meaningless debug info>
> ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
> Enter <ruleset> <address>
>> /map cyrus +Hosting/Abuse at domain.com
> map_lookup: cyrus (+Hosting/Abuse at domain.com) no match (68)
>
> (creating hosting/abuse at domain.com, that is the same folder but in
> lowercase)
>> /map cyrus +Hosting/Abuse at domain.com
> map_lookup: cyrus (+Hosting/Abuse at domain.com)
> map_rewrite(+hosting/abuse at domain.com), av =
>         +Hosting/Abuse at domain.com
> map_rewrite => +hosting/abuse at domain.com<OK>
> returns +hosting/abuse at domain.com<OK> (0)
>
> socketmapClient.pl :
>
> ./socketmapClient.pl unix:/var/run/cyrus/socket/smmap cyrus
> "+Hosting/Abuse at domain.com"
> +Hosting/Abuse at domain.com => OK +Hosting/Abuse at domain.com
>
> So socketmap daemon works as expected.

Sendmail's maps traditionally turn looked up key into lowercase.
It can be (usually) turned off by adding -f switch to map definition.
[ I have reported "missing -f" in socket as bug myself :-) ]

-- 
[pl>en: Andrew] Andrzej Adam Filip : anfi at onet.eu
It is common sense to take a method and try it.  If it fails,
admit it frankly and try another.  But above all, try something.
  -- Franklin D. Roosevelt

From nodens2099 at gmail.com  Sat Apr  4 09:18:21 2009
From: nodens2099 at gmail.com (Clement Hermann (nodens))
Date: Sat, 04 Apr 2009 15:18:21 +0200
Subject: [sendmail] lmtp, cyrusv2d, shared folders and case
In-Reply-To: <i5mstfrp78@harold.anfi.chickenkiller.com>
References: <49D3B1D3.3040905@gmail.com>	<xsic81ch78@maria.ugly.jumpingcrab.com>
	<49D46E72.6070803@gmail.com>	<blm8jqjl78@shane.huge.strangled.net>
	<49D48357.1080208@gmail.com>	<pw4jqgxf78@shane.huge.strangled.net>
	<49D6216F.1040709@gmail.com>
	<i5mstfrp78@harold.anfi.chickenkiller.com>
Message-ID: <49D75E1D.3070308@gmail.com>

Andrzej Adam Filip a ?crit :
> nodens2099 <nodens2099 at gmail.com> wrote:
>>
>> ./socketmapClient.pl unix:/var/run/cyrus/socket/smmap cyrus
>> "+Hosting/Abuse at domain.com"
>> +Hosting/Abuse at domain.com => OK +Hosting/Abuse at domain.com
>>
>> So socketmap daemon works as expected.
> 
> Sendmail's maps traditionally turn looked up key into lowercase.
> It can be (usually) turned off by adding -f switch to map definition.
> [ I have reported "missing -f" in socket as bug myself :-) ]
> 

Thanks ! It works now that I have added the -f swith to Kcyrus in
mrs_cyrus m4.



-- 
Clement Hermann (nodens)
- "L'air pur ? c'est pas en RL, ?a ? c'est pas hors charte ?"
Jean in L'Histoire des Pingouins, http://tnemeth.free.fr/fmbl/linuxsf/

Vous trouverez ma clef publique sur le serveur public pgp.mit.edu.
Please find my public key on the public keyserver pgp.mit.edu.

From mills at cc.umanitoba.ca  Sat Apr  4 09:19:07 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Sat, 4 Apr 2009 08:19:07 -0500
Subject: Graceful degradation in overload conditions
Message-ID: <20090404131907.GA14362@cc.umanitoba.ca>

We had a situation recently where our Cyrus back-end server
encountered a memory shortage.  The initial result was slow response,
but that provoked an increase in the number of imapd, pop3d, and lmtpd
processes, making the problem much worse.  The lmtpd processes
accumulated because of slow delivery of incoming e-mail, but the
others were likely a result of users starting more sessions.  The
results were not pretty!

What can be done to provide a graceful degradation of service in
overload conditions, so that the server protects itself against a
resource shortage?  I did put an upper limit on lmtpd children.
Sendmail will just queue incoming messages when this limit is reached.

What about imapd and pop3d daemons, which also consume resources?
Are limits a good idea here too?  Users will complain to the help
desk when those limits are reached, of course.  Can the msg/shutdown
file be used to control imapd processes in a nicer manner?

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From wes at umich.edu  Sat Apr  4 13:19:05 2009
From: wes at umich.edu (Wesley Craig)
Date: Sat, 4 Apr 2009 13:19:05 -0400
Subject: Graceful degradation in overload conditions
In-Reply-To: <20090404131907.GA14362@cc.umanitoba.ca>
References: <20090404131907.GA14362@cc.umanitoba.ca>
Message-ID: <C9C5EB9C-1B57-4FCD-BA58-0580BFDA78D3@umich.edu>

On 04 Apr 2009, at 09:19, Gary Mills wrote:
> What about imapd and pop3d daemons, which also consume resources?
> Are limits a good idea here too?  Users will complain to the help
> desk when those limits are reached, of course.  Can the msg/shutdown
> file be used to control imapd processes in a nicer manner?

I typically set the lmtpd limits very low, i.e., just above  
provisioned demand, because, as you say, inbound mail can queue.   
Setting imapd & popd limits at somewhat higher than peek demand  
causes a fairly user friendly service degradation: mail delivery more  
or less stops, mail reading can continue, but that too will stop  
before load is so bad that nothing can be done.  I also tend to  
actively monitor server load, so in extreme situations I can, e.g.,  
stop or severely limit mail delivery or other lesser services to  
favor more important services.  At some point, tho, you just need to  
add more capacity.

:wes

From Eric.Luyten at vub.ac.be  Mon Apr  6 05:27:07 2009
From: Eric.Luyten at vub.ac.be (Eric Luyten)
Date: Mon, 6 Apr 2009 11:27:07 +0200 (CEST)
Subject: Graceful degradation in overload conditions
In-Reply-To: <20090404131907.GA14362@cc.umanitoba.ca>
References: <20090404131907.GA14362@cc.umanitoba.ca>
Message-ID: <56857.134.184.15.103.1239010027.squirrel@nuts.vub.ac.be>

On Sat, April 4, 2009 3:19 pm, Gary Mills wrote:
> We had a situation recently where our Cyrus back-end server
> encountered a memory shortage.  The initial result was slow response, but
> that provoked an increase in the number of imapd, pop3d, and lmtpd
> processes, making the problem much worse.  The lmtpd processes accumulated
> because of slow delivery of incoming e-mail, but the others were likely a
> result of users starting more sessions.  The results were not pretty!
>
> What can be done to provide a graceful degradation of service in
> overload conditions, so that the server protects itself against a resource
> shortage?  I did put an upper limit on lmtpd children. Sendmail will just
> queue incoming messages when this limit is reached.

Gary,

For what it's worth : our server, a Solaris 9 V1280 with 32 Gigs of RAM
and eight processors, serves 55k users owning 400k mailboxes, 32M msgs
making for 2.7 TB of mailspool content.
Last week we registered the following number of Postfix deliveries into
the Cyrus spool :

   Mon 207878
   Tue 190345
   Wed 183913
   Thu 175530
   Fri 149533
   Sat  55441
   Sun  49292

I have the lmtp process limit set to 10 (ten), coming down from 20, which
make for a bit too many "DBERROR db3: NN lockers" warnings.
Our delivery DB is a bottleneck ; I'm planning carving up our Cyrus into
multiple instances, then moving to multiple (front/back) servers, this
summer, also upgrading from 2.2.13 to 2.3.LATEST

Cheers,
Eric Luyten, Computing Centre, Brussels Free Universities.



From morgan at orst.edu  Mon Apr  6 15:22:11 2009
From: morgan at orst.edu (Andrew Morgan)
Date: Mon, 6 Apr 2009 12:22:11 -0700 (PDT)
Subject: Graceful degradation in overload conditions
In-Reply-To: <20090404131907.GA14362@cc.umanitoba.ca>
References: <20090404131907.GA14362@cc.umanitoba.ca>
Message-ID: <alpine.DEB.2.00.0904061213270.16474@shell.onid.oregonstate.edu>

On Sat, 4 Apr 2009, Gary Mills wrote:

> We had a situation recently where our Cyrus back-end server
> encountered a memory shortage.  The initial result was slow response,
> but that provoked an increase in the number of imapd, pop3d, and lmtpd
> processes, making the problem much worse.  The lmtpd processes
> accumulated because of slow delivery of incoming e-mail, but the
> others were likely a result of users starting more sessions.  The
> results were not pretty!
>
> What can be done to provide a graceful degradation of service in
> overload conditions, so that the server protects itself against a
> resource shortage?  I did put an upper limit on lmtpd children.
> Sendmail will just queue incoming messages when this limit is reached.
>
> What about imapd and pop3d daemons, which also consume resources?
> Are limits a good idea here too?  Users will complain to the help
> desk when those limits are reached, of course.  Can the msg/shutdown
> file be used to control imapd processes in a nicer manner?

You can set the lmtpd limits pretty low without affecting mail delivery. 
We limit to 100 here, which is still probably too high.

The limit on imapd processes really depends on your hardware.  If you can 
make some reasonable guess about the incremental amount of memory used by 
each imapd, then you could set the limit to something that wouldn't 
exhaust all the memory on the server.  That's what I've done here.  We set 
the limit to 1500 imapds processes on our 4GB server.  Our normal load is 
around 700 imapds, so that leaves lots of memory for caching I/O.

 	Andy

From jblaine at kickflop.net  Mon Apr  6 18:49:09 2009
From: jblaine at kickflop.net (Jeff Blaine)
Date: Mon, 06 Apr 2009 18:49:09 -0400
Subject: Weekly/Monthly record-keeping / maintenance?
In-Reply-To: <49D544C6.5020106@kickflop.net>
References: <49D2403C.50908@kickflop.net>
	<200903312243.18875.ml@awinkelmann.de>
	<49D281FF.90305@kickflop.net>
	<EE19A1EE-C3AC-4BCF-980D-C77A40BB9C37@umich.edu>
	<49D3C78F.5080908@kickflop.net> <49D4CECF.6040101@kickflop.net>
	<20090402221955.GB6898@brong.net> <49D544C6.5020106@kickflop.net>
Message-ID: <49DA86E5.90502@kickflop.net>

I lied, quoting works when you use the right quotes.

Double quotes, not single.

    my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");

should become

    my ($rc, $msg) = $cyrus->send('', '', "EXAMINE \"$mb\"");

... for a better imapdu.pl

Jeff Blaine wrote:
> I tried both
> 
> a) escaping spaces in $mb
> b) quoting $mb
> 
> Neither worked.
> 
> That's about as far as I delve into Perl anymore, so
> I'm happy to just retract my note to our users about
> this tool.
> 
> Bron Gondwana wrote:
>> On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
>>> Okay, well, for what it's worth, I fixed all of the
>>> problems prohibiting me from running a quota -f
>>> to completion.
>>>
>>> The problem: imapdu.pl is buggy
>>
>> Doh.
>>
>>> It fails to do the right thing with mailboxes containing
>>> a space in the name.
>>
>> Yeah, that's not entirely a surprise.  Spaces in names
>> confuse lots of stuff.
>>
>>> ...
>>> 1.46 MB 114 msgs INBOX.BMP
>>> 1.46 MB 114 msgs INBOX.Bio Stuff
>>> 0.00 bytes 0 msgs INBOX.Drafts
>>> 1.25 MB 36 msgs INBOX.HLT
>>> 1.25 MB 36 msgs INBOX.Information Retrieval
>>> ...
>>>
>>> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
>>>
>>> I don't suppose 'murch', the author of the code reads
>>> this list?
>>
>> Yeah, he's around, though he doesn't always see stuff as quickly
>> if it's only sent to the list.  I've CC'd him.
>>
>> That said, he's likely not the author - just the last person that
>> changed things.  Given that he did a giant sweeping copyright
>> update of nearly every file in the tree a couple of months ago...
>>
>>   my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
>>   if ($rc eq 'OK') {
>>   } else {
>>       print "failed: $mb: $msg\n";
>>   }
>>
>> Apart from being icky perl, that will fail to change mailboxes
>> because the EXAMINE command will have dodgy syntax.  I'm not
>> entirely sure why you're not seeing the 'failed' messages
>> though...  The "$mb" needs to be at least quoted - which is
>> why I generally use something like Mail::IMAPTalk that can
>> do correct protocol quoting.
>>
>> Ahh - it probably does output the "failed" stuff further up.
>>  
>> Bron.
>>
> 

From jwidera at oakton.edu  Fri Apr 10 14:35:03 2009
From: jwidera at oakton.edu (John Widera)
Date: Fri, 10 Apr 2009 13:35:03 -0500 (CDT)
Subject: Cyrus on 64-bit RHEL 5
Message-ID: <3005.10.70.3.113.1239388503.squirrel@borg>

We are in the planning stage of a major mail systems upgrade.  We've been
running Cyrus on 32-bit RHEL 5 for some time now.  We are considering 2
major changes that could impact administration and performance.

1) we're considering running Cyrus under VMware ESX Server.  We have one
Cyrus installion with about 2000 users, half of them very active users,
with a TOTAL spool size of of about 250GB, and another installion which
has more users but is under MUCH less load, and a TOTAL spool size of
~50GB.

2) we're also strongly considering running a 64-bit OS version of RHEL 5
to take advantage of the increased FS buffer caching.  Esp. in the busier
installation we suspect this will further improve performance (we're not
unhappy with current performance, but we foresee significant increase in
usage over time).

------------------

Can anyone share with us their production experience with running Cyrus in
a virtual (ESX?) env., and/or experience with running Cyrus (Simon
Matter's RPMs, optimally) in a 64-bit Linux OS?

We'd really appreciate hearing from people before we go much further in
the planning stage.  Thanks.

John Widera
jwidera #at# oakton #dot# edu





From pnfisher at berkeley.edu  Fri Apr 10 17:33:02 2009
From: pnfisher at berkeley.edu (Paul Fisher)
Date: Fri, 10 Apr 2009 14:33:02 -0700
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <3005.10.70.3.113.1239388503.squirrel@borg>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
Message-ID: <49DFBB0E.1080906@berkeley.edu>

John Widera wrote:
>
> Can anyone share with us their production experience with running
> Cyrus in a virtual (ESX?) env., and/or experience with running Cyrus
> (Simon Matter's RPMs, optimally) in a 64-bit Linux OS?

Berkeley runs Simon Matter's RPMs on 64-bit (x86-64) RHEL5, and we
haven't had any issues.  Performance has been fine.  We support around
70K users.

Our initial deployment of Cyrus was on 64-bit, so I'm not sure if
you'll run into any problems as part of the conversion.

Paul

From simon.matter at invoca.ch  Fri Apr 10 17:49:54 2009
From: simon.matter at invoca.ch (Simon Matter)
Date: Fri, 10 Apr 2009 23:49:54 +0200 (CEST)
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <3005.10.70.3.113.1239388503.squirrel@borg>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
Message-ID: <bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>

> Can anyone share with us their production experience with running Cyrus in
> a virtual (ESX?) env., and/or experience with running Cyrus (Simon
> Matter's RPMs, optimally) in a 64-bit Linux OS?

I'm not a fan of running anything that needs good performance on any kind
of virtualization. At least you me really check that you get good IO
performance out of it.

Running Cyrus on x86_64 Linux seems to work well. At least I didn't get
any problem report for a long time.
I have not been a fan of x86_64 because of it's mixed 32/64bit arch but
that's not really a problem these days. I was forced to migrate a server
from RHEL3/i386 to RHEL5/x86_64 and it has just worked. Both were running
my current RPMs and with the default database config used in the RPMs. I
could just shutdown the old box, rsync -aH to the new box and start it up.
However, you may search the archives because I think there can be problems
with certain database backends (BDB) when doing the migration. The RPM
should still take care of it because it converts all BDB to skiplist on
shutdown to make things easy to migrate.

Regards,
Simon


From pkatsovich at vtsystems.com  Fri Apr 10 17:54:22 2009
From: pkatsovich at vtsystems.com (Paul Katsovich)
Date: Fri, 10 Apr 2009 17:54:22 -0400
Subject: adding server to murder
Message-ID: <1239400462.28248.17.camel@devilbox.ad.nyo.fxserver.com>

Hi list,


I'm trying to add an existing cyrus-imap server (additional backend) to
an existing cyrus-murder.

Unfortunatelly I've run into a wall and the only error messages I'm
getting are:

cyrus/lmtpunix[8569]: kick_mupdate: can't connect to target: No such
file or directory

When I try to run: "ctl_mboxlist -mw" it returns:
"couldn't connect to mupdate server"

and this appears in my maillog:
"cyrus/ctl_mboxlist[9125]: connect(mupdateserver.domain.com) failed:
Invalid argument"

I have no idea where the invalid argument could be as I've compared
imapd.conf and cyrus.conf to the existing back ends, and everything
appears to be in place.

I've also confirmed that the various paths are accurate and the cyrus
user has necessary permissions.


I've also run mupdatetest, successfully.

cyrus at back_end$ mupdatetest mupdateserver.domain.com
S: * AUTH "CRAM-MD5" "PLAIN"
S: * STARTTLS
S: * PARTIAL-UPDATE
S: * OK MUPDATE "mupdateserver.domain.com" "Cyrus Murder"
"v2.3.7-Invoca-RPM-2.3.7-2.el5" "(master)"
C: A01 AUTHENTICATE "CRAM-MD5"
S: PDM1NTIwOTI4Ni4xNDY2MzM3MEBuanJoLWVtbDUxMC5meHNlcnZlci5jb20+
Please enter your password: 
C: Y3lydXMgMzg1ZTYyNTE0Nzc5YmI1ZGRlYWI1Nzk2MGJmNmMxY2M=
S: A01 OK "Authenticated"
Authenticated.
Security strength factor: 0


I've tried adding CYRUS_VERBOSE=2 , unfortunately that didn't produce
any more useful logging.

<imapd.conf>

configdirectory: /path/configdir
partition-default: /path/imap

admins: cyrus
sievedir: /path/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true

allowplaintext: yes
allowusermoves: 1
allowallsubscribe: yes

sasl_pwcheck_method: saslauthd
sasl_mech_list: plain starttls cram-md5

tls_cert_file: /path/our_cert.pem
tls_key_file: /path/our_cert.pem
tls_ca_file: /path/our_cert.pem


mupdate_server: mupdateserver.domain.com
mupdate_username: cyrus
mupdate_authname: cyrus
mupdate_port: 3905
mupdate_password: pw
mupdate_config: standard

proxy_authname: user
proxy_username: user
proxy_password: pw


lmtp_downcase_rcpt: 1
username_tolower: 1
normalizeuid: 1


defaultdomain: domain.com
virtdomains: userid

syslog_prefix: cyrus


Regards,

-  
Paul Katsovich



From morgan at orst.edu  Fri Apr 10 18:50:13 2009
From: morgan at orst.edu (Andrew Morgan)
Date: Fri, 10 Apr 2009 15:50:13 -0700 (PDT)
Subject: adding server to murder
In-Reply-To: <1239400462.28248.17.camel@devilbox.ad.nyo.fxserver.com>
References: <1239400462.28248.17.camel@devilbox.ad.nyo.fxserver.com>
Message-ID: <alpine.DEB.2.00.0904101548480.15756@shell.onid.oregonstate.edu>

On Fri, 10 Apr 2009, Paul Katsovich wrote:

> Hi list,
>
>
> I'm trying to add an existing cyrus-imap server (additional backend) to
> an existing cyrus-murder.
>
> Unfortunatelly I've run into a wall and the only error messages I'm
> getting are:
>
> cyrus/lmtpunix[8569]: kick_mupdate: can't connect to target: No such
> file or directory
>
> When I try to run: "ctl_mboxlist -mw" it returns:
> "couldn't connect to mupdate server"
>
> and this appears in my maillog:
> "cyrus/ctl_mboxlist[9125]: connect(mupdateserver.domain.com) failed:
> Invalid argument"
>
> I have no idea where the invalid argument could be as I've compared
> imapd.conf and cyrus.conf to the existing back ends, and everything
> appears to be in place.

Do you have an entry for mupdate in your /etc/services file?

mupdate         3905/tcp                        # Cyrus murder master


You might also try running strace on "ctl_mboxlist -mw" to see which 
system call is failing.

 	Andy

From brong at fastmail.fm  Fri Apr 10 21:01:52 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Sat, 11 Apr 2009 11:01:52 +1000
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
	<bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <20090411010152.GA23060@brong.net>

On Fri, Apr 10, 2009 at 11:49:54PM +0200, Simon Matter wrote:
> However, you may search the archives because I think there can be problems
> with certain database backends (BDB) when doing the migration. The RPM
> should still take care of it because it converts all BDB to skiplist on
> shutdown to make things easy to migrate.

I've always wondered why you didn't just convert to flat file for
migration.  It's simple and easier to read!

Bron ( ok, so _I_ can read skiplist files in a hex editor these days,
       but I don't wish the learning process on anybody )

From simon.matter at invoca.ch  Sat Apr 11 03:35:20 2009
From: simon.matter at invoca.ch (Simon Matter)
Date: Sat, 11 Apr 2009 09:35:20 +0200 (CEST)
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <20090411010152.GA23060@brong.net>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
	<bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
	<20090411010152.GA23060@brong.net>
Message-ID: <bbcae0373a1c11850bff136e6458dfdc.squirrel@webmail.bi.corp.invoca.ch>

> On Fri, Apr 10, 2009 at 11:49:54PM +0200, Simon Matter wrote:
>> However, you may search the archives because I think there can be
>> problems
>> with certain database backends (BDB) when doing the migration. The RPM
>> should still take care of it because it converts all BDB to skiplist on
>> shutdown to make things easy to migrate.
>
> I've always wondered why you didn't just convert to flat file for
> migration.  It's simple and easier to read!

That's true but aren't there file which can not be converted to flat
files, like duplicate_db? I don't remember why I decided to use skiplist
but I think there was something with flat?

Regards,
Simon

>
> Bron ( ok, so _I_ can read skiplist files in a hex editor these days,
>        but I don't wish the learning process on anybody )

PS: That's nothing - looking into my mailbox some people seem to think I
can read M$ Word files...



From dbucherml at hsolutions.ch  Sun Apr 12 16:14:48 2009
From: dbucherml at hsolutions.ch (Denis BUCHER)
Date: Sun, 12 Apr 2009 22:14:48 +0200
Subject: Searching for a Sieve web Autoreplier OR Outlook solution
Message-ID: <49E24BB8.7040404@hsolutions.ch>

Hello,

I'm searching for months without success for either one of these solutions :

a) Out-of-office (away) plugin for Outlook that would use Sieve ?

b) Web interface to manage an out-of-office message in Sieve ?

If someone could give me some advice, it would be very nice !

Denis

From awilliam at whitemice.org  Sun Apr 12 17:15:54 2009
From: awilliam at whitemice.org (Adam Tauno Williams)
Date: Sun, 12 Apr 2009 16:15:54 -0500
Subject: Searching for a Sieve web Autoreplier OR Outlook solution
In-Reply-To: <49E24BB8.7040404@hsolutions.ch>
References: <49E24BB8.7040404@hsolutions.ch>
Message-ID: <8873-SnapperMsg05076A73C6080A93@[75.205.131.203]>

>I'm searching for months without success for either one of these solutions :
>a) Out-of-office (away) plugin for Outlook that would use Sieve ?

No idea.

>b) Web interface to manage an out-of-office message in Sieve ?

Horde's Ingo provides a very nice web interface for configuring various, including SIEVE, mail 
filters.   There is also the old (unmaintained?) SmartSIEVE application.

From jmadden at ivytech.edu  Mon Apr 13 09:59:18 2009
From: jmadden at ivytech.edu (John Madden)
Date: Mon, 13 Apr 2009 09:59:18 -0400
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
	<bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <1239631158.21421.81.camel@quagmire>

On Fri, 2009-04-10 at 23:49 +0200, Simon Matter wrote:
> I'm not a fan of running anything that needs good performance on any
> kind
> of virtualization. At least you me really check that you get good IO
> performance out of it.

FWIW, building from source on RHEL 5 on a Xen DomU with 6 cores and 2GB
memory, currently delivering around 150msgs/sec via postal to ~350k
accounts (via LDAP).  Storage is all "Xen phy:" storage to a EMC DMX
(metadata) and IBM 4700 (SATA, mail storage), 6 LUNs of each.  I'm
pretty happy with those numbers.  Raw hardware would of course be
better, but I find the overhead of Xen to be worthwhile.

John




-- 
John Madden
Sr UNIX Systems Engineer
Ivy Tech Community College of Indiana
jmadden at ivytech.edu


From jwidera at oakton.edu  Mon Apr 13 12:24:56 2009
From: jwidera at oakton.edu (John Widera)
Date: Mon, 13 Apr 2009 11:24:56 -0500 (CDT)
Subject: Searching for a Sieve web Autoreplier OR Outlook solution
In-Reply-To: <49E24BB8.7040404@hsolutions.ch>
References: <49E24BB8.7040404@hsolutions.ch>
Message-ID: <1963.10.70.3.113.1239639896.squirrel@borg>

> Hello,
>
> I'm searching for months without success for either one of these solutions
> :
>
> a) Out-of-office (away) plugin for Outlook that would use Sieve ?
>
> b) Web interface to manage an out-of-office message in Sieve ?

Websieve?  Easysieve?

>
> If someone could give me some advice, it would be very nice !
>
> Denis
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>
>



From jwidera at oakton.edu  Mon Apr 13 13:04:46 2009
From: jwidera at oakton.edu (John Widera)
Date: Mon, 13 Apr 2009 12:04:46 -0500 (CDT)
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
	<bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <2218.10.70.3.113.1239642286.squirrel@borg>

Thanks all, for the input.  My post had a typo - we aren't currently on
RHEL 5.  We're on RHEL 3.  RHEL 3/i386 to RHEL 5/x86_64 is the exact path
we are going.

We're already running skiplist across the board so based on what we've
read/heard we don't expect any trouble in that regard.

Our plan is to build from cyrus-imapd-2.3.14-2.src.rpm.

But which of Simon's SASL RPMs is recommended for x86_64 linux?

--
We have another q. regarding FS buffer cache, but I'll post that separately.

Thanks again everyone.

>> Can anyone share with us their production experience with running Cyrus
>> in
>> a virtual (ESX?) env., and/or experience with running Cyrus (Simon
>> Matter's RPMs, optimally) in a 64-bit Linux OS?
>
> I'm not a fan of running anything that needs good performance on any kind
> of virtualization. At least you me really check that you get good IO
> performance out of it.
>
> Running Cyrus on x86_64 Linux seems to work well. At least I didn't get
> any problem report for a long time.
> I have not been a fan of x86_64 because of it's mixed 32/64bit arch but
> that's not really a problem these days. I was forced to migrate a server
> from RHEL3/i386 to RHEL5/x86_64 and it has just worked. Both were running
> my current RPMs and with the default database config used in the RPMs. I
> could just shutdown the old box, rsync -aH to the new box and start it up.
> However, you may search the archives because I think there can be problems
> with certain database backends (BDB) when doing the migration. The RPM
> should still take care of it because it converts all BDB to skiplist on
> shutdown to make things easy to migrate.
>
> Regards,
> Simon
>
>
>



From simon.matter at invoca.ch  Mon Apr 13 14:09:14 2009
From: simon.matter at invoca.ch (Simon Matter)
Date: Mon, 13 Apr 2009 20:09:14 +0200 (CEST)
Subject: Cyrus on 64-bit RHEL 5
In-Reply-To: <2218.10.70.3.113.1239642286.squirrel@borg>
References: <3005.10.70.3.113.1239388503.squirrel@borg>
	<bc035b13d6c9d64f489f29bd47a0d2e2.squirrel@webmail.bi.corp.invoca.ch>
	<2218.10.70.3.113.1239642286.squirrel@borg>
Message-ID: <acd6a0d9a7f6561b5023a4fe62966a70.squirrel@webmail.bi.corp.invoca.ch>

> Thanks all, for the input.  My post had a typo - we aren't currently on
> RHEL 5.  We're on RHEL 3.  RHEL 3/i386 to RHEL 5/x86_64 is the exact path
> we are going.
>
> We're already running skiplist across the board so based on what we've
> read/heard we don't expect any trouble in that regard.
>
> Our plan is to build from cyrus-imapd-2.3.14-2.src.rpm.
>
> But which of Simon's SASL RPMs is recommended for x86_64 linux?

Hi John,

For SASL, simply take the package from the distribution. The only reason
why I made my own SASL packages was that older distributions delivered too
old SASL versions. The version from RHEL5 is perfect for cyrus-imapd.

Regards,
Simon



From glad at cs.au.dk  Mon Apr 13 15:29:11 2009
From: glad at cs.au.dk (Michael Glad)
Date: Mon, 13 Apr 2009 21:29:11 +0200
Subject: How to clean up after syncserver
Message-ID: <49E39287.9020705@cs.au.dk>

Cyrus being restarted / sync server abending, apparently causes it to 
leave sub dirs in the 'sync.' dirs containing hard links  to messages.
They currently sum to 40k+ links on one of my replicas :-( .

I noticed them during an yet unsuccessful attempt to find out why 
message body inconsistencies now and then occur
between master and replica.

The 40k+ entries annoy me and I have a uneasy feeling that they may be 
involved in the creating the inconsistencies.
So is there a way to clean up the  sync. dirs --  can one just remove 
the sub dirs / hard links while the sync server is down?

 -- Michael

------------------------
The inconsistencies seem to arise when a user delete a message, thereby 
copying it to a trash folder.
Now and then, this action on the replica nukes a message file owned by 
another user on the same cyrus partition, so
that the trash file and the two user mailbox files are now hardlinked 
together.

Environment: 2.3.14 + FM patches as of 2009-03-31 on RHEL 5.3/x86_64.
Single instance store + fast rename + delayed delete + delayed expunge



From ktm at rice.edu  Mon Apr 13 15:34:09 2009
From: ktm at rice.edu (Kenneth Marshall)
Date: Mon, 13 Apr 2009 14:34:09 -0500
Subject: How to clean up after syncserver
In-Reply-To: <49E39287.9020705@cs.au.dk>
References: <49E39287.9020705@cs.au.dk>
Message-ID: <20090413193409.GC18845@it.is.rice.edu>

We are currently cleaning up after a similar analysis. Our current
plan is to clean up the sync. dirs on a restart of the sync_client.
If there is no advantage to saving state, maybe the sync_server could
do this automatically.

Cheers,
Ken

On Mon, Apr 13, 2009 at 09:29:11PM +0200, Michael Glad wrote:
> Cyrus being restarted / sync server abending, apparently causes it to 
> leave sub dirs in the 'sync.' dirs containing hard links  to messages.
> They currently sum to 40k+ links on one of my replicas :-( .
> 
> I noticed them during an yet unsuccessful attempt to find out why 
> message body inconsistencies now and then occur
> between master and replica.
> 
> The 40k+ entries annoy me and I have a uneasy feeling that they may be 
> involved in the creating the inconsistencies.
> So is there a way to clean up the  sync. dirs --  can one just remove 
> the sub dirs / hard links while the sync server is down?
> 
>  -- Michael
> 
> ------------------------
> The inconsistencies seem to arise when a user delete a message, thereby 
> copying it to a trash folder.
> Now and then, this action on the replica nukes a message file owned by 
> another user on the same cyrus partition, so
> that the trash file and the two user mailbox files are now hardlinked 
> together.
> 
> Environment: 2.3.14 + FM patches as of 2009-03-31 on RHEL 5.3/x86_64.
> Single instance store + fast rename + delayed delete + delayed expunge
> 
> 
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> 

From brong at fastmail.fm  Mon Apr 13 19:51:44 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Tue, 14 Apr 2009 09:51:44 +1000
Subject: How to clean up after syncserver
In-Reply-To: <49E39287.9020705@cs.au.dk>
References: <49E39287.9020705@cs.au.dk>
Message-ID: <1239666704.2816.1310328043@webmail.messagingengine.com>

On Mon, 13 Apr 2009 21:29 +0200, "Michael Glad" <glad at cs.au.dk> wrote:
> Cyrus being restarted / sync server abending, apparently causes it to 
> leave sub dirs in the 'sync.' dirs containing hard links  to messages.
> They currently sum to 40k+ links on one of my replicas :-( .

That's a lot of sync server failures!

> I noticed them during an yet unsuccessful attempt to find out why 
> message body inconsistencies now and then occur
> between master and replica.

We should have fixed that a while back - but yes, for a little while it would
just open the hardlinked file and truncate it without unlinking first - causing
random corruption.  Really, REALLY annoying.  The bug certainly doesn't
exist in 2.3.14 - I spent quite a while auditing all the paths that open files!

> The 40k+ entries annoy me and I have a uneasy feeling that they may be 
> involved in the creating the inconsistencies.
> So is there a way to clean up the  sync. dirs --  can one just remove 
> the sub dirs / hard links while the sync server is down?

Yes, you certainly can.

> ------------------------
> The inconsistencies seem to arise when a user delete a message, thereby 
> copying it to a trash folder.
> Now and then, this action on the replica nukes a message file owned by 
> another user on the same cyrus partition, so
> that the trash file and the two user mailbox files are now hardlinked 
> together.

It's any COPY action actually.  Trash is just a common target for copies.

> Environment: 2.3.14 + FM patches as of 2009-03-31 on RHEL 5.3/x86_64.
> Single instance store + fast rename + delayed delete + delayed expunge

Yikes... I really should go audit that fastrename code again :)  We're not
running it ourselves due to some worries about concurrency safety... I'm
pretty sure it's OK actually, but it could do with a re-read.

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm


From glad at cs.au.dk  Tue Apr 14 03:43:10 2009
From: glad at cs.au.dk (Michael Glad)
Date: Tue, 14 Apr 2009 09:43:10 +0200
Subject: How to clean up after syncserver
In-Reply-To: <1239666704.2816.1310328043@webmail.messagingengine.com>
References: <49E39287.9020705@cs.au.dk>
	<1239666704.2816.1310328043@webmail.messagingengine.com>
Message-ID: <49E43E8E.2030100@cs.au.dk>

Bron Gondwana wrote:
> On Mon, 13 Apr 2009 21:29 +0200, "Michael Glad" <glad at cs.au.dk> wrote:
>   
>> Cyrus being restarted / sync server abending, apparently causes it to 
>> leave sub dirs in the 'sync.' dirs containing hard links  to messages.
>> They currently sum to 40k+ links on one of my replicas :-( .
>>     
>
> That's a lot of sync server failures!
>
>   
I restart cyrus on my replicas once a day as part of the backup 
procedure. And whatever hard links
that sync_server has left in its sync. subdirs contribute to the 40k count.

And it seems that sync_server has a potential of leaving links:

[root at replica# date; find sync. g/user/glad -inum 136643179 -ls
Tue Apr 14 09:25:44 CEST 2009
136643179    4 -rw-------   3 cyrus    mail          935 Apr 14 09:15 
sync./29594/65.
136643179    4 -rw-------   3 cyrus    mail          935 Apr 14 09:15 
g/user/glad/71229.
136643179    4 -rw-------   3 cyrus    mail          935 Apr 14 09:15 
g/user/glad/Incoming/499.

Why doesn't it (after 10 minutes & counting) remove the sync./29594/65. 
link after successfully having created the
two proper  links (I have a Sieve script that files a copy of all mails 
in the Incoming folder)?

As I write this sentence, 15 minutes have passed and the link is gone. 
But leaving all those links around for extended
periods seems like an invitation for bad  things to happen and for 
garbage to pile up. :-(

>> I noticed them during an yet unsuccessful attempt to find out why 
>> message body inconsistencies now and then occur
>> between master and replica.
>>     
>
> We should have fixed that a while back - but yes, for a little while it would
> just open the hardlinked file and truncate it without unlinking first - causing
> random corruption.  Really, REALLY annoying.  The bug certainly doesn't
> exist in 2.3.14 - I spent quite a while auditing all the paths that open files!
>   
I've browsed the sync_server code and have noticed that care is being 
taken not to overwrite
left-over links but it seems that this is actually still happening. I'm 
quite sure that I see the
problems with 2.3.14. 
>> The 40k+ entries annoy me and I have a uneasy feeling that they may be 
>> involved in the creating the inconsistencies.
>> So is there a way to clean up the  sync. dirs --  can one just remove 
>> the sub dirs / hard links while the sync server is down?
>>     
>
> Yes, you certainly can.
>
>   
>> ------------------------
>> The inconsistencies seem to arise when a user delete a message, thereby 
>> copying it to a trash folder.
>> Now and then, this action on the replica nukes a message file owned by 
>> another user on the same cyrus partition, so
>> that the trash file and the two user mailbox files are now hardlinked 
>> together.
>>     
>
> It's any COPY action actually.  Trash is just a common target for copies.
>
>   
>> Environment: 2.3.14 + FM patches as of 2009-03-31 on RHEL 5.3/x86_64.
>> Single instance store + fast rename + delayed delete + delayed expunge
>>     
>
> Yikes... I really should go audit that fastrename code again :)  We're not
> running it ourselves due to some worries about concurrency safety... I'm
> pretty sure it's OK actually, but it could do with a re-read.
>   
OK.

    -- Michael

From dom.lalot at gmail.com  Tue Apr 14 04:32:28 2009
From: dom.lalot at gmail.com (LALOT Dominique)
Date: Tue, 14 Apr 2009 10:32:28 +0200
Subject: limit tcp sessions opened by an IMAP client
Message-ID: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>

Hello,

I've looked at google before asking, but I didn't find something.
Some imap clients are using many tcp connexions. I would like to know if
there is a way to limit them?

Thanks

Dom

-- 
Dominique LALOT
Ing?nieur Syst?mes et R?seaux
http://annuaire.univmed.fr/showuser.php?uid=lalot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090414/8b0e3e77/attachment.html 

From brong at fastmail.fm  Tue Apr 14 04:45:20 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Tue, 14 Apr 2009 18:45:20 +1000
Subject: How to clean up after syncserver
In-Reply-To: <49E43E8E.2030100@cs.au.dk>
References: <49E39287.9020705@cs.au.dk>
	<1239666704.2816.1310328043@webmail.messagingengine.com>
	<49E43E8E.2030100@cs.au.dk>
Message-ID: <20090414084520.GA10012@brong.net>

On Tue, Apr 14, 2009 at 09:43:10AM +0200, Michael Glad wrote:
> Bron Gondwana wrote:
> > On Mon, 13 Apr 2009 21:29 +0200, "Michael Glad" <glad at cs.au.dk> wrote:
> >   
> >> Cyrus being restarted / sync server abending, apparently causes it to 
> >> leave sub dirs in the 'sync.' dirs containing hard links  to messages.
> >> They currently sum to 40k+ links on one of my replicas :-( .
> >>     
> >
> > That's a lot of sync server failures!
> >
> >   
> I restart cyrus on my replicas once a day as part of the backup 
> procedure. And whatever hard links

Gosh - what a pain.  I would get too annoyed by the "sync_client
bailing out" notifications (we have a daemon that watches the
imapd.log and makes a noise for anything that's not everyday
notices)

> that sync_server has left in its sync. subdirs contribute to the 40k count.
> 
> And it seems that sync_server has a potential of leaving links:
> 
> [root at replica# date; find sync. g/user/glad -inum 136643179 -ls
> Tue Apr 14 09:25:44 CEST 2009
> 136643179    4 -rw-------   3 cyrus    mail          935 Apr 14 09:15 
> sync./29594/65.
> 136643179    4 -rw-------   3 cyrus    mail          935 Apr 14 09:15 
> g/user/glad/71229.
> 136643179    4 -rw-------   3 cyrus    mail          935 Apr 14 09:15 
> g/user/glad/Incoming/499.
> 
> Why doesn't it (after 10 minutes & counting) remove the sync./29594/65. 
> link after successfully having created the
> two proper  links (I have a Sieve script that files a copy of all mails 
> in the Incoming folder)?

In a single sync run it will keep all temporary files around
just in case there's a COPY coming up that it wants to serve
from the existing file (thanks to GUID uniqueness it can tell
that it's the same file)

Once the single run is finished, then it will unlink all the
files it used.

The reason it's taking so long - probably because you stopped
the replica for a while, so it has a big file to process.  We
split them into 10k chunks when restarting... just so that
progress can be monitored a bit more easily.

> As I write this sentence, 15 minutes have passed and the link is gone. 
> But leaving all those links around for extended
> periods seems like an invitation for bad  things to happen and for 
> garbage to pile up. :-(

So don't shut down your replica then.  We do our backups by
fcntl locking the cyrus.* files (header, index, expunge in that
order) and then duplicating them.  There's no need to lock the
data files - they never change, so you either get the whole
thing or nothing - and if nothing then the file was already
deleted by now.

I'm tempted to write a backup utility that can give you a
consistent snapshot of a single user as a tar file.  The
paths would be such that you could just concatenate a whole
lot of them together and gzip it as the backup!

(also an incremental one - given a previous tar file, back
up just what's changed)
 
> >> I noticed them during an yet unsuccessful attempt to find out why 
> >> message body inconsistencies now and then occur
> >> between master and replica.
> >>     
> >
> > We should have fixed that a while back - but yes, for a little while it would
> > just open the hardlinked file and truncate it without unlinking first - causing
> > random corruption.  Really, REALLY annoying.  The bug certainly doesn't
> > exist in 2.3.14 - I spent quite a while auditing all the paths that open files!
> >   
> I've browsed the sync_server code and have noticed that care is being 
> taken not to overwrite
> left-over links but it seems that this is actually still happening. I'm 
> quite sure that I see the
> problems with 2.3.14. 

I'm guessing they are old ones.  Have you consistency checked 
your entire server?  If your replica has ALWAYS been your
replica then a reconstruct -G on every folder followed by a 
sync_client -u on every user should fix it (it will take
a little while, but at least it can be done one user at a
time with a reconstruct -r -G and then a sync_client -u)

Regards,

Bron ( annoyingly, I haven't set reconstruct to tell you if
       the GUID of any message changes.  I probably should! )

From mayak at australsat.com  Tue Apr 14 05:04:33 2009
From: mayak at australsat.com (mayak chunder-qwern)
Date: Tue, 14 Apr 2009 11:04:33 +0200
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
Message-ID: <1239699873.30777.0.camel@sumatra.ccuse.com>

bonjour dominique,

iptables is well suited to do this. google rate limiting and maximum
connections.

cheers

mayak

On Tue, 2009-04-14 at 10:32 +0200, LALOT Dominique wrote:
> Hello,
> 
> I've looked at google before asking, but I didn't find something.
> Some imap clients are using many tcp connexions. I would like to know
> if there is a way to limit them?
> 
> Thanks
> 
> Dom
> 
> -- 
> Dominique LALOT
> Ing?nieur Syst?mes et R?seaux
> http://annuaire.univmed.fr/showuser.php?uid=lalot
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


From mathieu.kretchner at sophia.inria.fr  Tue Apr 14 05:27:50 2009
From: mathieu.kretchner at sophia.inria.fr (Mathieu Kretchner)
Date: Tue, 14 Apr 2009 11:27:50 +0200
Subject: restore seen file
Message-ID: <49E45716.8080002@sophia.inria.fr>

Hello,

Is it possible to restore seen file even if the mailbox has changed ?

If yes how ?

thanks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mathieu_kretchner.vcf
Type: text/x-vcard
Size: 258 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090414/c9cb68b2/attachment.vcf 

From brennan at columbia.edu  Tue Apr 14 09:06:26 2009
From: brennan at columbia.edu (Joseph Brennan)
Date: Tue, 14 Apr 2009 09:06:26 -0400
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
Message-ID: <906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>


LALOT Dominique <dom.lalot at gmail.com> wrote:

> Hello,
>
> I've looked at google before asking, but I didn't find something.
> Some imap clients are using many tcp connexions. I would like to know if
> there is a way to limit them?


This could make the client fail and increase your helpdesk calls.  Do
you mean more than five?

Whatever you do should check both host and user, so that you don't cut
off multiple users on a timeshare host or a firewall gateway.


Joseph Brennan
Lead Email Systems Engineer
Columbia University Information Technology



From dom.lalot at gmail.com  Tue Apr 14 10:53:25 2009
From: dom.lalot at gmail.com (LALOT Dominique)
Date: Tue, 14 Apr 2009 16:53:25 +0200
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
	<906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>
Message-ID: <1617f8010904140753r3d1842b6hc4b734f3ca9a2b1f@mail.gmail.com>

Look at this one:

[root at smtp ~]# host 82.240.88.126
126.88.240.82.in-addr.arpa domain name pointer
val13-2-82-240-88-126.fbx.proxad.net.
[root at smtp ~]# netstat -atpn | grep 82.240.88.126
tcp        0      0 139.124.132.126:993         82.240.88.126:60250
ESTABLISHED 9209/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60229
ESTABLISHED 8824/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60235
ESTABLISHED 8016/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60234
ESTABLISHED 8570/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60265
ESTABLISHED 10316/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60180
ESTABLISHED 3795/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60190
ESTABLISHED 5258/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60167
ESTABLISHED 5882/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60213
ESTABLISHED 6758/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60219
ESTABLISHED 8421/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60196
ESTABLISHED 7486/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60206
ESTABLISHED 7520/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:63218
ESTABLISHED 6288/imapd
tcp        0      0 139.124.132.126:993         82.240.88.126:60158
ESTABLISHED 5504/imapd

I don't know how many processes we can have with a decent speed. For the
moment, it turns to be around 1000 processes, but I don't know the max whe
can stand.
So the idea of mayak can be  a solution. Filter with iptables

193.218.15.25 13
82.240.88.126 16
80.13.69.148 12

for the top, I got lines like this:
Apr 14 16:10:25 smtp imaps[13462]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:11:43 smtp imaps[13530]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:11:43 smtp imaps[31581]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:12:41 smtp imaps[13644]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:12:42 smtp imaps[13481]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:15:08 smtp imaps[14234]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:15:08 smtp imaps[29088]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:17:14 smtp imaps[14080]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in
Apr 14 16:17:15 smtp imaps[14212]: login:
val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] xxxxx plaintext+TLS
User logged in

Checking mail a little bit too much.



2009/4/14 Joseph Brennan <brennan at columbia.edu>

>
> LALOT Dominique <dom.lalot at gmail.com> wrote:
>
> > Hello,
> >
> > I've looked at google before asking, but I didn't find something.
> > Some imap clients are using many tcp connexions. I would like to know if
> > there is a way to limit them?
>
>
> This could make the client fail and increase your helpdesk calls.  Do
> you mean more than five?
>
> Whatever you do should check both host and user, so that you don't cut
> off multiple users on a timeshare host or a firewall gateway.
>
>
> Joseph Brennan
> Lead Email Systems Engineer
> Columbia University Information Technology
>
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>



-- 
Dominique LALOT
Ing?nieur Syst?mes et R?seaux
http://annuaire.univmed.fr/showuser.php?uid=lalot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090414/39265d41/attachment-0001.html 

From brennan at columbia.edu  Tue Apr 14 11:23:12 2009
From: brennan at columbia.edu (Joseph Brennan)
Date: Tue, 14 Apr 2009 11:23:12 -0400
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <1617f8010904140753r3d1842b6hc4b734f3ca9a2b1f@mail.gmail.com>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>	
	<906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>
	<1617f8010904140753r3d1842b6hc4b734f3ca9a2b1f@mail.gmail.com>
Message-ID: <4B14CFE575B582882533278D@sodor.cc.columbia.edu>


Strange to see so many logins spread over a short time.  They seem to
be in pairs, which is the way some clients start up.  I wonder if the
client thinks the connection has dropped, and so it starts new sessions.
I realize the server's netstat shows them as still connected.

It might be interesting to log sessions and see what's going on.  Or
to strace live processes.  And of course ask the user what it looks
like from his/her end.

Joseph Brennan
Lead Email Systems Engineer
Columbia University Information Technology





From dom.lalot at gmail.com  Tue Apr 14 11:35:37 2009
From: dom.lalot at gmail.com (LALOT Dominique)
Date: Tue, 14 Apr 2009 17:35:37 +0200
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <4B14CFE575B582882533278D@sodor.cc.columbia.edu>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
	<906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>
	<1617f8010904140753r3d1842b6hc4b734f3ca9a2b1f@mail.gmail.com>
	<4B14CFE575B582882533278D@sodor.cc.columbia.edu>
Message-ID: <1617f8010904140835r2d84b970if958661507cab322@mail.gmail.com>

2009/4/14 Joseph Brennan <brennan at columbia.edu>

>
> Strange to see so many logins spread over a short time.  They seem to
> be in pairs, which is the way some clients start up.  I wonder if the
> client thinks the connection has dropped, and so it starts new sessions.
> I realize the server's netstat shows them as still connected.
>
> It might be interesting to log sessions and see what's going on.  Or
> to strace live processes.  And of course ask the user what it looks
> like from his/her end.


That's what I will do. But I'm  trying to find configuration options to get
the server stronger faced to bad behaved clients. I've seen once entourage
on macosx ignoring 5xx code from our smtp server, and trying to upload a
50Mo file every minute.
I don't know what will be this one.

Dom


>
> Joseph Brennan
> Lead Email Systems Engineer
> Columbia University Information Technology
>
>
>
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>



-- 
Dominique LALOT
Ing?nieur Syst?mes et R?seaux
http://annuaire.univmed.fr/showuser.php?uid=lalot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090414/473f4b82/attachment.html 

From brennan at columbia.edu  Tue Apr 14 11:53:06 2009
From: brennan at columbia.edu (Joseph Brennan)
Date: Tue, 14 Apr 2009 11:53:06 -0400
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <1617f8010904140835r2d84b970if958661507cab322@mail.gmail.com>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>	
	<906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>	
	<1617f8010904140753r3d1842b6hc4b734f3ca9a2b1f@mail.gmail.com>	
	<4B14CFE575B582882533278D@sodor.cc.columbia.edu>
	<1617f8010904140835r2d84b970if958661507cab322@mail.gmail.com>
Message-ID: <5799839CF519CE10307E99F1@sodor.cc.columbia.edu>


LALOT Dominique <dom.lalot at gmail.com> wrote:

> . I've seen once entourage on macosx ignoring 5xx code from our smtp
> server, and trying to upload a 50Mo file every minute.


Outlook will try every second, under some conditions!

Funny, I was thinking Outlook Express for this imap problem.  I've seen
it start a new imap login to see whether there's new mail in the inbox
it already has open (this is horrible in U Wash imap, where the new
session kills the older one).  If these were evenly timed, like every
5 minutes, I would have said Outlook Express.  But these are at irregular
intervals, so I think it's something else.

Joseph Brennan
Lead Email Systems Engineer
Columbia University Information Technology



From jwidera at oakton.edu  Tue Apr 14 16:14:39 2009
From: jwidera at oakton.edu (John Widera)
Date: Tue, 14 Apr 2009 15:14:39 -0500 (CDT)
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
Message-ID: <1605.10.70.3.113.1239740079.squirrel@borg>

Hi,

Last week we posted to the list for info about running Cyrus on 64-bit
RHEL.  But we also needed to ask a more direct question about caching,
though, and didn't.  So we are now...

That q. is, does Cyrus actually benefit from large amounts of memory in an
environment with just a few thousand users ( <1000 simultaneous sessions)
and, say, up to 750GB of mail spool?

Our plan is to throw 12-16GB at it, with the purpose of vastly increasing
the FS buffer cache (and decreasing I/O).  Or, will that just be a waste
of RAM?

Some indications are that, yes, it does improve performance notably:

http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernels-cache-vs-inode-memory/

Anyone have any specific thoughts?  Is there any other benefit we might
see from large memory allocation in 64-bit architecture?

Thanks,

John Widera
Oakton Community College
Des Plaines and Skokie, IL


From dom.lalot at gmail.com  Wed Apr 15 03:06:27 2009
From: dom.lalot at gmail.com (LALOT Dominique)
Date: Wed, 15 Apr 2009 09:06:27 +0200
Subject: limit tcp sessions opened by an IMAP client
In-Reply-To: <5799839CF519CE10307E99F1@sodor.cc.columbia.edu>
References: <1617f8010904140132x384ab71el44beae958919a1a5@mail.gmail.com>
	<906D17EA7F39F3C40051F8ED@sodor.cc.columbia.edu>
	<1617f8010904140753r3d1842b6hc4b734f3ca9a2b1f@mail.gmail.com>
	<4B14CFE575B582882533278D@sodor.cc.columbia.edu>
	<1617f8010904140835r2d84b970if958661507cab322@mail.gmail.com>
	<5799839CF519CE10307E99F1@sodor.cc.columbia.edu>
Message-ID: <1617f8010904150006h7955c721we0666f71fb832d2c@mail.gmail.com>

Outlook 2007 has won the price..
It's a pity there is no options like, for a given address, no more than 5
simultaneous connexions

Dom

2009/4/14 Joseph Brennan <brennan at columbia.edu>

>
> LALOT Dominique <dom.lalot at gmail.com> wrote:
>
> > . I've seen once entourage on macosx ignoring 5xx code from our smtp
> > server, and trying to upload a 50Mo file every minute.
>
>
> Outlook will try every second, under some conditions!
>
> Funny, I was thinking Outlook Express for this imap problem.  I've seen
> it start a new imap login to see whether there's new mail in the inbox
> it already has open (this is horrible in U Wash imap, where the new
> session kills the older one).  If these were evenly timed, like every
> 5 minutes, I would have said Outlook Express.  But these are at irregular
> intervals, so I think it's something else.
>
> Joseph Brennan
> Lead Email Systems Engineer
> Columbia University Information Technology
>
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>



-- 
Dominique LALOT
Ing?nieur Syst?mes et R?seaux
http://annuaire.univmed.fr/showuser.php?uid=lalot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090415/0e287ba4/attachment.html 

From mathieu.kretchner at sophia.inria.fr  Wed Apr 15 08:56:32 2009
From: mathieu.kretchner at sophia.inria.fr (Mathieu Kretchner)
Date: Wed, 15 Apr 2009 14:56:32 +0200
Subject: restore seen file
In-Reply-To: <49E45716.8080002@sophia.inria.fr>
References: <49E45716.8080002@sophia.inria.fr>
Message-ID: <49E5D980.2090602@sophia.inria.fr>


No answer !

So could I say that there is no possibility to restore an old seen file
for an user ?

thanks for your help

Mathieu Kretchner a ?crit :
> Hello,
> 
> Is it possible to restore seen file even if the mailbox has changed ?
> 
> If yes how ?
> 
> thanks
> 
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mathieu_kretchner.vcf
Type: text/x-vcard
Size: 258 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090415/ab329dc2/attachment.vcf 

From Eric.Luyten at vub.ac.be  Wed Apr 15 09:02:05 2009
From: Eric.Luyten at vub.ac.be (Eric Luyten)
Date: Wed, 15 Apr 2009 15:02:05 +0200 (CEST)
Subject: restore seen file
In-Reply-To: <49E5D980.2090602@sophia.inria.fr>
References: <49E45716.8080002@sophia.inria.fr>
	<49E5D980.2090602@sophia.inria.fr>
Message-ID: <59993.134.184.15.103.1239800525.squirrel@nuts.vub.ac.be>

On Wed, April 15, 2009 2:56 pm, Mathieu Kretchner wrote:
>

> No answer !
>
>
> So could I say that there is no possibility to restore an old seen file
> for an user ?


Hmmmmm... I'd be inclined to think you might get away with it if and when
you have also restored the original cyrus.header and/or cyrus.index files.


Eric.



> Mathieu Kretchner a ?crit :
>
>> Hello,
>>
>>
>> Is it possible to restore seen file even if the mailbox has changed ?
>>
>>
>> If yes how ?
>>
>>
>> thanks
>>
>> ----
>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html



From champ at umbc.edu  Wed Apr 15 09:11:25 2009
From: champ at umbc.edu (Tim Champ)
Date: Wed, 15 Apr 2009 09:11:25 -0400
Subject: Question about upgrading and mismatched backends
Message-ID: <49E5DCFD.8070907@umbc.edu>

Hello all.  My first time to post, I only recently joined the list.  I'm 
digging in deeply on an inherited cyrus install, and looking to upgrade.

My goal is to put a new backend server in place for our setup.  Our 
basic setup is 3 front-ends, 4 back-ends and a mupdate server.

I'm looking to add a back-end for multiple reasons, but I would like to 
set it up with cyrus 2.3.14 to kill two birds with one stone.  I realize 
some additional options for the config file now exist, with delayed 
delete for folders (which we're looking forward to) and others.

I've googled and read through quite a few list archives, but I haven't 
seen anything to make me think this won't work (or will work).  Anyone 
have any experience, opinions, etc?  The list seems to have quite a few 
well-informed people on it, so I'm hoping you can help!

Thanks in advance!
Tim Champ
Sr. Unix Systems Specialist
UMBC

From dave64 at andrew.cmu.edu  Wed Apr 15 09:30:37 2009
From: dave64 at andrew.cmu.edu (Dave McMurtrie)
Date: Wed, 15 Apr 2009 09:30:37 -0400
Subject: Question about upgrading and mismatched backends
In-Reply-To: <49E5DCFD.8070907@umbc.edu>
References: <49E5DCFD.8070907@umbc.edu>
Message-ID: <49E5E17D.1060207@andrew.cmu.edu>

Tim Champ wrote:
> Hello all.  My first time to post, I only recently joined the list.  I'm 
> digging in deeply on an inherited cyrus install, and looking to upgrade.
> 
> My goal is to put a new backend server in place for our setup.  Our 
> basic setup is 3 front-ends, 4 back-ends and a mupdate server.
> 
> I'm looking to add a back-end for multiple reasons, but I would like to 
> set it up with cyrus 2.3.14 to kill two birds with one stone.  I realize 
> some additional options for the config file now exist, with delayed 
> delete for folders (which we're looking forward to) and others.

Hi Tim,

You don't mention what version the other members of your murder are running.

There was a change to the sieve protocol wrt how it doles out a
capability response.  2.3.14 should work with old or new sieve clients,
so that shouldn't be a problem.

I'm not aware of any lmtp changes that would cause breakage between
versions.

I'm not aware of any mupdate changes that would cause breakage between
versions.

Ken mentioned to me that there may have been some changes to how XFER
works (moving mailboxes between backend servers), but he couldn't
remember details off the top of his head.  Even if he did, we'd need to
know which versions your other nodes are running.

Thanks,

Dave
-- 
Dave McMurtrie, SPE
Email Systems Team Leader
Carnegie Mellon University,
Computing Services

From champ at umbc.edu  Wed Apr 15 09:43:26 2009
From: champ at umbc.edu (Tim Champ)
Date: Wed, 15 Apr 2009 09:43:26 -0400
Subject: Question about upgrading and mismatched backends
In-Reply-To: <49E5E17D.1060207@andrew.cmu.edu>
References: <49E5DCFD.8070907@umbc.edu> <49E5E17D.1060207@andrew.cmu.edu>
Message-ID: <49E5E47E.3080203@umbc.edu>

Dave McMurtrie wrote:
> Tim Champ wrote:
>   
>> Hello all.  My first time to post, I only recently joined the list.  I'm 
>> digging in deeply on an inherited cyrus install, and looking to upgrade.
>>
>> My goal is to put a new backend server in place for our setup.  Our 
>> basic setup is 3 front-ends, 4 back-ends and a mupdate server.
>>
>> I'm looking to add a back-end for multiple reasons, but I would like to 
>> set it up with cyrus 2.3.14 to kill two birds with one stone.  I realize 
>> some additional options for the config file now exist, with delayed 
>> delete for folders (which we're looking forward to) and others.
>>     
>
> Hi Tim,
>
> You don't mention what version the other members of your murder are running.
>
> There was a change to the sieve protocol wrt how it doles out a
> capability response.  2.3.14 should work with old or new sieve clients,
> so that shouldn't be a problem.
>
> I'm not aware of any lmtp changes that would cause breakage between
> versions.
>
> I'm not aware of any mupdate changes that would cause breakage between
> versions.
>
> Ken mentioned to me that there may have been some changes to how XFER
> works (moving mailboxes between backend servers), but he couldn't
> remember details off the top of his head.  Even if he did, we'd need to
> know which versions your other nodes are running.
>   

Completely thought it, and didn't type it!  Sorry!  The current version 
of all parts is 2.3.8 with some patches from fastmail applied.  Also, we 
have the patch for the sieve xfer bug that existed in that version.  
I've read through each of the fastmail patches, and most appear 
integrated into the current releases of Cyrus.  The only ones of note 
that I was unsure of are named:

cyrus-md5uuid-2.3.8
imapd-disable-referrals
unified-murder-xfer-fix

I don't know how much this helps anyone to have the names, but I wanted 
to throw them out there.  The md5uuid one did a lot of changes in order 
to deal with these, but I don't see (yet) how they were integrated into 
the current releases.  I also don't see this patch currently from 
fastmail.  I did see that SHA1 is now the preferred way of deal with 
this, but I wanted to confirm the backward compatibility if possible.

That "xfer" fix above did only minimal changes, but I didn't see them 
directly in the code.  I was unsure if this was somehow re-worked to 
make that patch un-needed.  It only appears to have done changes to 
mboxlist.c to "allow localcreate to go through if we're creating the 
mailbox locally and it currently exists on a remote server to facilitate 
mailbox moves" as the comment says in the patch.

Hope that info helps, thanks for the fast response and help.  If more 
info is needed, please let me know.
 
Tim



From brong at fastmail.fm  Wed Apr 15 10:02:57 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Thu, 16 Apr 2009 00:02:57 +1000
Subject: Question about upgrading and mismatched backends
In-Reply-To: <49E5E47E.3080203@umbc.edu>
References: <49E5DCFD.8070907@umbc.edu> <49E5E17D.1060207@andrew.cmu.edu>
	<49E5E47E.3080203@umbc.edu>
Message-ID: <20090415140257.GB25469@brong.net>

On Wed, Apr 15, 2009 at 09:43:26AM -0400, Tim Champ wrote:
> Dave McMurtrie wrote:
> > Tim Champ wrote:
> >   
> >> Hello all.  My first time to post, I only recently joined the list.  I'm 
> >> digging in deeply on an inherited cyrus install, and looking to upgrade.
> >>
> >> My goal is to put a new backend server in place for our setup.  Our 
> >> basic setup is 3 front-ends, 4 back-ends and a mupdate server.
> >>
> >> I'm looking to add a back-end for multiple reasons, but I would like to 
> >> set it up with cyrus 2.3.14 to kill two birds with one stone.  I realize 
> >> some additional options for the config file now exist, with delayed 
> >> delete for folders (which we're looking forward to) and others.
> >>     
> >
> > Hi Tim,
> >
> > You don't mention what version the other members of your murder are running.
> >
> > There was a change to the sieve protocol wrt how it doles out a
> > capability response.  2.3.14 should work with old or new sieve clients,
> > so that shouldn't be a problem.
> >
> > I'm not aware of any lmtp changes that would cause breakage between
> > versions.
> >
> > I'm not aware of any mupdate changes that would cause breakage between
> > versions.
> >
> > Ken mentioned to me that there may have been some changes to how XFER
> > works (moving mailboxes between backend servers), but he couldn't
> > remember details off the top of his head.  Even if he did, we'd need to
> > know which versions your other nodes are running.
> >   
> 
> Completely thought it, and didn't type it!  Sorry!  The current version 
> of all parts is 2.3.8 with some patches from fastmail applied.  Also, we 
> have the patch for the sieve xfer bug that existed in that version.  
> I've read through each of the fastmail patches, and most appear 
> integrated into the current releases of Cyrus.  The only ones of note 
> that I was unsure of are named:

Sorry I didn't get back to you today - I completely forgot about it!

Yes - most of them are upstream.  The naming has changed now that
they are absorbed into git - but the commit messages are pretty close
to what I had as the patch headers.  What's on the webpage now is
what we applied against 2.3.14 - a couple of them have gone into
CVS, and I don't (yet) have a nice way to give a patch list against
2.3.14 that's also 100% up-to-date.  Maybe create a divergent branch
and have a "CVS bits" patch at the top of the list always....

> cyrus-md5uuid-2.3.8

You don't want that any more - it's been replaced by sha1 GUIDs.

> imapd-disable-referrals

Not us.

> unified-murder-xfer-fix

Not us.

> I don't know how much this helps anyone to have the names, but I wanted 
> to throw them out there.  The md5uuid one did a lot of changes in order 
> to deal with these, but I don't see (yet) how they were integrated into 
> the current releases.  I also don't see this patch currently from 
> fastmail.  I did see that SHA1 is now the preferred way of deal with 
> this, but I wanted to confirm the backward compatibility if possible.

It will only sha1 new GUIDs.  The old md5 UUID will be placed at the
start of the space, so it will look like 02xxxxxxxxxxxxxxxxxx0000000000
(excuse my inaccurate counting, I don't care that much ;)  Something
like that.  They all start with 02 and end with a big block of zeros
anyway.

Bron.

From cyrus at puri.jet2web.at  Wed Apr 15 10:18:46 2009
From: cyrus at puri.jet2web.at (Klemens Puritscher)
Date: Wed, 15 Apr 2009 16:18:46 +0200 (CEST)
Subject: Message contains NUL characters - howto dump?
Message-ID: <mtranet.20090415161846.1974787582@telekom.at>

Hello,

I have a problem with one of our customers.
When he forwards an email with the thunderbird email client (windows version), the lmtp-daemon on my cyrus-imapd (v2.3.13) rejects those emails with the error "554 5.6.0 Message contains NUL characters".
...ok, that's clear, there are "NUL" characters in the email.

But I would show my customer, where the "NUL" character is.

For tests, I generate a testmail, with "echo -e "From:<sender at example.com>\nTo:<recipient at example.com>\nSubject: test\n\ntest\0000test\n.\n" > mail_with_NUL.txt

Now I dump the lmtp-session on the cyrus-imapd host with:
tcpdump -vv -XX -s 65535 -n -i eth1 "port lmtp

and I see the "NUL" character:
...
        0x0230:  7065 6564 2e61 740d 0a0d 0a74 6573 7400  peed.at....test.
        0x0240:  7465 7374 0d0a 2e0d 0a                   test.....
...
65 = e
73 = s
74 = t
00 = NUL

...ok, fine, I can find the "NUL" character.

But when I dump the lmtp-session with the customer email (which get's the error "554 5.6.0 Message contains NUL characters"), I cannot find this "NUL" character.

Can someone tell me, what I did wrong?

Thanks in advance.

Klemens


From blake at ispn.net  Wed Apr 15 11:27:27 2009
From: blake at ispn.net (Blake Hudson)
Date: Wed, 15 Apr 2009 10:27:27 -0500
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <1605.10.70.3.113.1239740079.squirrel@borg>
References: <1605.10.70.3.113.1239740079.squirrel@borg>
Message-ID: <49E5FCDF.7030801@ispn.net>

We're able to utilize ~ 4GB of cache on a 32bit  installation of 
Cyrus/Postfix (we're using ext3 - dunno if it has a different cache 
limitation compared to reiser). This installation serves ~15k mailboxes 
and has between 250 and 500 active POP/IMAP sessions at any given time.

Unfortunately, we do not currently make use of the full 24GB RAM in 
these machines. Perhaps with a 64 bit installation would resolve that... 
anyway, here are the mem stats.

----
             total       used       free     shared    buffers     cached
Mem:      24906720    9973956   14932764          0     298412    4513164
-/+ buffers/cache:    5162380   19744340
Swap:      1052248          0    1052248
----

If we are able to make use of 10GB (half of which is cache) on a 
similarly loaded machine, you should be able use just as much (if not 
more on a 64bit installation).

--Blake

-------- Original Message  --------
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
From: John Widera <jwidera at oakton.edu>
To: info-cyrus at lists.andrew.cmu.edu
Cc: data center <datacenter at oakton.edu>
Date: Tuesday, April 14, 2009 3:14:39 PM
> Hi,
>
> Last week we posted to the list for info about running Cyrus on 64-bit
> RHEL.  But we also needed to ask a more direct question about caching,
> though, and didn't.  So we are now...
>
> That q. is, does Cyrus actually benefit from large amounts of memory in an
> environment with just a few thousand users ( <1000 simultaneous sessions)
> and, say, up to 750GB of mail spool?
>
> Our plan is to throw 12-16GB at it, with the purpose of vastly increasing
> the FS buffer cache (and decreasing I/O).  Or, will that just be a waste
> of RAM?
>
> Some indications are that, yes, it does improve performance notably:
>
> http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernels-cache-vs-inode-memory/
>
> Anyone have any specific thoughts?  Is there any other benefit we might
> see from large memory allocation in 64-bit architecture?
>
> Thanks,
>
> John Widera
> Oakton Community College
> Des Plaines and Skokie, IL
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>   


From ml at awinkelmann.de  Wed Apr 15 15:27:20 2009
From: ml at awinkelmann.de (Andreas Winkelmann)
Date: Wed, 15 Apr 2009 21:27:20 +0200
Subject: restore seen file
In-Reply-To: <49E5D980.2090602@sophia.inria.fr>
References: <49E45716.8080002@sophia.inria.fr>
	<49E5D980.2090602@sophia.inria.fr>
Message-ID: <200904152127.21028.ml@awinkelmann.de>

Am Mittwoch 15 April 2009 14:56:32 schrieb Mathieu Kretchner:

> No answer !
>
> So could I say that there is no possibility to restore an old seen file
> for an user ?

What means "old"? What has changed?

Each Mailbox has a unique number which is in the seen-File and each Message 
has a unique number which is also in the seen-File.

Sure you can restore an old seen-File, but the work you have to do while or 
after doing this depends on the changes you have made to the Mailbox and the 
Messages.

--
Andreas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090415/3a4f5ef6/attachment.html 

From morgan at orst.edu  Wed Apr 15 15:28:11 2009
From: morgan at orst.edu (Andrew Morgan)
Date: Wed, 15 Apr 2009 12:28:11 -0700 (PDT)
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <1605.10.70.3.113.1239740079.squirrel@borg>
References: <1605.10.70.3.113.1239740079.squirrel@borg>
Message-ID: <alpine.DEB.2.00.0904151220180.9160@shell.onid.oregonstate.edu>

On Tue, 14 Apr 2009, John Widera wrote:

> Hi,
>
> Last week we posted to the list for info about running Cyrus on 64-bit
> RHEL.  But we also needed to ask a more direct question about caching,
> though, and didn't.  So we are now...
>
> That q. is, does Cyrus actually benefit from large amounts of memory in an
> environment with just a few thousand users ( <1000 simultaneous sessions)
> and, say, up to 750GB of mail spool?
>
> Our plan is to throw 12-16GB at it, with the purpose of vastly increasing
> the FS buffer cache (and decreasing I/O).  Or, will that just be a waste
> of RAM?
>
> Some indications are that, yes, it does improve performance notably:
>
> http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernels-cache-vs-inode-memory/
>
> Anyone have any specific thoughts?  Is there any other benefit we might
> see from large memory allocation in 64-bit architecture?

Funny you mention this, because I was just looking at the load average 
charts we maintain for our cyrus servers.  Over the christmas break, I 
installed a 64-bit kernel (the Debian amd64 kernel) on our backends, which 
each have 4GB of memory.  This is a 32-bit architecture install running a 
64-bit kernel.  The load average on each box is HALF what it was before 
the upgrade.

I know that it is hard for a 32-bit kernel to cache a large number of 
small files, which is frequently the case with cyrus.  It has something to 
do with the size of the lookup table growing too large for "lowmem" to 
contain.  So even if you have lots of memory in a server, it may not be 
able to use it all for file caching if you are trying to cache a lot of 
small files.  The 64-bit kernel does not have a split lowmem/highmem 
architecture, so it can cache a lot more files.

Here is the output of "free" on one of our cyrus backends right now:

              total       used       free     shared    buffers     cached
Mem:       4051576    4024116      27460          0     414200    2703212
-/+ buffers/cache:     906704    3144872
Swap:      2000052        660    1999392

You can see the performance difference on our Cacti graphs at:

   http://admin.onid.oregonstate.edu/cacti/graph.php?rra_id=all&local_graph_id=31

I would definately recommend installing a 64-bit kernel.

 	Andy

From robm at fastmail.fm  Wed Apr 15 20:58:15 2009
From: robm at fastmail.fm (Rob Mueller)
Date: Thu, 16 Apr 2009 10:58:15 +1000
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <1605.10.70.3.113.1239740079.squirrel@borg>
References: <1605.10.70.3.113.1239740079.squirrel@borg>
Message-ID: <ABCCC6006A224F01B7EF299540CB96C1@Atticus>


> Our plan is to throw 12-16GB at it, with the purpose of vastly increasing
> the FS buffer cache (and decreasing I/O).  Or, will that just be a waste
> of RAM?
>
> Some indications are that, yes, it does improve performance notably:
>
> http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernels-cache-vs-inode-memory/
>
> Anyone have any specific thoughts?  Is there any other benefit we might
> see from large memory allocation in 64-bit architecture?

Given that I wrote that blog post, I can only tell you that in our 
environment, 64-bit kernels made a big difference.

Your environment sounds pretty small though, so you might want to see what 
your current load actually is. If it's already < 1, then adding extra RAM + 
a 64-bit kernel isn't going to buy you that much more.

On the other hand, RAM is pretty cheap, and it might be worth doing it and 
going to 64-bit so you don't have to deal with it one day in the future...

Rob


From Hagedorn at uni-koeln.de  Thu Apr 16 04:21:08 2009
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Thu, 16 Apr 2009 10:21:08 +0200
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <ABCCC6006A224F01B7EF299540CB96C1@Atticus>
References: <1605.10.70.3.113.1239740079.squirrel@borg>
	<ABCCC6006A224F01B7EF299540CB96C1@Atticus>
Message-ID: <0061DD93886AC210F085086F@tyrion.rrz.uni-koeln.de>

--On 16. April 2009 10:58:15 +1000 Rob Mueller <robm at fastmail.fm> wrote:

>> http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernel
>> s-cache-vs-inode-memory/
>>
>> Anyone have any specific thoughts?  Is there any other benefit we might
>> see from large memory allocation in 64-bit architecture?
>
> Given that I wrote that blog post, I can only tell you that in our
> environment, 64-bit kernels made a big difference.

I wonder if ext3 behaves differently, Red Hat's 32-bit behaves differently, 
or if something altogether different is going on. We are currently running 
RHEL 3 in 32-bit mode, our servers have 16 GB, and most of it is used for 
caching:

# free
             total       used       free     shared    buffers     cached
Mem:      16214344   16197612      16732          0      86944   13415172
-/+ buffers/cache:    2695496   13518848
Swap:      4192944       8436    4184508

So it would seem that a 64-bit kernel wouldn't improve on that, right? Or 
is that a difference between 2.4 and 2.6?
-- 
     .:.Sebastian Hagedorn - RZKR-R1 (Geb?ude 52), Zimmer 18.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
.:.Universit?t zu K?ln / Cologne University - ? +49-221-478-5587.:.
                   .:.:.:.Skype: shagedorn.:.:.:.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090416/a8b7a3f3/attachment.bin 

From mathieu.kretchner at sophia.inria.fr  Thu Apr 16 07:37:22 2009
From: mathieu.kretchner at sophia.inria.fr (Mathieu Kretchner)
Date: Thu, 16 Apr 2009 13:37:22 +0200
Subject: restore seen file
In-Reply-To: <200904152127.21028.ml@awinkelmann.de>
References: <49E45716.8080002@sophia.inria.fr>	<49E5D980.2090602@sophia.inria.fr>
	<200904152127.21028.ml@awinkelmann.de>
Message-ID: <49E71872.7080903@sophia.inria.fr>

So thank you all and specially John Widera for your help,

Just a simple replace of the new seen file by an old one allowed me to
restore the seen flag on thousand mails.

Andreas Winkelmann a ?crit :
> Am Mittwoch 15 April 2009 14:56:32 schrieb Mathieu Kretchner:
> 
> 
>> No answer !
>>
>> So could I say that there is no possibility to restore an old seen file
>> for an user ?
> 
> 
> What means "old"? What has changed?
> 
> 
> Each Mailbox has a unique number which is in the seen-File and each
> Message has a unique number which is also in the seen-File.
> 
> 
> Sure you can restore an old seen-File, but the work you have to do while
> or after doing this depends on the changes you have made to the Mailbox
> and the Messages.
> 
> 
> --
> Andreas
> 
> 
> 
> ------------------------------------------------------------------------
> 
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mathieu_kretchner.vcf
Type: text/x-vcard
Size: 258 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090416/95d712d0/attachment.vcf 

From mathieu.kretchner at sophia.inria.fr  Thu Apr 16 08:47:41 2009
From: mathieu.kretchner at sophia.inria.fr (Mathieu Kretchner)
Date: Thu, 16 Apr 2009 14:47:41 +0200
Subject: restore seen file
In-Reply-To: <49E71872.7080903@sophia.inria.fr>
References: <49E45716.8080002@sophia.inria.fr>	<49E5D980.2090602@sophia.inria.fr>	<200904152127.21028.ml@awinkelmann.de>
	<49E71872.7080903@sophia.inria.fr>
Message-ID: <49E728ED.1020203@sophia.inria.fr>

Here you could find the discussion between John and I about the topic :


> > John Widera a ?crit :
>> >> Glad it worked.
>> >>
>>> >>> yes it's working, thanks a lot for the time you've spent teach
me how
>>> >>> to
>>> >>> do.
>>> >>>
>>>> >>>> We also have a webmail app running, whenever people call to
complain
>>>> >>>> that
>>>> >>>> something like that is wrong in T-bird I ask them to check in
webmail.
>>>> >>>> If
>>>> >>>> it looks OK in webmail, it's a T-bird problem.  99.9% of the
time, it
>>>> >>>> is
>>>> >>>> a
>>>> >>>> T-bird problem.
>>> >>> I've noticed this before
>>> >>>
>>> >>> John Widera a ?crit :
>>>> >>>> This is a **THUNDERBIRD** problem, not an IMAP problem.
Thunderbird
>>>> >>>> will
>>>> >>>> give you all sorts of trouble with indexing.  We use it here, and
>>>> >>>> experience lots of corruped file issues with it (the corrupted
files
>>>> >>>> are
>>>> >>>> the Thunderbird files, not the Cyrus files).
>>>> >>>>
>>>> >>>> TIP:
>>>> >>>> Find the Thunderbird directory for the user (ours are in
$HOME) and
>>>> >>>> (if
>>>> >>>> IMAP) rename the IMAPMAIL folder inside it to IMAPMAIL.old.  Close
>>>> >>>> Thunderbird first or you can't rename.  Then relaunch
Thunderbird and
>>>> >>>> log
>>>> >>>> in.  Let it rebuild the index.  THEN check to see if you can
see all
>>>> >>>> your
>>>> >>>> mail.  Bet you can.
>>>> >>>>
>>>> >>>> We also have a webmail app running, whenever people call to
complain
>>>> >>>> that
>>>> >>>> something like that is wrong in T-bird I ask them to check in
webmail.
>>>> >>>> If
>>>> >>>> it looks OK in webmail, it's a T-bird problem.  99.9% of the
time, it
>>>> >>>> is
>>>> >>>> a
>>>> >>>> T-bird problem.
>>>> >>>>
>>>> >>>> John
>>>> >>>>
>>>>> >>>>> Yes you've right I've tried with a test user  and It's
working fine.
>>>>> >>>>> But
>>>>> >>>>> I can't understand why I can't see new mail for my other
user (a real
>>>>> >>>>> one). Thunderbird say 190 mails unread but we can see only 1
mail
>>>>> >>>>> unread !
>>>>> >>>>> And new mail seems to be lost (althougth this didn't happen
with my
>>>>> >>>>> test
>>>>> >>>>> user where everything works fine)
>>>>> >>>>>
>>>>> >>>>> But anyway, thanks for your help, and I keep on my
investigation.
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> John Widera a ?crit :
>>>>>> >>>>>> Forget what I said earlier, it was early in the morning
and I wasn't
>>>>>> >>>>>> thinking right.
>>>>>> >>>>>>
>>>>>> >>>>>> But you CAN just copy the old one over the newer one, if
you are OK
>>>>>> >>>>>> with
>>>>>> >>>>>> losing your seen state of any msgs delivered since the last
>>>>>> >>>>>> timestamp
>>>>>> >>>>>> of
>>>>>> >>>>>> the old seen file.
>>>>>> >>>>>>
>>>>>> >>>>>> If you wanted to spend the time you could probably dump
both old and
>>>>>> >>>>>> new
>>>>>> >>>>>> seen files out to flat file, then merge the two by hand,
and dump
>>>>>> >>>>>> them
>>>>>> >>>>>> back to the DB format of your liking (I assume you aren
not using
>>>>>> >>>>>> flat
>>>>>> >>>>>> file now).  Use cvt_cyrusdb for this.
>>>>>> >>>>>>
>>>>>> >>>>>> You could even script this in PERL pretty easily if you
wanted to.
>>>>>> >>>>>> Then
>>>>>> >>>>>> you'd maintain the seen state of new files too.
>>>>>> >>>>>>
>>>>>> >>>>>> John
>>>>>> >>>>>>
>>>>>>> >>>>>>> Ok thank you for the help, I'll try this test
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> John Widera a ?crit :
>>>>>>>> >>>>>>>> I've never done exactly that but I have to believe you
can.  Have
>>>>>>>> >>>>>>>> a
>>>>>>>> >>>>>>>> test
>>>>>>>> >>>>>>>> user?  Try it on the test user.  Copy the seen file to
/tmp then
>>>>>>>> >>>>>>>> send
>>>>>>>> >>>>>>>> the
>>>>>>>> >>>>>>>> user a couple dozen msgs.  Read some, leave the others
unread.
>>>>>>>> >>>>>>>> Then
>>>>>>>> >>>>>>>> copy
>>>>>>>> >>>>>>>> the old seen file back over the current one and run a
reconstruct
>>>>>>>> >>>>>>>> as
>>>>>>>> >>>>>>>> Cyrus
>>>>>>>> >>>>>>>> user on the mailbox.  It's a super easy test.
>>>>>>>> >>>>>>>> (/usr/cyrus/bin/reconstruct
>>>>>>>> >>>>>>>> -r user.<nameofyourtestuser> or whatever your path).
If it
>>>>>>>> >>>>>>>> doesn't
>>>>>>>> >>>>>>>> work,
>>>>>>>> >>>>>>>> well, just delete the test user!
>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>>> No answer !
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> So could I say that there is no possibility to
restore an old
>>>>>>>>> >>>>>>>>> seen
>>>>>>>>> >>>>>>>>> file
>>>>>>>>> >>>>>>>>> for an user ?
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> thanks for your help
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> Mathieu Kretchner a ?crit :
>>>>>>>>>> >>>>>>>>>> Hello,
>>>>>>>>>> >>>>>>>>>>
>>>>>>>>>> >>>>>>>>>> Is it possible to restore seen file even if the
mailbox has
>>>>>>>>>> >>>>>>>>>> changed
>>>>>>>>>> >>>>>>>>>> ?
>>>>>>>>>> >>>>>>>>>>
>>>>>>>>>> >>>>>>>>>> If yes how ?
>>>>>>>>>> >>>>>>>>>>
>>>>>>>>>> >>>>>>>>>> thanks
>>>>>>>>>> >>>>>>>>>>
>>>>>>>>>> >>>>>>>>>> ----
>>>>>>>>>> >>>>>>>>>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>>>>>>>>>> >>>>>>>>>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>>>>>>>>>> >>>>>>>>>> List Archives/Info:
>>>>>>>>>> >>>>>>>>>> http://asg.web.cmu.edu/cyrus/mailing-list.html
>>>>>>>>> >>>>>>>>> ----
>>>>>>>>> >>>>>>>>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>>>>>>>>> >>>>>>>>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>>>>>>>>> >>>>>>>>> List Archives/Info:
>>>>>>>>> >>>>>>>>> http://asg.web.cmu.edu/cyrus/mailing-list.html
>>>> >>>>
>> >>
>> >>
> >
















Mathieu Kretchner a ?crit :
> So thank you all and specially John Widera for your help,
> 
> Just a simple replace of the new seen file by an old one allowed me to
> restore the seen flag on thousand mails.
> 
> Andreas Winkelmann a ?crit :
>> Am Mittwoch 15 April 2009 14:56:32 schrieb Mathieu Kretchner:
>>
>>
>>> No answer !
>>>
>>> So could I say that there is no possibility to restore an old seen file
>>> for an user ?
>>
>> What means "old"? What has changed?
>>
>>
>> Each Mailbox has a unique number which is in the seen-File and each
>> Message has a unique number which is also in the seen-File.
>>
>>
>> Sure you can restore an old seen-File, but the work you have to do while
>> or after doing this depends on the changes you have made to the Mailbox
>> and the Messages.
>>
>>
>> --
>> Andreas
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> ----
>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>> ----
>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mathieu_kretchner.vcf
Type: text/x-vcard
Size: 258 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090416/6a7190ae/attachment-0001.vcf 

From cyrus at puri.jet2web.at  Thu Apr 16 11:07:16 2009
From: cyrus at puri.jet2web.at (Klemens Puritscher)
Date: Thu, 16 Apr 2009 17:07:16 +0200 (CEST)
Subject: =?utf-8?Q?WG=3A_Message_contains_NUL_characters_-_howto_dump=3F?=
Message-ID: <mtranet.20090416170716.1317120424@telekom.at>

Please, can somebody help me?
Maybe, it's a problem in the cyrus lmtp-daemon? (I don't think so...)

thx

Klemens

----- Weitergeleitete Nachricht -----
Datum: Wed, 15 Apr 2009 16:18:46 +0200 (CEST)
Von: Klemens Puritscher <cyrus at puri.jet2web.at>
An: info-cyrus at lists.andrew.cmu.edu
Betreff: *****POSSIBLE SPAM***** Message contains NUL characters - howto dump?

Hello,

I have a problem with one of our customers.
When he forwards an email with the thunderbird email client (windows version), the lmtp-daemon on my cyrus-imapd (v2.3.13) rejects those emails with the error "554 5.6.0 Message contains NUL characters".
...ok, that's clear, there are "NUL" characters in the email.

But I would show my customer, where the "NUL" character is.

For tests, I generate a testmail, with "echo -e "From:<sender at example.com>\nTo:<recipient at example.com>\nSubject: test\n\ntest\0000test\n.\n" > mail_with_NUL.txt

Now I dump the lmtp-session on the cyrus-imapd host with:
tcpdump -vv -XX -s 65535 -n -i eth1 "port lmtp

and I see the "NUL" character:
...
        0x0230:  7065 6564 2e61 740d 0a0d 0a74 6573 7400  peed.at....test.
        0x0240:  7465 7374 0d0a 2e0d 0a                   test.....
...
65 = e
73 = s
74 = t
00 = NUL

...ok, fine, I can find the "NUL" character.

But when I dump the lmtp-session with the customer email (which get's the error "554 5.6.0 Message contains NUL characters"), I cannot find this "NUL" character.

Can someone tell me, what I did wrong?

Thanks in advance.

Klemens

----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


From kendrick.hernandez at umbc.edu  Thu Apr 16 11:08:21 2009
From: kendrick.hernandez at umbc.edu (Kendrick Hernandez)
Date: Thu, 16 Apr 2009 11:08:21 -0400
Subject: Question about bulk mailbox transfers between backends
Message-ID: <49E749E5.5060108@umbc.edu>

Hi folks,

We're currently running a Cyrus 2.3.8 murder and are looking to bulk move 
several hundred mailboxes from one backend server to another (running either 
2.3.8 or 2.3.14).

We've done successful tests of single transfers using cyradm's xfer command, but 
I'm wondering if anyone's already written a script for this, before we set out 
to re-invent the wheel?

I've been searching the list archives and I've come across some threads 
suggesting the use of imapsync, but I'm unfamiliar with that utility. Any 
suggestions?


Best regards,
k-

-- 
: Kendrick Hernandez
: UNIX Systems Administrator
: Division of Information Technology
: University of Maryland, Baltimore County

From morgan at orst.edu  Thu Apr 16 12:36:57 2009
From: morgan at orst.edu (Andrew Morgan)
Date: Thu, 16 Apr 2009 09:36:57 -0700 (PDT)
Subject: Question about bulk mailbox transfers between backends
In-Reply-To: <49E749E5.5060108@umbc.edu>
References: <49E749E5.5060108@umbc.edu>
Message-ID: <alpine.DEB.2.00.0904160935400.9160@shell.onid.oregonstate.edu>

On Thu, 16 Apr 2009, Kendrick Hernandez wrote:

> Hi folks,
>
> We're currently running a Cyrus 2.3.8 murder and are looking to bulk move
> several hundred mailboxes from one backend server to another (running either
> 2.3.8 or 2.3.14).
>
> We've done successful tests of single transfers using cyradm's xfer command, but
> I'm wondering if anyone's already written a script for this, before we set out
> to re-invent the wheel?
>
> I've been searching the list archives and I've come across some threads
> suggesting the use of imapsync, but I'm unfamiliar with that utility. Any
> suggestions?

I have some sample scripts you can use at:

   http://oregonstate.edu/~morgan/cyrus/public/

Check out batch-rename-with-backend.pl, plus the various scripts it 
requires.

 	Andy

From morgan at orst.edu  Thu Apr 16 12:52:15 2009
From: morgan at orst.edu (Andrew Morgan)
Date: Thu, 16 Apr 2009 09:52:15 -0700 (PDT)
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <0061DD93886AC210F085086F@tyrion.rrz.uni-koeln.de>
References: <1605.10.70.3.113.1239740079.squirrel@borg>
	<ABCCC6006A224F01B7EF299540CB96C1@Atticus>
	<0061DD93886AC210F085086F@tyrion.rrz.uni-koeln.de>
Message-ID: <alpine.DEB.2.00.0904160949310.9160@shell.onid.oregonstate.edu>

On Thu, 16 Apr 2009, Sebastian Hagedorn wrote:

> --On 16. April 2009 10:58:15 +1000 Rob Mueller <robm at fastmail.fm> wrote:
>
>>> http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernel
>>> s-cache-vs-inode-memory/
>>> 
>>> Anyone have any specific thoughts?  Is there any other benefit we might
>>> see from large memory allocation in 64-bit architecture?
>> 
>> Given that I wrote that blog post, I can only tell you that in our
>> environment, 64-bit kernels made a big difference.
>
> I wonder if ext3 behaves differently, Red Hat's 32-bit behaves differently, 
> or if something altogether different is going on. We are currently running 
> RHEL 3 in 32-bit mode, our servers have 16 GB, and most of it is used for 
> caching:
>
> # free
>            total       used       free     shared    buffers     cached
> Mem:      16214344   16197612      16732          0      86944   13415172
> -/+ buffers/cache:    2695496   13518848
> Swap:      4192944       8436    4184508
>
> So it would seem that a 64-bit kernel wouldn't improve on that, right? Or is 
> that a difference between 2.4 and 2.6?

That's interesting, and not what I expected.  :)  What does "cat 
/proc/meminfo" show?  My 64-bit kernel running cyrus has:

MemTotal:      4051576 kB
MemFree:         31452 kB
Buffers:        478716 kB
Cached:        2471240 kB
SwapCached:          0 kB
Active:        1562596 kB
Inactive:      1768236 kB
SwapTotal:     2000052 kB
SwapFree:      1999396 kB
Dirty:            1484 kB
Writeback:          40 kB
AnonPages:      380888 kB
Mapped:          68340 kB
Slab:           601940 kB
SReclaimable:   500604 kB
SUnreclaim:     101336 kB
PageTables:      41220 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   4025840 kB
Committed_AS:   659524 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    280476 kB
VmallocChunk: 34359457535 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     2048 kB

On an older system running a 32-bit "686" kernel, it shows:

MemTotal:      2076312 kB
MemFree:         73008 kB
Buffers:         61916 kB
Cached:        1512400 kB
SwapCached:          0 kB
Active:         429292 kB
Inactive:      1467944 kB
HighTotal:     1179392 kB
HighFree:         8400 kB
LowTotal:       896920 kB
LowFree:         64608 kB
SwapTotal:     2000052 kB
SwapFree:      1999988 kB
Dirty:             140 kB
Writeback:           0 kB
AnonPages:      322960 kB
Mapped:          24180 kB
Slab:            77336 kB
SReclaimable:    45836 kB
SUnreclaim:      31500 kB
PageTables:       5900 kB
NFS_Unstable:       12 kB
Bounce:              0 kB
CommitLimit:   3038208 kB
Committed_AS:   598964 kB
VmallocTotal:   114680 kB
VmallocUsed:      3936 kB
VmallocChunk:   110188 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     4096 kB


Note the HighTotal, HighFree, LowTotal, and LowFree lines that are not 
present in a 64-bit kernel.

 	Andy

From david.lang at digitalinsight.com  Thu Apr 16 15:15:56 2009
From: david.lang at digitalinsight.com (David Lang)
Date: Thu, 16 Apr 2009 12:15:56 -0700 (PDT)
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <0061DD93886AC210F085086F@tyrion.rrz.uni-koeln.de>
References: <1605.10.70.3.113.1239740079.squirrel@borg><ABCCC6006A224F01B7EF299540CB96C1@Atticus>
	<0061DD93886AC210F085086F@tyrion.rrz.uni-koeln.de>
Message-ID: <alpine.DEB.2.00.0904161211540.20717@qynat-qrfxgbc.qvtvgnyvafvtug.pbz>

On Thu, 16 Apr 2009, Sebastian Hagedorn wrote:

> --On 16. April 2009 10:58:15 +1000 Rob Mueller <robm at fastmail.fm> wrote:
>
>>> http://blog.fastmail.fm/2007/09/21/reiserfs-bugs-32-bit-vs-64-bit-kernel
>>> s-cache-vs-inode-memory/
>>>
>>> Anyone have any specific thoughts?  Is there any other benefit we might
>>> see from large memory allocation in 64-bit architecture?
>>
>> Given that I wrote that blog post, I can only tell you that in our
>> environment, 64-bit kernels made a big difference.
>
> I wonder if ext3 behaves differently, Red Hat's 32-bit behaves differently,
> or if something altogether different is going on. We are currently running
> RHEL 3 in 32-bit mode, our servers have 16 GB, and most of it is used for
> caching:
>
> # free
>             total       used       free     shared    buffers     cached
> Mem:      16214344   16197612      16732          0      86944   13415172
> -/+ buffers/cache:    2695496   13518848
> Swap:      4192944       8436    4184508
>
> So it would seem that a 64-bit kernel wouldn't improve on that, right? Or
> is that a difference between 2.4 and 2.6?

64 bit kernels will be significantly more efficiant, and more reliable with that 
much memory.

in addition, in 64 bit mode the system is able to use twice as many registers on 
the CPU, which can frequently be a significant win in and of itself (even on 
machines with 1G of ram)

I've run both kernels on the same system and always have found that the 64 bit 
kernel is an advantage.

I tend to do the same thing for userspace (unless I am running something that 
doesn't work with 64 bit userspace), but there the benifit is more hit-and-miss

David Lang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ATT1922831.dat
Type: application/pgp-signature
Size: 201 bytes
Desc: ATT1922831.dat
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090416/034fd9b2/attachment.bin 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ATT1922832.txt
Url: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090416/034fd9b2/attachment.txt 

From Hagedorn at uni-koeln.de  Thu Apr 16 16:43:00 2009
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Thu, 16 Apr 2009 22:43:00 +0200
Subject: Does Cyrus benefit greatly from increased FS buffer cache?
In-Reply-To: <alpine.DEB.2.00.0904160949310.9160@shell.onid.oregonstate.edu>
References: <1605.10.70.3.113.1239740079.squirrel@borg>
	<ABCCC6006A224F01B7EF299540CB96C1@Atticus>
	<0061DD93886AC210F085086F@tyrion.rrz.uni-koeln.de>
	<alpine.DEB.2.00.0904160949310.9160@shell.onid.oregonstate.edu>
Message-ID: <FBEB68694C54811FB33CBE0E@G5.local>

-- Andrew Morgan <morgan at orst.edu> is rumored to have mumbled on 16. April 
2009 09:52:15 -0700 regarding Re: Does Cyrus benefit greatly from increased 
FS buffer cache?:

>> So it would seem that a 64-bit kernel wouldn't improve on that, right?
>> Or is  that a difference between 2.4 and 2.6?
>
> That's interesting, and not what I expected.  :)  What does "cat
> /proc/meminfo" show?

# cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  16603488256 16582602752 20885504        0 96661504 14362886144
Swap: 4293574656  8638464 4284936192
MemTotal:     16214344 kB
MemFree:         20396 kB
MemShared:           0 kB
Buffers:         94396 kB
Cached:       14018624 kB
SwapCached:       7632 kB
Active:       12502776 kB
ActiveAnon:    2275396 kB
ActiveCache:  10227380 kB
Inact_dirty:   2378304 kB
Inact_laundry:  453980 kB
Inact_clean:    284768 kB
Inact_target:  3123964 kB
HighTotal:    15597440 kB
HighFree:         5724 kB
LowTotal:       616904 kB
LowFree:         14672 kB
SwapTotal:     4192944 kB
SwapFree:      4184508 kB
Committed_AS:  2391184 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB
--
Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10
Regionales Rechenzentrum (RRZK)
Universit?t zu K?ln / Cologne University - Tel. +49-221-478-5587
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090416/258b4a1f/attachment.bin 

From cyrus at puri.jet2web.at  Fri Apr 17 03:29:14 2009
From: cyrus at puri.jet2web.at (Klemens Puritscher)
Date: Fri, 17 Apr 2009 09:29:14 +0200 (CEST)
Subject: =?utf-8?Q?AW=3A_Re=3A_Message_contains_NUL_characters_-_howto?=
	=?utf-8?Q?_dump=3F?=
Message-ID: <mtranet.20090417092914.670948422@telekom.at>


Phil Brutsche schrieb:
> The error message is being created by the LMTP service - NUL characters
> aren't valid in ASCII messages. The email in question is being generated
> incorrectly somewhere, somehow.

thanks for your reply.
I know that in the email must be a NUL character, but I cannot see this NUL character during a tcpdump.

Do you know, or someone else in this list, who can I safe find this NUL character?

> What you need to do is either have the MTA reject the message during the
> DATA portion of the SMTP transaction, or have the MTA remove the NUL
> characters before it passes the message on to the LMTP service.

Yes, this will be the next step.

> Your email headers indicate you are using Postfix as your MTA, and I am
> not familiar enough with that to tell you how to do what is necessary.

Yes, that's right for outgoing emails.
The MTA for incoming emails (mx host) is exim.

regards,
Klemens


> Klemens Puritscher wrote:
> > Hello,
> > 
> > I have a problem with one of our customers.
> > When he forwards an email with the thunderbird email client (windows
> > version), the lmtp-daemon on my cyrus-imapd (v2.3.13) rejects those
> > emails with the error "554 5.6.0 Message contains NUL characters".
> > ...ok, that's clear, there are "NUL" characters in the email.
> > 
> > But I would show my customer, where the "NUL" character is.
> > 
> > For tests, I generate a testmail, with "echo -e
> > "From:<sender at example.com>\nTo:<recipient at example.com>\nSubject:
> > test\n\ntest\0000test\n.\n" > mail_with_NUL.txt
> > 
> > Now I dump the lmtp-session on the cyrus-imapd host with:
> > tcpdump -vv -XX -s 65535 -n -i eth1 "port lmtp
> > 
> > and I see the "NUL" character:
> > ...
> >         0x0230:  7065 6564 2e61 740d 0a0d 0a74 6573 7400  peed.at....
> test.
> >         0x0240:  7465 7374 0d0a 2e0d 0a                   test.....
> > ...
> > 65 = e
> > 73 = s
> > 74 = t
> > 00 = NUL
> > 
> > ...ok, fine, I can find the "NUL" character.
> > 
> > But when I dump the lmtp-session with the customer email (which get's
> > the error "554 5.6.0 Message contains NUL characters"), I cannot find
> > this "NUL" character.
> > 
> > Can someone tell me, what I did wrong?
> > 
> > Thanks in advance.
> > 
> > Klemens
> > 
> > ----
> > Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> 
> 
> -- 
> 
> Phil Brutsche
> phil at optimumdata.com


From dvadell at linuxclusters.com.ar  Sat Apr 18 13:06:32 2009
From: dvadell at linuxclusters.com.ar (Diego M. Vadell)
Date: Sat, 18 Apr 2009 14:06:32 -0300
Subject: symlinking folders in the spool
Message-ID: <49EA0898.2040303@linuxclusters.com.ar>

Hi,

   I have a question about folders.  I want to make two folders that 
would have the same content. If you put a mail in one folder, I want to 
see it in both.

   The thing is I have a folder called "Enviados" ("Sent" in spanish) 
where our email clients put our sent messages. Lately we had some 
blackberries which put their sent messages in a folder called "Sent" and 
it looks like there is no way to configure them to put the sent items in 
the other folder.

    So I symlinked one to the other in the spool just for myself, made 
some simple tests and everything seems to work. Am I heading for 
disaster here? Is there a better way to do this that I have overlooked?

TIA
 -- Diego

From rebensburg at rz.uni-kiel.de  Mon Apr 20 02:13:53 2009
From: rebensburg at rz.uni-kiel.de (Markus Rebensburg)
Date: Mon, 20 Apr 2009 08:13:53 +0200
Subject: AW: Re: Message contains NUL characters - howto dump?
In-Reply-To: <mtranet.20090417092914.670948422@telekom.at>
References: <mtranet.20090417092914.670948422@telekom.at>
Message-ID: <49EC12A1.7090208@rz.uni-kiel.de>

Klemens Puritscher schrieb:
> Phil Brutsche schrieb:
>   
>> The error message is being created by the LMTP service - NUL characters
>> aren't valid in ASCII messages. The email in question is being generated
>> incorrectly somewhere, somehow.
>>     
>
> thanks for your reply.
> I know that in the email must be a NUL character, but I cannot see this NUL character during a tcpdump.
>
> Do you know, or someone else in this list, who can I safe find this NUL character?
>
>   
Maybe it is a problem of lines ine the email which are longer than the
standard allows. Cyrus has a fixed buffer for each line in the email. If
the line is longer than this buffer lmtp inserts a terminating string
character (NUL) itself. This could be the reason you cannot see the
Character in the tcp stream of your mail.
We have the same problem with the NUL character here produced by replies
to emails with some HTML Attachments which have only one linebreak in
the whole file.

Regards,
     Markus
>> What you need to do is either have the MTA reject the message during the
>> DATA portion of the SMTP transaction, or have the MTA remove the NUL
>> characters before it passes the message on to the LMTP service.
>>     
>
> Yes, this will be the next step.
>
>   
>> Your email headers indicate you are using Postfix as your MTA, and I am
>> not familiar enough with that to tell you how to do what is necessary.
>>     
>
> Yes, that's right for outgoing emails.
> The MTA for incoming emails (mx host) is exim.
>
> regards,
> Klemens
>
>
>   
>> Klemens Puritscher wrote:
>>     
>>> Hello,
>>>
>>> I have a problem with one of our customers.
>>> When he forwards an email with the thunderbird email client (windows
>>> version), the lmtp-daemon on my cyrus-imapd (v2.3.13) rejects those
>>> emails with the error "554 5.6.0 Message contains NUL characters".
>>> ...ok, that's clear, there are "NUL" characters in the email.
>>>
>>> But I would show my customer, where the "NUL" character is.
>>>
>>> For tests, I generate a testmail, with "echo -e
>>> "From:<sender at example.com>\nTo:<recipient at example.com>\nSubject:
>>> test\n\ntest\0000test\n.\n" > mail_with_NUL.txt
>>>
>>> Now I dump the lmtp-session on the cyrus-imapd host with:
>>> tcpdump -vv -XX -s 65535 -n -i eth1 "port lmtp
>>>
>>> and I see the "NUL" character:
>>> ...
>>>         0x0230:  7065 6564 2e61 740d 0a0d 0a74 6573 7400  peed.at....
>>>       
>> test.
>>     
>>>         0x0240:  7465 7374 0d0a 2e0d 0a                   test.....
>>> ...
>>> 65 = e
>>> 73 = s
>>> 74 = t
>>> 00 = NUL
>>>
>>> ...ok, fine, I can find the "NUL" character.
>>>
>>> But when I dump the lmtp-session with the customer email (which get's
>>> the error "554 5.6.0 Message contains NUL characters"), I cannot find
>>> this "NUL" character.
>>>
>>> Can someone tell me, what I did wrong?
>>>
>>> Thanks in advance.
>>>
>>> Klemens
>>>
>>> ----
>>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>>>       
>> -- 
>>
>> Phil Brutsche
>> phil at optimumdata.com
>>     
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090420/10257f8f/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rebensburg.vcf
Type: text/x-vcard
Size: 351 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090420/10257f8f/attachment.vcf 

From scrappy at hub.org  Mon Apr 20 08:34:27 2009
From: scrappy at hub.org (Marc G. Fournier)
Date: Mon, 20 Apr 2009 09:34:27 -0300 (ADT)
Subject: sieveshell error 'Bad protocol'
Message-ID: <20090420093110.H84471@hub.org>


sieveshell seems to be broken ... ?


# sieveshell --user=scrappy at hub.org localhost
connecting to localhost
Please enter your password:
Bad protocol from MANAGESIEVE server: lost connection


But everything seems to be responding fine:


# telnet localhost sieve
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
"IMPLEMENTATION" "Cyrus timsieved v2.3.12p2"
"SASL" "PLAIN LOGIN DIGEST-MD5 CRAM-MD5"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags 
notify envelope relational regex subaddress copy"
OK
qiut
Connection closed by foreign host.

# telnet localhost imap
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID AUTH=PLAIN AUTH=LOGIN 
AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR] hub.org Cyrus IMAP v2.3.12p2 server 
ready
. login scrappy at hub.org xxxx
. OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED ACL RIGHTS=kxte 
QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT 
CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT 
THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE SCAN IDLE LISTEXT 
LIST-SUBSCRIBED X-NETSCAPE URLAUTH] User logged in

Something I'm overlooking? :(

Thx

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy at hub.org                              MSN . scrappy at hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664

From cyrus at fiddaman.net  Mon Apr 20 14:15:12 2009
From: cyrus at fiddaman.net (Andy Fiddaman)
Date: Mon, 20 Apr 2009 18:15:12 +0000 (GMT)
Subject: Attachment corruption when downloading with Thunderbird...
Message-ID: <alpine.GSO.2.00.0904201808010.5817@areb.pvgehf-vg.arg>


Hi,

I'm running Cyrus IMAP 2.3.13 on Solaris (about to upgrade to 2.3.14) and a
number of my users who use Thunderbird have reported frequent corruption
of attachments.

This is particularly visible for images where they are obviously truncated.
The work-around they are using is to drag the message into another folder,
read it then drag it back.

The messages are never corrupt within Cyrus and look fine when viewed
with other clients which suggests a bug within Thunderbird or an interaction
issue.

I found some threads on the Mozilla lists which suggests that this is a
server side bug where the server reports the wrong size for message
parts and the suggested workaround is to set
mail.server.default.fetch_by_chunks to false within Thunderbird.

 (http://forums.mozillazine.org/viewtopic.php?f=29&t=1045555)

So, has anyone else had any reports of this behaviour or any reason to
believe that Thunderbird does not work well with Cyrus? I'm going to
enable telemetry for one of the users who has reported this and see
if I can see anything relevant in the IMAP session; any suggestions of
other places to look would be appreciated.

Thanks,

Andy


From brong at fastmail.fm  Mon Apr 20 21:03:58 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Tue, 21 Apr 2009 11:03:58 +1000
Subject: Attachment corruption when downloading with Thunderbird...
In-Reply-To: <alpine.GSO.2.00.0904201808010.5817@areb.pvgehf-vg.arg>
References: <alpine.GSO.2.00.0904201808010.5817@areb.pvgehf-vg.arg>
Message-ID: <20090421010358.GA6291@brong.net>

On Mon, Apr 20, 2009 at 06:15:12PM +0000, Andy Fiddaman wrote:
> I'm running Cyrus IMAP 2.3.13 on Solaris (about to upgrade to 2.3.14) and a
> number of my users who use Thunderbird have reported frequent corruption
> of attachments.
> 
> So, has anyone else had any reports of this behaviour or any reason to
> believe that Thunderbird does not work well with Cyrus? I'm going to
> enable telemetry for one of the users who has reported this and see
> if I can see anything relevant in the IMAP session; any suggestions of
> other places to look would be appreciated.

I'd love to see the telemetry.

Bron ( wondering if Thunderbird is fetching the encoded size and then
       fetching it decoded or something?? )

From raymond at sundland.com  Mon Apr 20 21:57:22 2009
From: raymond at sundland.com (Raymond T. Sundland)
Date: Mon, 20 Apr 2009 21:57:22 -0400
Subject: Attachment corruption when downloading with Thunderbird...
In-Reply-To: <20090421010358.GA6291@brong.net>
References: <alpine.GSO.2.00.0904201808010.5817@areb.pvgehf-vg.arg>
	<20090421010358.GA6291@brong.net>
Message-ID: <49ED2802.90702@sundland.com>

I use Thunderbird exclusively with Cyrus and have never had issues with 
attachments.  Am running cyrus 2.2, but I don't see why 2.3 would 
suddenly break that functionality.

Bron Gondwana wrote:
> On Mon, Apr 20, 2009 at 06:15:12PM +0000, Andy Fiddaman wrote:
>   
>> I'm running Cyrus IMAP 2.3.13 on Solaris (about to upgrade to 2.3.14) and a
>> number of my users who use Thunderbird have reported frequent corruption
>> of attachments.
>>
>> So, has anyone else had any reports of this behaviour or any reason to
>> believe that Thunderbird does not work well with Cyrus? I'm going to
>> enable telemetry for one of the users who has reported this and see
>> if I can see anything relevant in the IMAP session; any suggestions of
>> other places to look would be appreciated.
>>     
>
> I'd love to see the telemetry.
>
> Bron ( wondering if Thunderbird is fetching the encoded size and then
>        fetching it decoded or something?? )
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090420/b6f61f59/attachment.html 

From ted at lyncon.se  Tue Apr 21 03:50:58 2009
From: ted at lyncon.se (ted at lyncon.se)
Date: Tue, 21 Apr 2009 09:50:58 +0200
Subject: Attachment corruption when downloading with Thunderbird...
In-Reply-To: <49ED2802.90702@sundland.com>
References: <alpine.GSO.2.00.0904201808010.5817@areb.pvgehf-vg.arg>
	<20090421010358.GA6291@brong.net> <49ED2802.90702@sundland.com>
Message-ID: <1240300258.49ed7ae223f3b@lyncon.se>

Citerar "Raymond T. Sundland" <raymond at sundland.com>:
>
> I use Thunderbird exclusively with Cyrus and have never had issues with
> attachments.  Am running cyrus 2.2, but I don't see why 2.3 would
> suddenly break that functionality.

I'm running Thunderbird with Cyrus 2.3 and so far it's worked fine.

Br,
Ted

> Bron Gondwana wrote:
> > On Mon, Apr 20, 2009 at 06:15:12PM +0000, Andy Fiddaman wrote:
> >
> >> I'm running Cyrus IMAP 2.3.13 on Solaris (about to upgrade to 2.3.14) and
> a
> >> number of my users who use Thunderbird have reported frequent corruption
> >> of attachments.
> >>
> >> So, has anyone else had any reports of this behaviour or any reason to
> >> believe that Thunderbird does not work well with Cyrus? I'm going to
> >> enable telemetry for one of the users who has reported this and see
> >> if I can see anything relevant in the IMAP session; any suggestions of
> >> other places to look would be appreciated.
> >>
> >
> > I'd love to see the telemetry.
> >
> > Bron ( wondering if Thunderbird is fetching the encoded size and then
> >        fetching it decoded or something?? )
> > ----
> > Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> >
>




From andyjpb at ashurst.eu.org  Tue Apr 21 06:13:22 2009
From: andyjpb at ashurst.eu.org (Andy Bennett)
Date: Tue, 21 Apr 2009 11:13:22 +0100
Subject: Delivery to Shared Folders via authenticated SMTP then LMTP
Message-ID: <49ED9C42.60205@ashurst.eu.org>

Hi,

I'm having problems getting delivering messages via exim to Shared 
Folders under cyrus.

I've googled around and futzed with configuration options for an entire 
afternoon and not got very far so I'm wondering if anyone here can help me.

First, here's a few words about my configuration.

I'm running a Debian etch server with the cyrus-2.2 (2.2.13-10) packages 
installed. I'm using exim 4.63 as my MTA.

Exim's set up to relay outgoing mail via authenticated SMTP and incoming 
mail for a few domains.

SMTP authentication uses the same database as the cyrus IMAP server.
Here's how my plaintext exim authenticator works:
server_condition = ${if 
saslauthd{{${local_part:$2}}{$3}{smtpauth}{${domain:$2}}}{1}{0}}

I'm using cyrus in "virtdomains: userid" mode.

I'm doing delivery to cyrus over authenticated LMTP via a socket.

I'm running lmtp like this:
lmtp            cmd="lmtpd" listen="localhost:lmtp" prefork=0 maxchild=20

I have "lmtp_admins: exim" in /etc/imapd.conf

Exim is authenticating to the LMTP server with CRAM-MD5 as user exim.

Delivery works for users in all domains.

I have no "postuser:" setting in /etc/imapd.conf so I'm assuming that 
it's default and I can address shared folders with the "+xxx at domain" 
address.


I have created the following shared folders in cyradm:

shared.test at ashurst.eu.org (\HasNoChildren)
shared at ashurst.eu.org (\HasChildren)

...and here are the permissions:

shared at ashurst.eu.org:
   anyone lrs
shared.test at ashurst.eu.org:
   exim lrswipcda
   andyjpb at ashurst.eu.org lrswipcda
   anyone lrs


I can insert and delete messages in shared.test via IMAP when I'm 
authenticaed as andyjpb at ashurst.eu.org

Whatever permissions I give to andyjpb at ashurst.eu.org I can't do insert 
or delete messages in shared via IMAP when I'm authenticated as 
andyjpb at ashurst.eu.org
Are top level folders special?



With the ACLs above, I ran a test.

Sending messages to any user at any domain that I have set up, from 
anywhere, works fine.

I connected to my SMTP server, authenticated as andyjpb at ashurst.eu.org 
and sent a message to "+shared.test at ashurst.eu.org".

If the mailbox does not exist I get a message saying so.

If the mailbox does exist (as configured above) then I get a different 
error message, so I'm pretty happy that I've got the correct eMail 
address for the mailbox I created...

The message was accepted by exim and then immediately bounced.
... I don't do local part checking at RCPT time in submission mode.

Anyway, I switched on the Cyrus session logging for the exim user and 
here's what I got. It includes the error message that was sent in the 
bounce message.

-----

---------- exim Mon Apr 20 22:57:35 2009

 >1240264655>235 Authenticated!
<1240264655<MAIL FROM:<andyjpb at ashurst.eu.org> SIZE=2523
RCPT TO:<+shared.test at ashurst.eu.org>
DATA
 >1240264655>250 2.1.0 ok
550-You do not have permission to post a message to this mailbox.
550-Please contact the owner of this mailbox in order to submit
550-your message, or postmaster if you believe you
550-received this message in error.
550 5.7.1 Permission denied
503 5.5.1 No recipients
<1240264655<QUIT
 >1240264655>221 2.0.0 bye

-----

The log then continues with the successful delivery of the bounce 
message to andyjpb at ashurst.eu.org

The bounce message doesn't contain the "503 5.5.1 No recipients" line: 
it stops at "550 5.7.1 Permission denied"



So...

It looks like exim is authenticating as the exim user, which is in 
lmtp_admins. I also tried putting exim in admins and it didn't change 
anything.
Is there anyway of getting more information about who was authenticated 
and who was authorised?
Here's what I get in syslog:
-----
verify_user(ashurst.eu.org!shared.test) failed: Permission denied
-----



Here's the ACL that's on andyjpb at ashurst.eu.org's INBOX:

andyjpb at ashurst.eu.org lrswipcda


...so exim doesn't have 'p' rights there but it can still deliver mail 
there.

exim isn't in a domain: all the other users are. I'm not sure if that is 
an issue when using Cyrus in "virtdomains: user_id" mode, and I haven't 
got exim configured to connect to lmtp as a different user depending on 
the domain.

RCPT TO: in the error looks like the correct mailbox. MAIL FROM: is a 
user that has 'p' permission on the mailbox.

I don't see an AUTH line tho... I'm authenticating as exim who should be 
able to authorise as andyjpb at ashurst.eu.org. How can I be sure that that 
is happening? If it's not then as exim has 'p' rights on the mailbox it 
should be able to post as itself anyway.
I haven't done anything special in exim as the documentation led me to 
believe that the authentication automatically falls through.


If I give "anyone" 'p' rights then messages are delivered without errors.



As a last ditch attempt, I just reconfigured exim to use PLAIN rather 
than CRAM-MD5 when authenticating to LMTP so that I could explicitly 
send the exim authenticated sender along to LMTP.
Here's the authentication details I used:
-----
   client_send = $authenticated_sender^exim^<PASSWORD>
-----

I think that should send the exim authenticated sender along as the 
authorisation and exim and <PASSWORD> along as the authentication.




Does anyone have any idea what I am doing incorrectly or whether I 
should be doing something that am not?



Many thanks for your time.




Regards,
@ndy

-- 
andyjpb at ashurst.eu.org
http://www.ashurst.eu.org/
http://www.gonumber.com/andyjpb
0x7EBA75FF


From cyrus at puri.jet2web.at  Tue Apr 21 07:18:10 2009
From: cyrus at puri.jet2web.at (Klemens Puritscher)
Date: Tue, 21 Apr 2009 13:18:10 +0200 (CEST)
Subject: =?utf-8?Q?AW=3A_Re=3A_AW=3A_Re=3A_Message_contains_NUL_charac?=
	=?utf-8?Q?ters_-_howto_dump=3F?=
Message-ID: <mtranet.20090421131810.445354963@telekom.at>


Markus Rebensburg schrieb:
> Klemens Puritscher schrieb:
> > Phil Brutsche schrieb:
> >   
> >> The error message is being created by the LMTP service - NUL 
> characters
> >> aren't valid in ASCII messages. The email in question is being 
> generated
> >> incorrectly somewhere, somehow.
> >>     
> >
> > thanks for your reply.
> > I know that in the email must be a NUL character, but I cannot see 
> this NUL character during a tcpdump.
> >
> > Do you know, or someone else in this list, who can I safe find this 
> NUL character?
> >
> >   
> Maybe it is a problem of lines ine the email which are longer than the
> standard allows. Cyrus has a fixed buffer for each line in the email. If
> the line is longer than this buffer lmtp inserts a terminating string
> character (NUL) itself. This could be the reason you cannot see the
> Character in the tcp stream of your mail.
> We have the same problem with the NUL character here produced by replies
> to emails with some HTML Attachments which have only one linebreak in
> the whole file.


thanks for this hint.
This possibility should I have already fixed, in my exim-config:

[...]
acl_check_data:
  deny   message      = Line too long
         regex        = ^.{4000,}
  accept
[...]

(IMHO is the max. line lenght in emails 4000 characters.)

Are there other possibilities for the lmtp error "Message contains NUL characters"?

regards,
Klemens



> Regards,
>      Markus
> >> What you need to do is either have the MTA reject the message during 
> the
> >> DATA portion of the SMTP transaction, or have the MTA remove the NUL
> >> characters before it passes the message on to the LMTP service.
> >>     
> >
> > Yes, this will be the next step.
> >
> >   
> >> Your email headers indicate you are using Postfix as your MTA, and I 
> am
> >> not familiar enough with that to tell you how to do what is necessary.
> 
> >>     
> >
> > Yes, that's right for outgoing emails.
> > The MTA for incoming emails (mx host) is exim.
> >
> > regards,
> > Klemens
> >
> >
> >   
> >> Klemens Puritscher wrote:
> >>     
> >>> Hello,
> >>>
> >>> I have a problem with one of our customers.
> >>> When he forwards an email with the thunderbird email client (windows
> >>> version), the lmtp-daemon on my cyrus-imapd (v2.3.13) rejects those
> >>> emails with the error "554 5.6.0 Message contains NUL characters".
> >>> ...ok, that's clear, there are "NUL" characters in the email.
> >>>
> >>> But I would show my customer, where the "NUL" character is.
> >>>
> >>> For tests, I generate a testmail, with "echo -e
> >>> "From:<sender at example.com>\nTo:<recipient at example.com>\nSubject:
> >>> test\n\ntest\0000test\n.\n" > mail_with_NUL.txt
> >>>
> >>> Now I dump the lmtp-session on the cyrus-imapd host with:
> >>> tcpdump -vv -XX -s 65535 -n -i eth1 "port lmtp
> >>>
> >>> and I see the "NUL" character:
> >>> ...
> >>>         0x0230:  7065 6564 2e61 740d 0a0d 0a74 6573 7400  peed.at....
> 
> >>>       
> >> test.
> >>     
> >>>         0x0240:  7465 7374 0d0a 2e0d 0a                   test.....
> >>> ...
> >>> 65 = e
> >>> 73 = s
> >>> 74 = t
> >>> 00 = NUL
> >>>
> >>> ...ok, fine, I can find the "NUL" character.
> >>>
> >>> But when I dump the lmtp-session with the customer email (which 
> get's
> >>> the error "554 5.6.0 Message contains NUL characters"), I cannot 
> find
> >>> this "NUL" character.
> >>>
> >>> Can someone tell me, what I did wrong?
> >>>
> >>> Thanks in advance.
> >>>
> >>> Klemens
> >>>
> >>> ----
> >>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> >>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> >>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> >>>       
> >> -- 
> >>
> >> Phil Brutsche
> >> phil at optimumdata.com
> >>     
> >
> > ----
> > Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> >
> >   
> 


From andreas.moroder at sb-brixen.it  Tue Apr 21 08:16:22 2009
From: andreas.moroder at sb-brixen.it (Andreas Moroder)
Date: Tue, 21 Apr 2009 14:16:22 +0200
Subject: imp webmail, cyrus imap and virus filtering
Message-ID: <49EDB916.9080105@sb-brixen.it>

 >
 >
 > You mean mail already already in your INBOXes received before you have
 > installed your trendmicros filter, or mail sent internally by your
 > user ?
 >
 > In the last case the simple solution is to ask your user to send email
 > directly to your trendmicro !
 > If this is not possible you can configure your trendmicros as a filter
 > for your postfix !
 > But if you want keep your trendmicro in front for your incoming email,
 > and have postfix in front for your local users, this is an unusual
 > configuration, ask the postfix mailing list for information to do that
 > !
 >
 > Regards
 >
Hello Alain,

I reanalyzed our actual configuration and found that the problem is more 
limited,because when I send a mail via imp webmail then imp passes the 
mail to postfix and the antivirus.

The problem that remains is about drafts. When a user saves a mail as 
draft, then it is not sent but simply stored by cyrus. This way the mail 
is not scanned. The user can use the drafts as a file storage and then 
recall the files from another PC. Is there a solution for this case ?

Thanks
Andreas

From brennan at columbia.edu  Tue Apr 21 09:08:19 2009
From: brennan at columbia.edu (Joseph Brennan)
Date: Tue, 21 Apr 2009 09:08:19 -0400
Subject: AW: Re: AW: Re: Message contains NUL characters - howto dump?
In-Reply-To: <mtranet.20090421131810.445354963@telekom.at>
References: <mtranet.20090421131810.445354963@telekom.at>
Message-ID: <9A0B6702B5F87B202CB53DB0@[192.168.2.14]>


> (IMHO is the max. line lenght in emails 4000 characters.)


RFC 2821 sec 4.5.3.1 says the max length is 1000 characters including
the two CR LF characters.

However if the MTA fixes this, Cyrus won't see it.  Sendmail for example
breaks long lines at 997 characters and inserts ! CR LF.

Joseph Brennan
Lead Email Systems Engineer
Columbia University Information Technology


From Duncan.Gibb at SiriusIT.co.uk  Tue Apr 21 10:55:44 2009
From: Duncan.Gibb at SiriusIT.co.uk (Duncan Gibb)
Date: Tue, 21 Apr 2009 15:55:44 +0100
Subject: Delivery to Shared Folders via authenticated SMTP then LMTP
In-Reply-To: <49ED9C42.60205@ashurst.eu.org>
References: <49ED9C42.60205@ashurst.eu.org>
Message-ID: <49EDDE70.8050707@SiriusIT.co.uk>

Andy Bennett wrote:

AB> I'm running a Debian etch server with the cyrus-2.2 (2.2.13-10)
AB> packages installed. I'm using exim 4.63 as my MTA.

OK.  Not an untypical deployment...

AB> I have no "postuser:" setting in /etc/imapd.conf so I'm assuming
AB> that it's default and I can address shared folders with the
AB> "+xxx at domain" address.

The default postuser is the empty string, hence the need for "anyone"
ACLs you're seeing.

AB> I can insert and delete messages in shared.test via IMAP when I'm
AB> authenticaed as andyjpb at ashurst.eu.org

AB> I connected to my SMTP server, authenticated as
AB> andyjpb at ashurst.eu.org and sent a message to
AB> "+shared.test at ashurst.eu.org".

AB> The message was accepted by exim and then immediately bounced.

AB> MAIL FROM:<andyjpb at ashurst.eu.org> SIZE=2523
AB> RCPT TO:<+shared.test at ashurst.eu.org>
AB> 550-You do not have permission to post a message to this mailbox.

AB> I don't see an AUTH line tho... I'm authenticating as exim who
AB> should be able to authorise as andyjpb at ashurst.eu.org. How can I
AB> be sure that that is happening?

You should have lines in syslog (/var/log/maillog) from lmtpd of the form

  cyrus/lmtp[<PID>]: login: <MTA.HOSTNAME> [<MTA.IP>] <authzid>
      <SASL.MECH> User logged in

The authzid there will be the user as whom Exim authorized.  But I don't
think that's the problem (see below).

AB>    client_send = $authenticated_sender^exim^<PASSWORD>

AB> I think that should send the exim authenticated sender along
AB> as the authorisation and exim and <PASSWORD> along as the
AB> authentication.

It should, but not in the way you want.  The SASL authzid isn't what
lmtpd evaluates ACLs against.  To do what I think you want (ACLs for
delivery to shared mailboxes by users employing SMTPA), you need Exim to
pass the authenticated user from the SMTP transaction with the MUA into
the _MAIL_ line of the LMTP conversation.  You want Exim to say:

 MAIL FROM:<andyjpb at ashurst.eu.org> AUTH=<andyjpb at ashurst.eu.org>

To do that you probably want to add

  authenticated_sender = $authenticated_id

to the definition of your lmtp relay.

You can check Cyrus is doing what you expect by using openssl s_client
or gnutls-cli to have a manual LMTP conversation with it:

 <-  220 your.cyrus.box LMTP Cyrus v2.3.13-Sirius-2009:2.3.13-5 ready
 ->  lhlo authtest
 <-  250-your.cyrus.box
 <-  250-[..]
 <-  250-AUTH PLAIN LOGIN
 ->  auth plain base64.nonsense.or.go.back.to.cram-md5
 <-  235 Authenticated!
 ->  mail from:<arbitrary at mail.addr> AUTH=<andyjpb at ashurst.eu.org>
 <-  250 2.1.0 ok
 ->  rcpt to:<+shared.test at ashurst.eu.org>
 <-  250 2.1.5 ok
 ->  data
 <-  354 go ahead

etc...


Cheers


Duncan

-- 
Duncan Gibb - Technical Director
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk/ || t: +44 870 608 0063
Debian Cyrus Team
https://alioth.debian.org/projects/pkg-cyrus-imapd/

From andyjpb at ashurst.eu.org  Tue Apr 21 12:28:12 2009
From: andyjpb at ashurst.eu.org (Andy Bennett)
Date: Tue, 21 Apr 2009 17:28:12 +0100
Subject: Delivery to Shared Folders via authenticated SMTP then LMTP
In-Reply-To: <49EDDE70.8050707@SiriusIT.co.uk>
References: <49ED9C42.60205@ashurst.eu.org> <49EDDE70.8050707@SiriusIT.co.uk>
Message-ID: <49EDF41C.5070506@ashurst.eu.org>

Hi,

Thanks for your reply.


> You should have lines in syslog (/var/log/maillog) from lmtpd of the form
> 
>   cyrus/lmtp[<PID>]: login: <MTA.HOSTNAME> [<MTA.IP>] <authzid>
>       <SASL.MECH> User logged in
> 
> The authzid there will be the user as whom Exim authorized.  But I don't
> think that's the problem (see below).

I do.
<authzid> is exim.


> AB>    client_send = $authenticated_sender^exim^<PASSWORD>
> 
> AB> I think that should send the exim authenticated sender along
> AB> as the authorisation and exim and <PASSWORD> along as the
> AB> authentication.
> 
> It should, but not in the way you want.  The SASL authzid isn't what
> lmtpd evaluates ACLs against.  To do what I think you want (ACLs for
> delivery to shared mailboxes by users employing SMTPA), you need Exim to
> pass the authenticated user from the SMTP transaction with the MUA into
> the _MAIL_ line of the LMTP conversation.  You want Exim to say:
> 
>  MAIL FROM:<andyjpb at ashurst.eu.org> AUTH=<andyjpb at ashurst.eu.org>

Yes... I think that's what I'm looking for.

A review of the logs shows that when I was passing authorisation with 
client_send = $authenticated_sender^exim^<PASSWORD>

I was getting

cyrus/lmtp[<PID>]: login: <MTA.HOSTNAME> [<MTA.IP>] 
<$authenticated_sender> PLAIN User logged in

instead of the "exim" one above.

...but anyway.

Something more sinister is wrong.

I thought that messages were being delivered correctly in non shared 
folders scenarios because every test message I sent from external 
relays, such as gmail, were being received. However, the logs show 
things like this

-----
1 /var/log/exim4/rejectlog:2009-04-21 16:35:21 
H=cp-dublin.purplecloud.com (mx01-dublin.purplecloud.com) [91.194.74.36] 
F=<andyjpd at btopenworld.com> temporarily rejected RCPT 
<andyjpb at ashurst.eu.org>: response to "MAIL FROM:<>" from localhost 
[127.0.0.1] was: 430 Authentication required
-----

At first I thought that this was just for illegitimate mail that wasn't 
specifying MAIL FROM: properly; I get a lot of spam that is backscatter 
from bounces.

However, I eventually noticed legitimate ones such as traffic to this list

-----
2009-04-21 15:10:27 H=mx2.andrew.cmu.edu [128.2.11.36] 
F=<info-cyrus-bounces+andyjpb=ashurst.eu.org at lists.andrew.cmu.edu> 
temporarily rejected RCPT <andyjpb at ashurst.eu.org>: response to "MAIL 
FROM:<>" from localhost [127.0.0.1] was: 430 Authentication require
-----

Your reply went to the list and directly to me: the direct one came 
through but the one from mailman got stuck between my smtp and lmtp 
servers and was therefore temporarily rejected.


For now, I've gone back to using lmtp in "lmtp -a" mode and it seems to 
have fixed things... Hopefully all the temporarily rejected mail will 
start to come through in the next few hours.

However, I'm not ready to give up on getting authenticated lmtp and then 
shared folder delivery working.

Why do different things happen when running "lmtp -a" compared to "lmtp" 
and logging in as an lmtp_admin?


> To do that you probably want to add
> 
>   authenticated_sender = $authenticated_id
> 
> to the definition of your lmtp relay.

I'll give that a go just as soon as I've fixed the normal delivery, thanks.

It appeals to my common sense that the two problems are related: Do I 
need to pass authenticated_sender = exim to lmtp for all cases except 
when I have an SMTPA sender? Do I also need to grant 'p' rights to exim 
on users' INBOXes?

I'm not really clear why it is sometimes failing and sometimes 
succeeding in the non shared folders case.


> You can check Cyrus is doing what you expect by using openssl s_client
> or gnutls-cli to have a manual LMTP conversation with it:
> 
>  <-  220 your.cyrus.box LMTP Cyrus v2.3.13-Sirius-2009:2.3.13-5 ready
>  ->  lhlo authtest
>  <-  250-your.cyrus.box
>  <-  250-[..]
>  <-  250-AUTH PLAIN LOGIN
>  ->  auth plain base64.nonsense.or.go.back.to.cram-md5
>  <-  235 Authenticated!
>  ->  mail from:<arbitrary at mail.addr> AUTH=<andyjpb at ashurst.eu.org>
>  <-  250 2.1.0 ok
>  ->  rcpt to:<+shared.test at ashurst.eu.org>
>  <-  250 2.1.5 ok
>  ->  data
>  <-  354 go ahead

Yeah. I might try that... Although I told exim to avoid TLS with the 
LMTP server for now so that I might debug it and so I might be able to 
just telnet to the lmtp port.



Thanks for your help.




Regards,
@ndy

-- 
andyjpb at ashurst.eu.org
http://www.ashurst.eu.org/
http://www.gonumber.com/andyjpb
0x7EBA75FF


From pobox at verysmall.org  Wed Apr 22 03:39:29 2009
From: pobox at verysmall.org (Iv Ray)
Date: Wed, 22 Apr 2009 09:39:29 +0200
Subject: initial slowness
In-Reply-To: <1075531C-41AB-4610-9A52-7CF95EAD3977@verysmall.org>
References: <EFA6B8CC-D282-444A-A592-A4FA96DD0F1A@verysmall.org>
	<49A482A8.4020303@sundland.com>
	<1075531C-41AB-4610-9A52-7CF95EAD3977@verysmall.org>
Message-ID: <98264C0C-580D-4199-AE67-CDD224AD69ED@verysmall.org>


On 25.02.2009, at 01:15, Iv Ray wrote:

> On 25.02.2009, at 00:28, Raymond T. Sundland wrote:
>
>> The other option I ran into some time ago... and it had to due to
>> defining multiple authentication schemes in the SASL configuration.
>> When you define multiple, it will try each one in order.  I stripped
>> it down to just PLAIN, which is what I wanted to use, and it was
>> much quicker.  The cyradm script will prompt for the password before
>> actually trying to connect, which is why it's slow AFTER the
>> password is entered.
>
> Raymond,
>
> Thanks a lot for the hints.
>
> I'll try them and write back if success.
>
> Iv

Hi Raymond,

I recompiled Cyrus SASL leaving only PLAIN, but the slowness continues.

I guess I should ask at the Cyrus SASL list...

Thanks anyway,
Iv

From ml at awinkelmann.de  Wed Apr 22 05:49:41 2009
From: ml at awinkelmann.de (Andreas Winkelmann)
Date: Wed, 22 Apr 2009 11:49:41 +0200 (CEST)
Subject: initial slowness
In-Reply-To: <98264C0C-580D-4199-AE67-CDD224AD69ED@verysmall.org>
References: <EFA6B8CC-D282-444A-A592-A4FA96DD0F1A@verysmall.org>
	<49A482A8.4020303@sundland.com>
	<1075531C-41AB-4610-9A52-7CF95EAD3977@verysmall.org>
	<98264C0C-580D-4199-AE67-CDD224AD69ED@verysmall.org>
Message-ID: <819d65b41c0a69c7bc201d09ef3c2067.squirrel@a-angels.ath.cx>

>>> The other option I ran into some time ago... and it had to due to
>>> defining multiple authentication schemes in the SASL configuration.
>>> When you define multiple, it will try each one in order.  I stripped
>>> it down to just PLAIN, which is what I wanted to use, and it was
>>> much quicker.  The cyradm script will prompt for the password before
>>> actually trying to connect, which is why it's slow AFTER the
>>> password is entered.

>> I'll try them and write back if success.

> I recompiled Cyrus SASL leaving only PLAIN, but the slowness continues.

Re-compiling does not remove already installed Mechanismsm. Mechanisms are
Files which are dynamially loaded on startup and used if they are present.

You can disable others than plain with "sasl_mech_list: plain" in your
imapd.conf.

But is this relly the reason?

> I guess I should ask at the Cyrus SASL list...

Use imtest to Login to your IMAP-Server and look where the time is spend.

Look in the Logs for errors/warnings.

--
Andreas


From ana.ribas at upcnet.es  Wed Apr 22 06:20:31 2009
From: ana.ribas at upcnet.es (Ana Ribas Roca)
Date: Wed, 22 Apr 2009 12:20:31 +0200
Subject: much information from cyrus log
Message-ID: <20090422122031.203819yybofayjxr@vader.upc.es>

Hello

I'm trying to get as much information as possible from the cyrus log.  
I've tried several modification of the syslog configuration  
filewithout success.

I also?create a folder at /var/log/.. per user getting as much  
information I want but... it's in different files and folders, and  
is?per user based solution, which is difficult to administrate.

Any clue on how to configure cyrus and syslog to retrieve all this info?

Thanks a lot

- ANNA -

From ml at awinkelmann.de  Wed Apr 22 06:40:09 2009
From: ml at awinkelmann.de (Andreas Winkelmann)
Date: Wed, 22 Apr 2009 12:40:09 +0200 (CEST)
Subject: much information from cyrus log
In-Reply-To: <20090422122031.203819yybofayjxr@vader.upc.es>
References: <20090422122031.203819yybofayjxr@vader.upc.es>
Message-ID: <03a0e812a16045234a721822063fb88e.squirrel@a-angels.ath.cx>

> I'm trying to get as much information as possible from the cyrus log.
> I've tried several modification of the syslog configuration
> filewithout success.
>
> I also?create a folder at /var/log/.. per user getting as much
> information I want but... it's in different files and folders, and
> is?per user based solution, which is difficult to administrate.
>
> Any clue on how to configure cyrus and syslog to retrieve all this info?

Maybe you should go the other way around. Tell what information you need.

Cyrus sends alot of information with LOG_DEBUG to syslog, check if you
catch these Messages. The directories you mentioned are telemetry Logs,
these Dirs are in $configdirectory, which is normally not in /var/log/...

--
Andreas


From Duncan.Gibb at SiriusIT.co.uk  Wed Apr 22 09:01:15 2009
From: Duncan.Gibb at SiriusIT.co.uk (Duncan Gibb)
Date: Wed, 22 Apr 2009 14:01:15 +0100
Subject: Delivery to Shared Folders via authenticated SMTP then LMTP
In-Reply-To: <49EDF41C.5070506@ashurst.eu.org>
References: <49ED9C42.60205@ashurst.eu.org> <49EDDE70.8050707@SiriusIT.co.uk>
	<49EDF41C.5070506@ashurst.eu.org>
Message-ID: <49EF151B.7000002@SiriusIT.co.uk>

Andy Bennett wrote:

DG> To do what I think you want (ACLs for delivery to shared
DG> mailboxes by users employing SMTPA), you need Exim to pass
DG> the authenticated user from the SMTP transaction with the
DG> MUA into the _MAIL_ line of the LMTP conversation.

DG>  MAIL FROM:<andyjpb at ashurst.eu.org> AUTH=<andyjpb at ashurst.eu.org>

AB> Something more sinister is wrong.

AB> I thought that messages were being delivered correctly in
AB> non shared folders scenarios because every test message I
AB> sent from external relays, such as gmail, were being received.

AB> However, I eventually noticed [rejections of] legitimate ones
AB> such as traffic to this list

> 2009-04-21 15:10:27 H=mx2.andrew.cmu.edu [128.2.11.36] 
> F=<info-cyrus-bounces+andyjpb=ashurst.eu.org at lists.andrew.cmu.edu> 
> temporarily rejected RCPT <andyjpb at ashurst.eu.org>: response to "MAIL 
> FROM:<>" from localhost [127.0.0.1] was: 430 Authentication require

AB> Your reply went to the list and directly to me: the direct one
AB> came through but the one from mailman got stuck between my smtp
AB> and lmtp servers and was therefore temporarily rejected.

That sounds like an Exim problem, and hence might be better discussed on
an Exim list.  I've not seem what you've done with your shared mailbox
delivery rules, but is it possible the + characters in mailman
envelope-from addresses are confusing it?


AB> For now, I've gone back to using lmtp in "lmtp -a" mode and it
AB> seems to have fixed things... Hopefully all the temporarily
AB> rejected mail will start to come through in the next few hours.

IMHO "lmtpd -a" is the work of the devil ;-)  Not to be used unless the
MTAs and the front-end Cyruses are on a secure network segment, you
really really trust your MTAs and don't have auto-* enabled...


AB> Why do different things happen when running "lmtp -a" compared
AB> to "lmtp" and logging in as an lmtp_admin?

The only difference between "lmtpd" and "lmtpd -a" should be that the
latter fakes a SASL external auth for the default lmtp admin, "postman".

When you say "logging in", I presume you're talking about an IMAP
session.  IMAP and LMTP necessarily present different views of the
mailbox namespace: in the IMAP case the view you get is the authzid's
view; in the LMTP case the view of the namespace we're interested in is
normally the recipient's view.

The same goes for ACLs.  In the IMAP case the relevant ACLs are those
which apply for the authzid, and being an lmtp_admin (rather than an
(imap(s)_)admin) doesn't grant any special rights in this case.  In the
LMTP case, the there are multiple cases to consider:

For delivery to a user, the ACLs outside of the user's private IMAP
space (INBOX and children) which need to be considered are those for the
user, since they control where a sieve script may put mail exactly as
the ACLs in the IMAP case control what the MUA may do.  For delivery to
a shared folder, you have the case where you use a postuser - in which
case the ACLs for the postuser apply - and the the case where you use an
authenticated sender, in which case you can create ACLs by sender...

You're doing the last of these, which is the most complex.


AB> Do I need to pass authenticated_sender = exim to lmtp for all
AB> cases except when I have an SMTPA sender?

I don't think so.  If the SMTP transaction is not authenticated, Exim
should not add the "AUTH=" cause to the MAIL line, and should pass an
empty authzid when it does its LMTP AUTH.


AB> Do I also need to grant 'p' rights to exim on users' INBOXes?

That right is implicit in being able to do lmtp at all - or, if you
prefer, it's the receiving user's ACL which counts here.


AB> I'm not really clear why it is sometimes failing and sometimes
AB> succeeding in the non shared folders case.

I think you need to find some examples of working and not working, and
log exactly what Exim is saying to lmtpd.  Then you can figure out
whether Exim's correct and lmtpd is broken, or lmtpd is doing the Right
Thing in response to something weird coming in from Exim.


Cheers


Duncan

-- 
Duncan Gibb - Technical Director
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk/ || t: +44 870 608 0063
Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/

From andyjpb at ashurst.eu.org  Wed Apr 22 14:20:28 2009
From: andyjpb at ashurst.eu.org (Andy Bennett)
Date: Wed, 22 Apr 2009 19:20:28 +0100
Subject: Delivery to Shared Folders via authenticated SMTP then LMTP
In-Reply-To: <49EF151B.7000002@SiriusIT.co.uk>
References: <49ED9C42.60205@ashurst.eu.org> <49EDDE70.8050707@SiriusIT.co.uk>
	<49EDF41C.5070506@ashurst.eu.org> <49EF151B.7000002@SiriusIT.co.uk>
Message-ID: <49EF5FEC.1090509@ashurst.eu.org>

Hi,

I've managed to get things working now.


>> 2009-04-21 15:10:27 H=mx2.andrew.cmu.edu [128.2.11.36] 
>> F=<info-cyrus-bounces+andyjpb=ashurst.eu.org at lists.andrew.cmu.edu> 
>> temporarily rejected RCPT <andyjpb at ashurst.eu.org>: response to "MAIL 
>> FROM:<>" from localhost [127.0.0.1] was: 430 Authentication require
> 
> AB> Your reply went to the list and directly to me: the direct one
> AB> came through but the one from mailman got stuck between my smtp
> AB> and lmtp servers and was therefore temporarily rejected.
> 
> That sounds like an Exim problem, and hence might be better discussed on
> an Exim list.  I've not seem what you've done with your shared mailbox
> delivery rules, but is it possible the + characters in mailman
> envelope-from addresses are confusing it?

You were right: it was an exim problem. When I switched from preauth'd 
LMTP to LMTPA I couldn't do callforwards from exim to see if the 
recipient mailbox existed. This is because exim doesn't bother with the 
auth during the callforward and was therefore getting the "430 
Authentication required" messages.
I think it was working when I sent my test mails it because exim can 
cache the results of a callforward and I was testing from a limited 
number of accounts.


> AB> For now, I've gone back to using lmtp in "lmtp -a" mode and it
> AB> seems to have fixed things... Hopefully all the temporarily
> AB> rejected mail will start to come through in the next few hours.
> 
> IMHO "lmtpd -a" is the work of the devil ;-)  Not to be used unless the
> MTAs and the front-end Cyruses are on a secure network segment, you
> really really trust your MTAs and don't have auto-* enabled...

I agree and that's why I was trying to move to an authenticated lmtp set up.


> AB> Why do different things happen when running "lmtp -a" compared
> AB> to "lmtp" and logging in as an lmtp_admin?
> 
> The only difference between "lmtpd" and "lmtpd -a" should be that the
> latter fakes a SASL external auth for the default lmtp admin, "postman".
> 
> When you say "logging in", I presume you're talking about an IMAP
> session.  IMAP and LMTP necessarily present different views of the
> mailbox namespace: in the IMAP case the view you get is the authzid's
> view; in the LMTP case the view of the namespace we're interested in is
> normally the recipient's view.

No, I was talking about the "cyrus/lmtp: login: <host> [<ip>] exim 
CRAM-MD5 User logged in" messages.


> AB> I'm not really clear why it is sometimes failing and sometimes
> AB> succeeding in the non shared folders case.
> 
> I think you need to find some examples of working and not working, and
> log exactly what Exim is saying to lmtpd.  Then you can figure out
> whether Exim's correct and lmtpd is broken, or lmtpd is doing the Right
> Thing in response to something weird coming in from Exim.

Thanks. As explained above, exim was doing callforwards without 
authentication and so mail was being deferred at that stage.



 > You should have lines in syslog (/var/log/maillog) from lmtpd of the form
 >
 >   cyrus/lmtp[<PID>]: login: <MTA.HOSTNAME> [<MTA.IP>] <authzid>
 >       <SASL.MECH> User logged in
 >
 > The authzid there will be the user as whom Exim authorized.  But I don't
 > think that's the problem (see below).
 >
 > AB>    client_send = $authenticated_sender^exim^<PASSWORD>
 >
 > AB> I think that should send the exim authenticated sender along
 > AB> as the authorisation and exim and <PASSWORD> along as the
 > AB> authentication.
 >
 > It should, but not in the way you want.  The SASL authzid isn't what
 > lmtpd evaluates ACLs against.  To do what I think you want (ACLs for
 > delivery to shared mailboxes by users employing SMTPA), you need Exim to
 > pass the authenticated user from the SMTP transaction with the MUA into
 > the _MAIL_ line of the LMTP conversation.  You want Exim to say:
 >
 >  MAIL FROM:<andyjpb at ashurst.eu.org> AUTH=<andyjpb at ashurst.eu.org>
 >
 > To do that you probably want to add
 >
 >   authenticated_sender = $authenticated_id
 >
 > to the definition of your lmtp relay.

I'm getting <authzid> = exim even when I'm setting authenticated_sender 
= $authenicated_id so it looks like it's putting the authentication ID 
there, not the authorisation id.

Anyway, it's now respecting the ACLs of the user that authenticated to 
SMTP so I'm happy.

Setting authenticated_sender = $authenticated_id doesn't seem to do any 
harm when the SMTP connection is not authenticated so it works for 
external mail to regular inboxes as well.


I've had to ditch the callforwards but I didn't want to blindly accept 
mail for my domains and then have to bounce undeliverable mail later so 
I'm now generating lists of INBOXes that I can check against during the 
"RCPT TO" phase. I'll probably need to turn these into maps if I add 
another cyrus server so that I know where to route mail for different 
local parts.




Many thanks for all your help.







Regards,
@ndy

-- 
andyjpb at ashurst.eu.org
http://www.ashurst.eu.org/
http://www.gonumber.com/andyjpb
0x7EBA75FF


From ana.ribas at upcnet.es  Thu Apr 23 06:06:36 2009
From: ana.ribas at upcnet.es (Ana Ribas Roca)
Date: Thu, 23 Apr 2009 12:06:36 +0200
Subject: much information from cyrus log
In-Reply-To: <03a0e812a16045234a721822063fb88e.squirrel@a-angels.ath.cx>
References: <20090422122031.203819yybofayjxr@vader.upc.es>
	<03a0e812a16045234a721822063fb88e.squirrel@a-angels.ath.cx>
Message-ID: <20090423120636.59506sl5638frk4s@vader.upc.es>


We need the information about which ip is connecting to which mailbox,  
and what is this user doing: opening, deleting, authenticating, ...

Thanks in advance

- ANNA -

Quoting "Andreas Winkelmann" <ml at awinkelmann.de>:

>> I'm trying to get as much information as possible from the cyrus log.
>> I've tried several modification of the syslog configuration
>> filewithout success.
>>
>> I also?create a folder at /var/log/.. per user getting as much
>> information I want but... it's in different files and folders, and
>> is?per user based solution, which is difficult to administrate.
>>
>> Any clue on how to configure cyrus and syslog to retrieve all this info?
>
> Maybe you should go the other way around. Tell what information you need.
>
> Cyrus sends alot of information with LOG_DEBUG to syslog, check if you
> catch these Messages. The directories you mentioned are telemetry Logs,
> these Dirs are in $configdirectory, which is normally not in /var/log/...
>
> --
> Andreas
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>


From imaplist at unixontap.com  Thu Apr 23 12:23:46 2009
From: imaplist at unixontap.com (Colin Jaccino)
Date: Thu, 23 Apr 2009 12:23:46 -0400 (EDT)
Subject: No subject
Message-ID: <65487.24.248.74.254.1240503826.squirrel@fishbowl.bitwheel.net>

Greets!
  I am researching lemonade compliance and what email servers may be able
to support key extensions, especially RFC 5465 - IMAP NOTIFY.  As it
stands, no support appears to be available from any vendor.  Cyrus 2.4
looks like it will support much of the new lemonade functionality, but
little information has been made available.  Can anyone shed light on
when we might expect a 2.4 or lemonade-oriented release and what might
be included?

Thanks!

Colin Jaccino


From champ at umbc.edu  Thu Apr 23 12:52:55 2009
From: champ at umbc.edu (Tim Champ)
Date: Thu, 23 Apr 2009 12:52:55 -0400
Subject: Using the quota command question
Message-ID: <49F09CE7.9000804@umbc.edu>

Hello all.

Quick (hopefully) question - the man page for "quota" says it isn't 
recommended to do a "-f" when specifying a user.  Due to some issues 
that would take a while to explain, we will need to fix quite a few 
quotas on users as we "xfer" them to a new system.

Is this an outdated statement in the man page?  If not, what is the 
risk?  I'd hate to have to run "quota -f" across a server of many 
thousand users just to fix one.

Thanks for any help!
Tim Champ
UMBC DoIT Unix Infrastructure Team

From champ at umbc.edu  Thu Apr 23 12:53:54 2009
From: champ at umbc.edu (Tim Champ)
Date: Thu, 23 Apr 2009 12:53:54 -0400
Subject: Using the quota command question
In-Reply-To: <49F09CE7.9000804@umbc.edu>
References: <49F09CE7.9000804@umbc.edu>
Message-ID: <49F09D22.9020605@umbc.edu>

Tim Champ wrote:
> Hello all.
>
> Quick (hopefully) question - the man page for "quota" says it isn't 
> recommended to do a "-f" when specifying a user.  Due to some issues 
> that would take a while to explain, we will need to fix quite a few 
> quotas on users as we "xfer" them to a new system.
>
> Is this an outdated statement in the man page?  If not, what is the 
> risk?  I'd hate to have to run "quota -f" across a server of many 
> thousand users just to fix one.
>
> Thanks for any help!
> Tim Champ
> UMBC DoIT Unix Infrastructure Team
>

Sigh -- I forgot to add our version - 2.3.8.  Sorry about that, and the 
resulting double mail.  The version of software is the same on the 
to/from machines.

If you have any questions, I'm happy to answer them.  Thanks!

Tim

From morgan at orst.edu  Thu Apr 23 13:29:01 2009
From: morgan at orst.edu (Andrew Morgan)
Date: Thu, 23 Apr 2009 10:29:01 -0700 (PDT)
Subject: Using the quota command question
In-Reply-To: <49F09CE7.9000804@umbc.edu>
References: <49F09CE7.9000804@umbc.edu>
Message-ID: <alpine.DEB.2.00.0904231027490.9160@shell.onid.oregonstate.edu>

On Thu, 23 Apr 2009, Tim Champ wrote:

> Hello all.
>
> Quick (hopefully) question - the man page for "quota" says it isn't 
> recommended to do a "-f" when specifying a user.  Due to some issues 
> that would take a while to explain, we will need to fix quite a few 
> quotas on users as we "xfer" them to a new system.
>
> Is this an outdated statement in the man page?  If not, what is the 
> risk?  I'd hate to have to run "quota -f" across a server of many 
> thousand users just to fix one.

I use "quota -f user.foo" all the time to fix quotes on mailboxes restored 
from backups.  I've never had any errors or problems, so I don't know why 
the manpage gives that recommendation.

 	Andy

From bsh at freemail.hu  Thu Apr 23 13:46:42 2009
From: bsh at freemail.hu (=?ISO-8859-2?Q?K=F5v=E1ri_J=E1nos?=)
Date: Thu, 23 Apr 2009 19:46:42 +0200
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
Message-ID: <49F0A982.6070906@freemail.hu>

An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090423/97313a0b/attachment.html 

From mills at cc.umanitoba.ca  Thu Apr 23 14:57:20 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Thu, 23 Apr 2009 13:57:20 -0500
Subject: Multiple IMAP connections from new IMAP clients
Message-ID: <20090423185720.GA9270@cc.umanitoba.ca>

We've had a problem recently with the number of imapd processes on our
Cyrus front-end increasing steadily until it filled the process table.
It seems that some recent IMAP clients will normally open a number of
IMAP connections to their server, and will open more based on user
activity.  Each of these causes a new imapd process to be spawned on
the front-end.  As far as I know, the server treats each connection
independantly, even though the client may consider one to be permanent
and the others to be transient.

What are people doing to protect their Cyrus servers from this
increasing number of connections, each of which consumes resources on
the server?  This problem is going to get worse as more sophisticated
clients become popular.  Is many small front-ends the solution?

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From dwhite at olp.net  Thu Apr 23 15:21:08 2009
From: dwhite at olp.net (Dan White)
Date: Thu, 23 Apr 2009 14:21:08 -0500
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <49F0A982.6070906@freemail.hu>
References: <49F0A982.6070906@freemail.hu>
Message-ID: <49F0BFA4.9060509@olp.net>

K?v?ri J?nos wrote:
> I have a postfix relay server and a (local) cyrus imap server on the 
> same machine. Everything was fine until I thought, I change the imap 
> authentication from sasldb to saslauth, to have global authentication 
> on postfix and cyrus.
> Postfix uses saslauthd, which is configured for PAM. It works 
> perfectly, with plain/login/cram/digest mechanisms, with or without 
> tls/ssl, absolutely no problems with it. Saslauth tests are all fine 
> obviously.
> So I decided to use this with cyrus imap too. Set it to use the same 
> saslauth daemon, and plain, login, cram-md5 and digest-md5 mechs.
> Since then, I can not login with plain or login mechs, because they 
> aren't being offered at all by cyrus imapd. I can login with cram or 
> digest fine.
> I understand that plain login isn't offered by default, only after a 
> successfull tls session setup, but if I understand correctly, the 
> "allowplaintext: yes" option should still force imapd to offer plain 
> logins. But it doesn't. I tried it with different sasl_min|max_levels, 
> to no avail.
> This is the first thing I don't understand.
> The second is: after establishing a tls or ssl connection, plain and 
> login are offered, but I can not login with these mechs.
> (I'm using imtest to test it all.)
> However, with "testsaslauth", I am able to authenticate fine.
>
> I'm quite new to cyrus and linux systems, but I read all kinds of 
> manuals and FAQs nd documentation, and googled a lot, but I was unable 
> to find the culprit. So you are my last hope.
> If nothing else works, I leave it as is, with digest and cram it works 
> and it's more secure. Or I go back to sasldb, which is less 
> comfortable for me...

Please include the following information, so we can get a better idea of 
your setup:

Postfix and Cyrus IMAP version
Postfix SASL config:
  grep sasl main.cf
  cat /etc/postfix/sasl/smtpd.conf (or wherever smtpd.conf it located on 
your system)

Your cyrus imap.conf config

saslauthd does not support cram-md5 or digest-md5, so you may be (also) 
using the sasldb auxprop in Postfix.

- Dan

From nic at onlight.com  Thu Apr 23 15:23:10 2009
From: nic at onlight.com (Nic Bernstein)
Date: Thu, 23 Apr 2009 14:23:10 -0500
Subject: Multiple IMAP connections from new IMAP clients
In-Reply-To: <20090423185720.GA9270@cc.umanitoba.ca>
References: <20090423185720.GA9270@cc.umanitoba.ca>
Message-ID: <49F0C01E.6090708@onlight.com>

On 04/23/2009 01:57 PM, Gary Mills wrote:
> We've had a problem recently with the number of imapd processes on our
> Cyrus front-end increasing steadily until it filled the process table.
> It seems that some recent IMAP clients will normally open a number of
> IMAP connections to their server, and will open more based on user
> activity.  Each of these causes a new imapd process to be spawned on
> the front-end.  As far as I know, the server treats each connection
> independantly, even though the client may consider one to be permanent
> and the others to be transient.
>
> What are people doing to protect their Cyrus servers from this
> increasing number of connections, each of which consumes resources on
> the server?  This problem is going to get worse as more sophisticated
> clients become popular.  Is many small front-ends the solution?
>
>   
We've been using imapproxyd to help solve just this kind of problem.  
Haven't used it with a murder, but expect it could still be useful.

Cheers,
    -nic

-- 
Nic Bernstein                             nic at onlight.com
Onlight llc.                              www.onlight.com
2266 North Prospect Avenue #610	          v. 414.272.4477
Milwaukee, Wisconsin  53202-6306          f. 414.290.0335


From mills at cc.umanitoba.ca  Thu Apr 23 20:42:59 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Thu, 23 Apr 2009 19:42:59 -0500
Subject: Multiple IMAP connections from new IMAP clients
In-Reply-To: <49F0C01E.6090708@onlight.com>
References: <20090423185720.GA9270@cc.umanitoba.ca>
	<49F0C01E.6090708@onlight.com>
Message-ID: <20090424004259.GA13385@cc.umanitoba.ca>

On Thu, Apr 23, 2009 at 02:23:10PM -0500, Nic Bernstein wrote:
> On 04/23/2009 01:57 PM, Gary Mills wrote:
> >We've had a problem recently with the number of imapd processes on our
> >Cyrus front-end increasing steadily until it filled the process table.
> >It seems that some recent IMAP clients will normally open a number of
> >IMAP connections to their server, and will open more based on user
> >activity.  Each of these causes a new imapd process to be spawned on
> >the front-end.  As far as I know, the server treats each connection
> >independantly, even though the client may consider one to be permanent
> >and the others to be transient.
> >
> >What are people doing to protect their Cyrus servers from this
> >increasing number of connections, each of which consumes resources on
> >the server?  This problem is going to get worse as more sophisticated
> >clients become popular.  Is many small front-ends the solution?
> >  
> We've been using imapproxyd to help solve just this kind of problem.  
> Haven't used it with a murder, but expect it could still be useful.

Does it actually combine separate connections from a single client
into one connection to the server?  I don't know how it could do that
without violating the protocol.

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From bsh at freemail.hu  Fri Apr 24 02:52:21 2009
From: bsh at freemail.hu (=?ISO-8859-2?Q?K=F5v=E1ri_J=E1nos?=)
Date: Fri, 24 Apr 2009 08:52:21 +0200
Subject: 
In-Reply-To: <49F15D08.9090503@freemail.hu>
References: <49F15D08.9090503@freemail.hu>
Message-ID: <49F161A5.4050708@freemail.hu>

An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090424/62b90380/attachment.html 

From simon.matter at invoca.ch  Fri Apr 24 03:42:37 2009
From: simon.matter at invoca.ch (Simon Matter)
Date: Fri, 24 Apr 2009 09:42:37 +0200 (CEST)
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <49F0A982.6070906@freemail.hu>
References: <49F0A982.6070906@freemail.hu>
Message-ID: <97899ea9217e4ce4f8611fc18bcfd5b0.squirrel@webmail.bi.corp.invoca.ch>

> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
> <html>
> <head>
> </head>
> <body bgcolor="#ffffff" text="#000000">
> <font size="-1"><font face="Arial">Hello everyone!<br>
> <br>
> I'm new to this mailing list, actually, this is the first mailing list
> I've ever subscribed. :) So greetings to all from Hungary! (And excuse
> my really bad english, please)<br>

Hi,

allow me to give you two suggestions first:

1) Please configure your mailer to send mail in clear text, not html.
Otherwise configure it to send woth, text and html. Html only mails may
have problems for some users to get read and some people are annoyed by
html mails.

2)
Always use the "reply" or "reply all" function of your mailer when
replying to the list - and don't change the Subject of the mail. That way
people can follow the thread of the discussion.

> <br>
> I'm not sure if I can ask for help here, but I didn't find any answer
> elsewhere, so trying this out.<br>
> <br>
> I have a postfix relay server and a (local) cyrus imap server on the
> same machine. Everything was fine until I thought, I change the imap
> authentication from sasldb to saslauth, to have global authentication
> on postfix and cyrus.<br>
> Postfix uses saslauthd, which is configured for PAM. It works
> perfectly, with plain/login/cram/digest mechanisms, with or without
> tls/ssl, absolutely no problems with it. Saslauth tests are all fine
> obviously.<br>
> So I decided to use this with cyrus imap too. Set it to use the same
> saslauth daemon, and plain, login, cram-md5 and digest-md5 mechs.<br>
> Since then, I can not login with plain or login mechs, because they
> aren't being offered at all by cyrus imapd. I can login with cram or
> digest fine.<br>
> I understand that plain login isn't offered by default, only after a
> successfull tls session setup, but if I understand correctly, the
> "allowplaintext: yes" option should still force imapd to offer plain
> logins. But it doesn't. I tried it with different sasl_min|max_levels,
> to no avail.<br>

"allowplaintext: 1" should indeed enable plain. At least that works well
for me. I expect you are using the packages from a distribution, maybe
they have added some kind of "security" patches to make things more safe?
Could you try with the following line in your cyrus config:

sasl_mech_list: PLAIN

Regards,
Simon


From bsh at freemail.hu  Fri Apr 24 04:14:51 2009
From: bsh at freemail.hu (=?UTF-8?B?S8WRdsOhcmkgSsOhbm9z?=)
Date: Fri, 24 Apr 2009 10:14:51 +0200
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <97899ea9217e4ce4f8611fc18bcfd5b0.squirrel@webmail.bi.corp.invoca.ch>
References: <49F0A982.6070906@freemail.hu>
	<97899ea9217e4ce4f8611fc18bcfd5b0.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <49F174FB.6090909@freemail.hu>

Simon Matter ?rta:
>> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
>> <html>
>> <head>
>> </head>
>> <body bgcolor="#ffffff" text="#000000">
>> <font size="-1"><font face="Arial">Hello everyone!<br>
>> <br>
>> I'm new to this mailing list, actually, this is the first mailing list
>> I've ever subscribed. :) So greetings to all from Hungary! (And excuse
>> my really bad english, please)<br>
>>     
>
> Hi,
>
> allow me to give you two suggestions first:
>
> 1) Please configure your mailer to send mail in clear text, not html.
> Otherwise configure it to send woth, text and html. Html only mails may
> have problems for some users to get read and some people are annoyed by
> html mails.
>
> 2)
> Always use the "reply" or "reply all" function of your mailer when
> replying to the list - and don't change the Subject of the mail. That way
> people can follow the thread of the discussion.
>
>   
Thanks, will do!
>> <br>
>> I'm not sure if I can ask for help here, but I didn't find any answer
>> elsewhere, so trying this out.<br>
>> <br>
>> I have a postfix relay server and a (local) cyrus imap server on the
>> same machine. Everything was fine until I thought, I change the imap
>> authentication from sasldb to saslauth, to have global authentication
>> on postfix and cyrus.<br>
>> Postfix uses saslauthd, which is configured for PAM. It works
>> perfectly, with plain/login/cram/digest mechanisms, with or without
>> tls/ssl, absolutely no problems with it. Saslauth tests are all fine
>> obviously.<br>
>> So I decided to use this with cyrus imap too. Set it to use the same
>> saslauth daemon, and plain, login, cram-md5 and digest-md5 mechs.<br>
>> Since then, I can not login with plain or login mechs, because they
>> aren't being offered at all by cyrus imapd. I can login with cram or
>> digest fine.<br>
>> I understand that plain login isn't offered by default, only after a
>> successfull tls session setup, but if I understand correctly, the
>> "allowplaintext: yes" option should still force imapd to offer plain
>> logins. But it doesn't. I tried it with different sasl_min|max_levels,
>> to no avail.<br>
>>     
>
> "allowplaintext: 1" should indeed enable plain. At least that works well
> for me. I expect you are using the packages from a distribution, maybe
> they have added some kind of "security" patches to make things more safe?
> Could you try with the following line in your cyrus config:
>
> sasl_mech_list: PLAIN
>
> Regards,
> Simon
>
>   
yes, the server is running ubuntu 7.04 i386, 2.6.20-17-generic, and 
postfix and cyrus are installed via the ubuntu repositiories.

ok, first, this is what I get with sasl_mech_list=plain login cram-md5 
digest-md5:

imtest localhost
S: * OK some-server Cyrus IMAP4 v2.2.13-Debian-2.2.13-10ubuntu2 server ready
C: C01 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS 
NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND 
BINARY SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE IDLE 
STARTTLS AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR
S: C01 OK Completed
C: A01 AUTHENTICATE DIGEST-MD5
S: + bm9uY2U9IjNpZnh<snip>09bWQ1LXNlc3M=
Please enter your password:
C: dXNlcm5hbWU9<snip>ZDg5YzA0ZGYDE0YmI5YjQ=
S: + cnNwYXV0aD<snip>RjYmQ5N2JjOA==
C:
S: A01 OK Success (privacy protection)
Authenticated.
Security strength factor: 128
. logout
* BYE LOGOUT received
. OK Completed
Connection closed.

syslog says:

Apr 24 09:56:27 localhost cyrus/imap[7030]: login: localhost [127.0.0.1] user DIGEST-MD5 User logged in



and this is with only PLAIN mech:

imtest localhost
S: * OK piller-server Cyrus IMAP4 v2.2.13-Debian-2.2.13-10ubuntu2 server 
ready
C: C01 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS 
NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND 
BINARY SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE IDLE 
STARTTLS
S: C01 OK Completed
Please enter your password:
C: L01 LOGIN user {7}
S: + go ahead
C: <omitted>
S: L01 NO Login failed: generic failure
Authentication failed. generic failure
Security strength factor: 0
C: Q01 LOGOUT
Connection closed.

Apr 24 10:02:25 localhost cyrus/imap[7147]: badlogin: localhost [127.0.0.1] plaintext user SASL(-1): generic failure: checkpass failed



From simon.matter at invoca.ch  Fri Apr 24 04:28:52 2009
From: simon.matter at invoca.ch (Simon Matter)
Date: Fri, 24 Apr 2009 10:28:52 +0200 (CEST)
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <49F174FB.6090909@freemail.hu>
References: <49F0A982.6070906@freemail.hu>
	<97899ea9217e4ce4f8611fc18bcfd5b0.squirrel@webmail.bi.corp.invoca.ch>
	<49F174FB.6090909@freemail.hu>
Message-ID: <b71d8a8f8fe4fc366af265cfd57b4c58.squirrel@webmail.bi.corp.invoca.ch>

>>> I have a postfix relay server and a (local) cyrus imap server on the
>>> same machine. Everything was fine until I thought, I change the imap
>>> authentication from sasldb to saslauth, to have global authentication
>>> on postfix and cyrus.<br>
>>> Postfix uses saslauthd, which is configured for PAM. It works
>>> perfectly, with plain/login/cram/digest mechanisms, with or without
>>> tls/ssl, absolutely no problems with it. Saslauth tests are all fine
>>> obviously.<br>
>>> So I decided to use this with cyrus imap too. Set it to use the same
>>> saslauth daemon, and plain, login, cram-md5 and digest-md5 mechs.<br>
>>> Since then, I can not login with plain or login mechs, because they
>>> aren't being offered at all by cyrus imapd. I can login with cram or
>>> digest fine.<br>
>>> I understand that plain login isn't offered by default, only after a
>>> successfull tls session setup, but if I understand correctly, the
>>> "allowplaintext: yes" option should still force imapd to offer plain
>>> logins. But it doesn't. I tried it with different sasl_min|max_levels,
>>> to no avail.<br>
>>>
>>
>> "allowplaintext: 1" should indeed enable plain. At least that works well
>> for me. I expect you are using the packages from a distribution, maybe
>> they have added some kind of "security" patches to make things more
>> safe?
>> Could you try with the following line in your cyrus config:
>>
>> sasl_mech_list: PLAIN
>>
>> Regards,
>> Simon
>>
>>
> yes, the server is running ubuntu 7.04 i386, 2.6.20-17-generic, and
> postfix and cyrus are installed via the ubuntu repositiories.

Can you check which cyrus-sasl-* packages you have installed? Most
distributions split cyrus?-sasl into multiple packages and maybe you have
not installed the -plain package?

Simon


From bsh at freemail.hu  Fri Apr 24 05:29:29 2009
From: bsh at freemail.hu (=?UTF-8?B?S8WRdsOhcmkgSsOhbm9z?=)
Date: Fri, 24 Apr 2009 11:29:29 +0200
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <b71d8a8f8fe4fc366af265cfd57b4c58.squirrel@webmail.bi.corp.invoca.ch>
References: <49F0A982.6070906@freemail.hu>
	<97899ea9217e4ce4f8611fc18bcfd5b0.squirrel@webmail.bi.corp.invoca.ch>
	<49F174FB.6090909@freemail.hu>
	<b71d8a8f8fe4fc366af265cfd57b4c58.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <49F18679.3020107@freemail.hu>



Simon Matter ?rta:
>>>> I have a postfix relay server and a (local) cyrus imap server on the
>>>> same machine. Everything was fine until I thought, I change the imap
>>>> authentication from sasldb to saslauth, to have global authentication
>>>> on postfix and cyrus.<br>
>>>> Postfix uses saslauthd, which is configured for PAM. It works
>>>> perfectly, with plain/login/cram/digest mechanisms, with or without
>>>> tls/ssl, absolutely no problems with it. Saslauth tests are all fine
>>>> obviously.<br>
>>>> So I decided to use this with cyrus imap too. Set it to use the same
>>>> saslauth daemon, and plain, login, cram-md5 and digest-md5 mechs.<br>
>>>> Since then, I can not login with plain or login mechs, because they
>>>> aren't being offered at all by cyrus imapd. I can login with cram or
>>>> digest fine.<br>
>>>> I understand that plain login isn't offered by default, only after a
>>>> successfull tls session setup, but if I understand correctly, the
>>>> "allowplaintext: yes" option should still force imapd to offer plain
>>>> logins. But it doesn't. I tried it with different sasl_min|max_levels,
>>>> to no avail.<br>
>>>>
>>>>         
>>> "allowplaintext: 1" should indeed enable plain. At least that works well
>>> for me. I expect you are using the packages from a distribution, maybe
>>> they have added some kind of "security" patches to make things more
>>> safe?
>>> Could you try with the following line in your cyrus config:
>>>
>>> sasl_mech_list: PLAIN
>>>
>>> Regards,
>>> Simon
>>>
>>>
>>>       
>> yes, the server is running ubuntu 7.04 i386, 2.6.20-17-generic, and
>> postfix and cyrus are installed via the ubuntu repositiories.
>>     
>
> Can you check which cyrus-sasl-* packages you have installed? Most
> distributions split cyrus?-sasl into multiple packages and maybe you have
> not installed the -plain package?
>
> Simon
>
>   

I have these installed:
cyrus-admin-2.2 (2.2.13-10ubuntu2), cyrus-clients-2.2 
(2.2.13-10ubuntu2), cyrus-common-2.2 (2.2.13-10ubuntu2), cyrus-imapd-2.2 
(2.2.13-10ubuntu2), cyrus-murder-2.2 (2.2.13-10ubuntu2), 
libauthen-sasl-cyrus-perl (0.13-server-1), libauthen-sasl-perl (2.10-1), 
libcyrus-imap-perl22 (2.2.13-10ubuntu2), libsasl2-2 
(2.1.22.dsfg1-8ubuntu2), libsasl2-modules (2.1.22.dfsg1-8ubuntu2), 
sasl2-bin (2.1.22.dfsg1-8ubuntu2)

And these AREN'T installed:
libsasl2-modules-gssapi-heimdal, libsasl2-modules-gssapi-mit, 
libsasl2-modules-ldap, libsasl2-modules-otp, libsasl2-modules-sql.

Can't seem to find separate -plain packages or anything similar.

Postfix shows this, when in smtpd.conf the mech_list is set to PLAIN only:

Apr 24 11:13:56 localhost postfix/smtpd[8026]: connect from client4[192.168.2.126]
Apr 24 11:13:56 localhost postfix/smtpd[8026]: 4C4319CDF8: client=client4[192.168.2.126], sasl_method=PLAIN, sasl_username=user at piller-server

when it's set to LOGIN only:

Apr 24 11:16:42 localhost postfix/smtpd[8178]: connect from client4[192.168.2.126]
Apr 24 11:16:42 localhost postfix/smtpd[8178]: 839B69CDF8: client=client4[192.168.2.126], sasl_method=LOGIN, sasl_username=user at piller-server

with CRAM-MD5 only:

Apr 24 11:18:24 localhost postfix/smtpd[8299]: connect from client4[192.168.2.126]
Apr 24 11:18:24 localhost postfix/smtpd[8299]: 8164B9CDF8: client=client4[192.168.2.126], sasl_method=CRAM-MD5, sasl_username=user at piller-server


Janos

From Eric.Luyten at vub.ac.be  Fri Apr 24 09:28:49 2009
From: Eric.Luyten at vub.ac.be (Eric Luyten)
Date: Fri, 24 Apr 2009 15:28:49 +0200 (CEST)
Subject: How to eliminate writing the duplicate delivery database ?
Message-ID: <55936.134.184.15.103.1240579729.squirrel@nuts.vub.ac.be>

A couple weeks ago, in order to accommodate local users (not) receiving
mail messages written by some Outlook/Exchange correspondants (Message-Id
gets re-used...) I disabled Cyrus duplicate suppression by adding the
following line to my imapd.conf :

   duplicatesuppression: 0


Somewhat naively I thought that Cyrus wouldn't be writing the duplicate
delivery database anymore, but this is not the case.

Is there, other than by patching the C code, a way to disable writing
delivery.db ?

This is a Cyrus 2.2 system, upgrading to 2.3 this summer.


Regards,
Eric Luyten, Computing Centre VUB/ULB.


From dwhite at olp.net  Fri Apr 24 10:07:28 2009
From: dwhite at olp.net (Dan White)
Date: Fri, 24 Apr 2009 09:07:28 -0500
Subject: 
In-Reply-To: <49F161A5.4050708@freemail.hu>
References: <49F15D08.9090503@freemail.hu> <49F161A5.4050708@freemail.hu>
Message-ID: <49F1C7A0.6030905@olp.net>

K?v?ri J?nos wrote:
> K?v?ri J?nos wrote:
>> >/ Postfix uses saslauthd, which is configured for PAM. It works 
>> />/ perfectly, with plain/login/cram/digest mechanisms, with or without 
>> />/ tls/ssl, absolutely no problems with it. Saslauth tests are all fine 
>> />/ obviously.
>> />/ So I decided to use this with cyrus imap too. Set it to use the same 
>> />/ saslauth daemon, and plain, login, cram-md5 and digest-md5 mechs.
>> />/ Since then, I can not login with plain or login mechs, because they 
>> />/ aren't being offered at all by cyrus imapd. I can login with cram or 
>> />/ digest fine.
>> />/ I understand that plain login isn't offered by default, only after a 
>> />/ successfull tls session setup, but if I understand correctly, the 
>> />/ "allowplaintext: yes" option should still force imapd to offer plain 
>> />/ logins. But it doesn't. I tried it with different sasl_min|max_levels, 
>> />/ to no avail.
>> /
>> Please include the following information, so we can get a better idea of 
>> your setup:
>>
>> Postfix and Cyrus IMAP version
>> Postfix SASL config:
>>   grep sasl main.cf
>>   cat /etc/postfix/sasl/smtpd.conf (or wherever smtpd.conf it located on 
>> your system)
>>
>>
>>     
> Hello Dan,
>
> Postfix version: 2.5.4
> Cyrus IMAP version: 2.2.13
>
> smtpd_sasl_auth_enable = yes
>
> /cat /etc/postfix/sasl/smtpd.conf/
> saslauthd_version: 2
> pwcheck_method: saslauthd
> mech_list: plain login cram-md5 digest-md5
>
> /cat /etc/imapd.conf/
> allowplaintext: yes
> saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux
> sasl_pwcheck_method: saslauthd
> sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5
> sasl_auto_transition: no
>
> /cat /etc/default/saslauthd/
> START=yes
> PWDIR="/var/spool/postfix/var/run/saslauthd"
> PARAMS="-m ${PWDIR}"
> PIDFILE="${PWDIR}/saslauthd.pid"
> MECHANISMS="pam"
> MECH_OPTIONS=""
> THREADS=5
> OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd"
> /#(I think the options line is wrong, the -m part is unneded, but it 
> was like that, and it works...)/


The way that you have postfix configured, it will use saslauthd (only) 
for plain and login. It (via SASL) will use your auxprop store to 
authenticate the cram-md5 and digest-md5 mechanisms. I'm assuming that 
you have configured your users in /etc/sasldb2, since you are 
authenticating to imapd via digest-md5.

'allowplaintext: yes' should be all you need to support plain/login on 
an in-the-clear connection. Since they are being offered after a TLS 
connection, it's almost if there's a typo in your config for that command.

also:

saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux

is a typo in /etc/imapd.conf. It should be:

sasl_saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux

For trouble shooting, you might want to comment out 
'sasl_pwcheck_method: saslauthd', which will direct imapd to use all 
available pw_check methods (including auxprop) for plain/login.

- Dan

From michael.menge at zdv.uni-tuebingen.de  Fri Apr 24 10:17:32 2009
From: michael.menge at zdv.uni-tuebingen.de (Michael Menge)
Date: Fri, 24 Apr 2009 16:17:32 +0200
Subject: How to eliminate writing the duplicate delivery database ?
In-Reply-To: <55936.134.184.15.103.1240579729.squirrel@nuts.vub.ac.be>
References: <55936.134.184.15.103.1240579729.squirrel@nuts.vub.ac.be>
Message-ID: <20090424161732.57977coczttwpo1o@webmail.uni-tuebingen.de>

Quoting Eric Luyten <Eric.Luyten at vub.ac.be>:

> A couple weeks ago, in order to accommodate local users (not) receiving
> mail messages written by some Outlook/Exchange correspondants (Message-Id
> gets re-used...) I disabled Cyrus duplicate suppression by adding the
> following line to my imapd.conf :
>
>    duplicatesuppression: 0
>
>
> Somewhat naively I thought that Cyrus wouldn't be writing the duplicate
> delivery database anymore, but this is not the case.

Cyrus still needs to write to the delivery database, e.g. for vacation
messages. But it will deliver messages even if they have the same
message-id.


>
> Is there, other than by patching the C code, a way to disable writing
> delivery.db ?

Why do you want do disable writing to the delivery.db?
>
> This is a Cyrus 2.2 system, upgrading to 2.3 this summer.
>
As far as i know it is the same with 2.3.x

--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universit?t T?bingen                   Fax.: (49) 7071/29-5912
Zentrum f?r Datenverarbeitung          mail:  
michael.menge at zdv.uni-tuebingen.de
W?chterstra?e 76
72074 T?bingen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5339 bytes
Desc: S/MIME krytographische Unterschrift
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090424/c366190a/attachment.bin 

From Eric.Luyten at vub.ac.be  Fri Apr 24 10:39:46 2009
From: Eric.Luyten at vub.ac.be (Eric Luyten)
Date: Fri, 24 Apr 2009 16:39:46 +0200 (CEST)
Subject: How to eliminate writing the duplicate delivery database ?
In-Reply-To: <20090424161732.57977coczttwpo1o@webmail.uni-tuebingen.de>
References: <55936.134.184.15.103.1240579729.squirrel@nuts.vub.ac.be>
	<20090424161732.57977coczttwpo1o@webmail.uni-tuebingen.de>
Message-ID: <56383.134.184.15.103.1240583986.squirrel@nuts.vub.ac.be>

On Fri, April 24, 2009 4:17 pm, Michael Menge wrote:
> Quoting Eric Luyten <Eric.Luyten at vub.ac.be>:
>
>
>> A couple weeks ago, in order to accommodate local users (not) receiving
>>  mail messages written by some Outlook/Exchange correspondants
>> (Message-Id
>> gets re-used...) I disabled Cyrus duplicate suppression by adding the
>> following line to my imapd.conf :
>>
>> duplicatesuppression: 0
>>
>>
>>
>> Somewhat naively I thought that Cyrus wouldn't be writing the duplicate
>>  delivery database anymore, but this is not the case.
>
> Cyrus still needs to write to the delivery database, e.g. for vacation
> messages.

Michael,

Thank you for bringing this to my attention !
(we're not using Sieve's vacation function, but that's another issue)


> But it will deliver messages even if they have the same message-id.

Correct. My configuration change had the desired effect.



>> Is there, other than by patching the C code, a way to
>> disabling writing delivery.db ?
>
> Why do you want do disable writing to the delivery.db?

It seemed like a lot of DB updates we could get rid of.
We're still with a Cyrus monolith (a single mailboxes.db
and deliver.db with far more writes to deliver.db, mailbox
creations and removals numbering only a few dozen a day,
while message deliveries can run into ten to twenty per
second).

On the partition where both databases reside we observe
double the amount of writes compare to any of the eight
message spools.


Eric Luyten, Computing Centre VUB/ULB.



From bsh at freemail.hu  Fri Apr 24 11:37:30 2009
From: bsh at freemail.hu (=?UTF-8?B?S8WRdsOhcmkgSsOhbm9z?=)
Date: Fri, 24 Apr 2009 17:37:30 +0200
Subject: 
In-Reply-To: <49F1C7A0.6030905@olp.net>
References: <49F15D08.9090503@freemail.hu> <49F161A5.4050708@freemail.hu>
	<49F1C7A0.6030905@olp.net>
Message-ID: <49F1DCBA.5010901@freemail.hu>



Dan White ?rta:
> K?v?ri J?nos wrote:
>> K?v?ri J?nos wrote:
>>> >/ Postfix uses saslauthd, which is configured for PAM. It works />/ 
>>> perfectly, with plain/login/cram/digest mechanisms, with or without 
>>> />/ tls/ssl, absolutely no problems with it. Saslauth tests are all 
>>> fine />/ obviously.
>>> />/ So I decided to use this with cyrus imap too. Set it to use the 
>>> same />/ saslauth daemon, and plain, login, cram-md5 and digest-md5 
>>> mechs.
>>> />/ Since then, I can not login with plain or login mechs, because 
>>> they />/ aren't being offered at all by cyrus imapd. I can login 
>>> with cram or />/ digest fine.
>>> />/ I understand that plain login isn't offered by default, only 
>>> after a />/ successfull tls session setup, but if I understand 
>>> correctly, the />/ "allowplaintext: yes" option should still force 
>>> imapd to offer plain />/ logins. But it doesn't. I tried it with 
>>> different sasl_min|max_levels, />/ to no avail.
>>> /
>>> Please include the following information, so we can get a better 
>>> idea of your setup:
>>>
>>> Postfix and Cyrus IMAP version
>>> Postfix SASL config:
>>>   grep sasl main.cf
>>>   cat /etc/postfix/sasl/smtpd.conf (or wherever smtpd.conf it 
>>> located on your system)
>>>
>>>
>>>     
>> Hello Dan,
>>
>> Postfix version: 2.5.4
>> Cyrus IMAP version: 2.2.13
>>
>> smtpd_sasl_auth_enable = yes
>>
>> /cat /etc/postfix/sasl/smtpd.conf/
>> saslauthd_version: 2
>> pwcheck_method: saslauthd
>> mech_list: plain login cram-md5 digest-md5
>>
>> /cat /etc/imapd.conf/
>> allowplaintext: yes
>> saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux
>> sasl_pwcheck_method: saslauthd
>> sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5
>> sasl_auto_transition: no
>>
>> /cat /etc/default/saslauthd/
>> START=yes
>> PWDIR="/var/spool/postfix/var/run/saslauthd"
>> PARAMS="-m ${PWDIR}"
>> PIDFILE="${PWDIR}/saslauthd.pid"
>> MECHANISMS="pam"
>> MECH_OPTIONS=""
>> THREADS=5
>> OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd"
>> /#(I think the options line is wrong, the -m part is unneded, but it 
>> was like that, and it works...)/
>
>
> The way that you have postfix configured, it will use saslauthd (only) 
> for plain and login. It (via SASL) will use your auxprop store to 
> authenticate the cram-md5 and digest-md5 mechanisms. I'm assuming that 
> you have configured your users in /etc/sasldb2, since you are 
> authenticating to imapd via digest-md5.
yes, I was using sasldb2 until recently, so the database is set up and 
still there.
> 'allowplaintext: yes' should be all you need to support plain/login on 
> an in-the-clear connection. Since they are being offered after a TLS 
> connection, it's almost if there's a typo in your config for that 
> command.
Hmmm, I see no typo there.
> also:
>
> saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux
>
> is a typo in /etc/imapd.conf. It should be:
>
> sasl_saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux
Thanks!
> For trouble shooting, you might want to comment out 
> 'sasl_pwcheck_method: saslauthd', which will direct imapd to use all 
> available pw_check methods (including auxprop) for plain/login.
>
> - Dan
I did that too. When it's commented out, the plain and login methods are 
still not being offered, but neither cram nor digest! And I can not 
login at all. Doesn't accept any passwords.
So I reverted it to saslauthd.

Regards,
Janos

From dwhite at olp.net  Fri Apr 24 12:10:29 2009
From: dwhite at olp.net (Dan White)
Date: Fri, 24 Apr 2009 11:10:29 -0500
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <49F1DCBA.5010901@freemail.hu>
References: <49F15D08.9090503@freemail.hu> <49F161A5.4050708@freemail.hu>
	<49F1C7A0.6030905@olp.net> <49F1DCBA.5010901@freemail.hu>
Message-ID: <49F1E475.7070609@olp.net>

K?v?ri J?nos wrote:
>
>> For trouble shooting, you might want to comment out 
>> 'sasl_pwcheck_method: saslauthd', which will direct imapd to use all 
>> available pw_check methods (including auxprop) for plain/login.
>>
> I did that too. When it's commented out, the plain and login methods 
> are still not being offered, but neither cram nor digest! And I can 
> not login at all. Doesn't accept any passwords.
> So I reverted it to saslauthd.

Try:

sasl_pwcheck_method: auxprop

and see if that works.

Also, since your Postfix works, try duplicating its config in 
/etc/imapd.conf:

sasl_saslauthd_version: 2
sasl_pwcheck_method: saslauthd
sasl_mech_list: plain login cram-md5 digest-md5

You'll also need:

sasl_saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux

and remove:

sasl_auto_transition: no

Make sure the cyrus user has permissions to access the mux:

sudo -u cyrus ls /var/spool/postfix/var/run/saslauthd/mux

- Dan

From dwhite at olp.net  Fri Apr 24 12:20:03 2009
From: dwhite at olp.net (Dan White)
Date: Fri, 24 Apr 2009 11:20:03 -0500
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <49F1E475.7070609@olp.net>
References: <49F15D08.9090503@freemail.hu> <49F161A5.4050708@freemail.hu>
	<49F1C7A0.6030905@olp.net> <49F1DCBA.5010901@freemail.hu>
	<49F1E475.7070609@olp.net>
Message-ID: <49F1E6B3.8010205@olp.net>

Dan White wrote:
>
> Also, since your Postfix works, try duplicating its config in 
> /etc/imapd.conf:
>

Is your postfix running chroot'd? If so, where is the sasldb2 file that 
it's using located? In /var/spool/postfix/etc ?

If so, try adding to /etc/imapd.conf:

sasl_sasldb_path: /var/spool/postfix/etc/sasldb2

- Dan

From dwhite at olp.net  Fri Apr 24 13:45:59 2009
From: dwhite at olp.net (Dan White)
Date: Fri, 24 Apr 2009 12:45:59 -0500
Subject: Cyrus Imap plaintext authentication with saslauth & PAM
In-Reply-To: <49F1F234.4040900@freemail.hu>
References: <49F15D08.9090503@freemail.hu> <49F161A5.4050708@freemail.hu>
	<49F1C7A0.6030905@olp.net> <49F1DCBA.5010901@freemail.hu>
	<49F1E475.7070609@olp.net> <49F1E6B3.8010205@olp.net>
	<49F1F234.4040900@freemail.hu>
Message-ID: <49F1FAD7.9080808@olp.net>

K?v?ri J?nos wrote:
> Dan White ?rta:
>>
>> Is your postfix running chroot'd? If so, where is the sasldb2 file 
>> that it's using located? In /var/spool/postfix/etc ?
>>
>> If so, try adding to /etc/imapd.conf:
>>
>> sasl_sasldb_path: /var/spool/postfix/etc/sasldb2
>>
>> - Dan
>>
> Yes, it's chroot'd.
> I have sasldb2 both in chroot and /etc. Both are readable by cyrus. I 
> don't think it's the problem, I remember when I forgot to add users to 
> sasldb2 and tried to login, I got an error message in the logs, 
> saying: no secret in the database or something. So it does find the 
> database. (But I can be wrong, it was quite some time ago...) But 
> previously I was using sasldb2 without problems, so I assuem it is set 
> up more or less correctly.
>
> And please keep in mind, that I *don't* want sasldb, this whole thing 
> with saslauthd is about avoiding sasldb2 and to use plaintext 
> authentication with PAM-only.
>
>
> Have a good weekend to everyone reading this! :)
>
> Janos

True,

I'm just trying to reproduce your Postfix environment in Cyrus imapd. I 
think you must be using sasldb when performing cram/disgest 
authentication, not PAM (since saslauthd/PAM do not support them).

- Dan

From dbucherml at hsolutions.ch  Sun Apr 26 14:21:27 2009
From: dbucherml at hsolutions.ch (Denis BUCHER)
Date: Sun, 26 Apr 2009 20:21:27 +0200
Subject: Searching for a Sieve web Autoreplier OR Outlook solution
In-Reply-To: <1963.10.70.3.113.1239639896.squirrel@borg>
References: <49E24BB8.7040404@hsolutions.ch>
	<1963.10.70.3.113.1239639896.squirrel@borg>
Message-ID: <49F4A627.1020508@hsolutions.ch>

Hello everyone,

I tried websieve and it was the perfect solution. The code and interface
are quite old, but easy to modify and simplify...

Have a nice week

Denis

John Widera a ?crit :
>> Hello,
>>
>> I'm searching for months without success for either one of these solutions
>> :
>>
>> a) Out-of-office (away) plugin for Outlook that would use Sieve ?
>>
>> b) Web interface to manage an out-of-office message in Sieve ?
> 
> Websieve?  Easysieve?
> 
>> If someone could give me some advice, it would be very nice !
>>
>> Denis
>> ----
>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>>
>>
> 
> 
> 


Denis Bucher

-- 

Denis Bucher   Horus Digital Solutions s?rl  Each problem has a solution
___________________________________________________________________________
   T?l. +41-22-8000625     Fax: +41-22-8000622      www.hsolutions.ch

From mills at cc.umanitoba.ca  Tue Apr 28 09:13:04 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Tue, 28 Apr 2009 08:13:04 -0500
Subject: Multiple copies of cyr_expire running
Message-ID: <20090428131304.GA9385@cc.umanitoba.ca>

I notice that there are two of these running today:

    $ ps -fp "$(pgrep cyr_expire)"
         UID   PID  PPID   C    STIME TTY         TIME CMD
       cyrus  2510   986   3 04:00:01 ?         219:28 cyr_expire -E 3
       cyrus 18280   986   3   Apr 27 ?        1580:15 cyr_expire -E 3

There are also lots of errors like this.  They refer to the same
message over and over again:

    Apr 28 08:07:56 castor cyr_expire[18280]: [ID 264569 local6.error] DBERROR: mydelete: error deleting <200904201356.n3KDuJes008536 at taygeta.cc.umanitoba.ca>: DB_NOTFOUND: No matching key/data pair found

Should I kill one of the cyr_expire processes?  Is there a safe way
to do this?  Is the duplicate delivery database broken?  Is there a
way to fix it?

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From awilliam at whitemice.org  Tue Apr 28 14:10:02 2009
From: awilliam at whitemice.org (Adam Tauno Williams)
Date: Tue, 28 Apr 2009 14:10:02 -0400
Subject: Multiple copies of cyr_expire running
In-Reply-To: <20090428131304.GA9385@cc.umanitoba.ca>
References: <20090428131304.GA9385@cc.umanitoba.ca>
Message-ID: <1240942202.5394.4.camel@linux-m3mt>

On Tue, 2009-04-28 at 08:13 -0500, Gary Mills wrote:
> I notice that there are two of these running today:
>     $ ps -fp "$(pgrep cyr_expire)"
>          UID   PID  PPID   C    STIME TTY         TIME CMD
>        cyrus  2510   986   3 04:00:01 ?         219:28 cyr_expire -E 3
>        cyrus 18280   986   3   Apr 27 ?        1580:15 cyr_expire -E 3
> There are also lots of errors like this.  They refer to the same
> message over and over again:
>     Apr 28 08:07:56 castor cyr_expire[18280]: [ID 264569 local6.error] DBERROR: mydelete: error deleting <200904201356.n3KDuJes008536 at taygeta.cc.umanitoba.ca>: DB_NOTFOUND: No matching key/data pair found
> Should I kill one of the cyr_expire processes?  Is there a safe way
> to do this?  

I'd kill -15 both of them.  Then watch to see if they get stuck again.

> Is the duplicate delivery database broken?  Is there a
> way to fix it?

There is no reason to fix it; I'd just delete it.  You maybe will be a
couple duplicates but no big deal.
-- 
OpenGroupware developer: awilliam at whitemice.org
<http://whitemiceconsulting.blogspot.com/>
OpenGroupare & Cyrus IMAPd documenation @
<http://docs.opengroupware.org/Members/whitemice/wmogag/file_view>


From mills at cc.umanitoba.ca  Tue Apr 28 14:55:01 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Tue, 28 Apr 2009 13:55:01 -0500
Subject: Multiple copies of cyr_expire running
In-Reply-To: <1240942202.5394.4.camel@linux-m3mt>
References: <20090428131304.GA9385@cc.umanitoba.ca>
	<1240942202.5394.4.camel@linux-m3mt>
Message-ID: <20090428185501.GA8759@cc.umanitoba.ca>

On Tue, Apr 28, 2009 at 02:10:02PM -0400, Adam Tauno Williams wrote:
> On Tue, 2009-04-28 at 08:13 -0500, Gary Mills wrote:
> > I notice that there are two of these running today:
> >     $ ps -fp "$(pgrep cyr_expire)"
> >          UID   PID  PPID   C    STIME TTY         TIME CMD
> >        cyrus  2510   986   3 04:00:01 ?         219:28 cyr_expire -E 3
> >        cyrus 18280   986   3   Apr 27 ?        1580:15 cyr_expire -E 3
> > There are also lots of errors like this.  They refer to the same
> > message over and over again:
> >     Apr 28 08:07:56 castor cyr_expire[18280]: [ID 264569 local6.error] DBERROR: mydelete: error deleting <200904201356.n3KDuJes008536 at taygeta.cc.umanitoba.ca>: DB_NOTFOUND: No matching key/data pair found
> > Should I kill one of the cyr_expire processes?  Is there a safe way
> > to do this?  
> 
> I'd kill -15 both of them.  Then watch to see if they get stuck again.

I did that last time around, with bad results.  POP3 stopped working.
I had to restart master to fix that.

> > Is the duplicate delivery database broken?  Is there a
> > way to fix it?
> 
> There is no reason to fix it; I'd just delete it.  You maybe will be a
> couple duplicates but no big deal.

I thought that some information need by the sieve vacation responder
was stored in that database.  I don't want to break that feature for
thousands of people.

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From brong at fastmail.fm  Tue Apr 28 20:12:03 2009
From: brong at fastmail.fm (Bron Gondwana)
Date: Wed, 29 Apr 2009 10:12:03 +1000
Subject: Multiple copies of cyr_expire running
In-Reply-To: <20090428185501.GA8759@cc.umanitoba.ca>
References: <20090428131304.GA9385@cc.umanitoba.ca>
	<1240942202.5394.4.camel@linux-m3mt>
	<20090428185501.GA8759@cc.umanitoba.ca>
Message-ID: <20090429001203.GB14794@brong.net>

On Tue, Apr 28, 2009 at 01:55:01PM -0500, Gary Mills wrote:
> On Tue, Apr 28, 2009 at 02:10:02PM -0400, Adam Tauno Williams wrote:
> > On Tue, 2009-04-28 at 08:13 -0500, Gary Mills wrote:
> > > I notice that there are two of these running today:
> > >     $ ps -fp "$(pgrep cyr_expire)"
> > >          UID   PID  PPID   C    STIME TTY         TIME CMD
> > >        cyrus  2510   986   3 04:00:01 ?         219:28 cyr_expire -E 3
> > >        cyrus 18280   986   3   Apr 27 ?        1580:15 cyr_expire -E 3
> > > There are also lots of errors like this.  They refer to the same
> > > message over and over again:
> > >     Apr 28 08:07:56 castor cyr_expire[18280]: [ID 264569 local6.error] DBERROR: mydelete: error deleting <200904201356.n3KDuJes008536 at taygeta.cc.umanitoba.ca>: DB_NOTFOUND: No matching key/data pair found

Bloody BDB.  I wish I understood it better.  Lots of people use it, so
it seems it must be something odd Cyrus does that causes it to be
relatively unreliable...

> > > Should I kill one of the cyr_expire processes?  Is there a safe way
> > > to do this?  
> > 
> > I'd kill -15 both of them.  Then watch to see if they get stuck again.
> 
> I did that last time around, with bad results.  POP3 stopped working.
> I had to restart master to fix that.
 
Odd - it shouldn't.  I have killed cyr_expire without problems before.

Then again, we only run it once per week, so it never wraps!

> > > Is the duplicate delivery database broken?  Is there a
> > > way to fix it?
> > 
> > There is no reason to fix it; I'd just delete it.  You maybe will be a
> > couple duplicates but no big deal.
> 
> I thought that some information need by the sieve vacation responder
> was stored in that database.  I don't want to break that feature for
> thousands of people.

It may send a vacation response again.  All it stores is the "vacation
already sent" data.

I would restart the master while deleting it though.

Bron ( yes, that does kick off all your users... )

From mills at cc.umanitoba.ca  Tue Apr 28 22:32:51 2009
From: mills at cc.umanitoba.ca (Gary Mills)
Date: Tue, 28 Apr 2009 21:32:51 -0500
Subject: Multiple copies of cyr_expire running
In-Reply-To: <20090429001203.GB14794@brong.net>
References: <20090428131304.GA9385@cc.umanitoba.ca>
	<1240942202.5394.4.camel@linux-m3mt>
	<20090428185501.GA8759@cc.umanitoba.ca>
	<20090429001203.GB14794@brong.net>
Message-ID: <20090429023251.GA9378@cc.umanitoba.ca>

On Wed, Apr 29, 2009 at 10:12:03AM +1000, Bron Gondwana wrote:
> On Tue, Apr 28, 2009 at 01:55:01PM -0500, Gary Mills wrote:
> > On Tue, Apr 28, 2009 at 02:10:02PM -0400, Adam Tauno Williams wrote:
> > > On Tue, 2009-04-28 at 08:13 -0500, Gary Mills wrote:
> > > > I notice that there are two of these running today:
> > > >     $ ps -fp "$(pgrep cyr_expire)"
> > > >          UID   PID  PPID   C    STIME TTY         TIME CMD
> > > >        cyrus  2510   986   3 04:00:01 ?         219:28 cyr_expire -E 3
> > > >        cyrus 18280   986   3   Apr 27 ?        1580:15 cyr_expire -E 3
> > > > There are also lots of errors like this.  They refer to the same
> > > > message over and over again:
> > > >     Apr 28 08:07:56 castor cyr_expire[18280]: [ID 264569 local6.error] DBERROR: mydelete: error deleting <200904201356.n3KDuJes008536 at taygeta.cc.umanitoba.ca>: DB_NOTFOUND: No matching key/data pair found
> 
> Bloody BDB.  I wish I understood it better.  Lots of people use it, so
> it seems it must be something odd Cyrus does that causes it to be
> relatively unreliable...

Yes, I hate that one too!  It's the only one.  The others are all
skiplist or flat.

> > > > Should I kill one of the cyr_expire processes?  Is there a safe way
> > > > to do this?  
> > > 
> > > I'd kill -15 both of them.  Then watch to see if they get stuck again.
> > 
> > I did that last time around, with bad results.  POP3 stopped working.
> > I had to restart master to fix that.
>  
> Odd - it shouldn't.  I have killed cyr_expire without problems before.

I didn't expect a problem either.

> Then again, we only run it once per week, so it never wraps!
> 
> > > > Is the duplicate delivery database broken?  Is there a
> > > > way to fix it?
> > > 
> > > There is no reason to fix it; I'd just delete it.  You maybe will be a
> > > couple duplicates but no big deal.
> > 
> > I thought that some information need by the sieve vacation responder
> > was stored in that database.  I don't want to break that feature for
> > thousands of people.
> 
> It may send a vacation response again.  All it stores is the "vacation
> already sent" data.

Okay, that's likely what I'll do.  I'll try your cyr_dbtool first to
see if it can delete that index entry.

> I would restart the master while deleting it though.
> Bron ( yes, that does kick off all your users... )

Yep.  Most e-mail clients seem to reconnect quickly, so it shouldn't
be too bad.

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-

From kleiner77 at gmx.net  Wed Apr 29 08:12:01 2009
From: kleiner77 at gmx.net (Stefan Palme)
Date: Wed, 29 Apr 2009 14:12:01 +0200
Subject: strange pop protocol?
Message-ID: <1241007121.7992.51.camel@devel.office.ancoso-development.de>

Hi all

Using the POP3 server of Cyrus IMAP Server 2.3.13 I can see the 
following POP3 protocol (sniffed via tcpdump and friends and 
obfuscated sensitive data):

----------------------------------------------------
+OK <98640940.1241005202 at imap.example.com> imap.example.com Cyrus POP3 v2.3.13-Gentoo server ready
USER username
+OK Name is a valid mailbox
PASS passphrase
+OK Mailbox locked and ready
UIDL
+OK unique-id listing follows
1 1157352258.220081
2 1157352258.220082
.
LIST
+OK scan listing follows
1 14363
2 360389
.
RETR 1
+OK Message follows
Return-Path: <sender at example.com>
Received: ...
/// here the email headers and body follow

RETR 2
DELE 1
DELE 2
QUIT
----------------------------------------------------

The point I am interested in is the fact, that both the UIDL and the LIST
commands show TWO mails. But when the client tries to retrieve the second
one (RETR 2), nothing is returned. 

This does not happen all the times - I've already seen non-empty "RETR 2"
calls during other client sessions.

Is this normal behaviour? Or do I have to afraid of mail loss, because after
the empty "RETR 2" the clients deletes BOTH messages from the server. Is this
strange message 2 really "not-existing", or do I loose some mail data anywhere?

Thanks and regards
-stefan-



From bond0113 at yahoo.com  Wed Apr 29 09:26:56 2009
From: bond0113 at yahoo.com (mami bond)
Date: Wed, 29 Apr 2009 06:26:56 -0700 (PDT)
Subject: Hello
Message-ID: <463682.8682.qm@web36201.mail.mud.yahoo.com>

First time writing e-mail for asking help.
I try to configure cyrus+web-cyradm on Ubuntu. everything want well, except the web-cyradm.
For month I am fighting this issue and in vain.
using  "unixhierarchysep: yes" I can create accounts using Web-cyradm (version: web-cyradm-svn-0.5.5)
in the cyrus username have slash and that ok. According the Cyrus changing "unixhierarchysep: no" should change slah to dot and it works but I cannot create user account anymore from Web-cyradm. the path is empty "/var/spool/cyrus/main/letter/user/username".
I would like to use "unixhierarchysep: no" and able to create account with web-cyradm.

How can I solve this issue?

Thank you



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090429/1a20d036/attachment.html 

From kleiner77 at gmx.net  Thu Apr 30 03:08:06 2009
From: kleiner77 at gmx.net (Stefan Palme)
Date: Thu, 30 Apr 2009 09:08:06 +0200
Subject: strange pop protocol?
In-Reply-To: <1241007121.7992.51.camel@devel.office.ancoso-development.de>
References: <1241007121.7992.51.camel@devel.office.ancoso-development.de>
Message-ID: <1241075287.12164.9.camel@devel.office.ancoso-development.de>


I've just took a look on the exact POP-3 spec. There seems to
be another problem:

> ----------------------------------------------------
> +OK <98640940.1241005202 at imap.example.com> imap.example.com Cyrus POP3 v2.3.13-Gentoo server ready
> USER username
> +OK Name is a valid mailbox
> PASS passphrase
> +OK Mailbox locked and ready
> ...
> RETR 2
> DELE 1
> DELE 2
> QUIT
> ----------------------------------------------------

As far as I understood, a POP3 server must ALWAYS send a response
(+OK or -ERR). But in my case the server does not response at all.

Any hints why/how this can happen? I see this behaviour very
often on the wire. Maybe it is worth mentioning that the client
in this case is MS Exchange.

Thanks and regards
-stefan-



From janne.peltonen at helsinki.fi  Thu Apr 30 03:55:50 2009
From: janne.peltonen at helsinki.fi (Janne Peltonen)
Date: Thu, 30 Apr 2009 10:55:50 +0300
Subject: Bug? xfermailbox seems to be broken but using rename to xfer a
	mailbox works just fine.
Message-ID: <20090430075550.GB8861@helsinki.fi>

Hi!

Am I doing something wrong? If I try to do an xfer from one backend to another
on a user, like so,

  xfer user.atest001r m2v3t.mappi.helsinki.fi

I get

Apr 30 10:24:45 m2cn1t m2v1t/imap[3793]: could not dump mailbox in m2v2t.mappi.helsinki.fi (unknown error)
Apr 30 10:24:45 m2cn1t m2v1t/imap[3793]: Could not move mailbox: user.atest001r, dump_mailbox() failed

in my murder frontend log, the command (in cyradm) never returns, and the
mailbox list on the frontend ends gets corrupted, like this:

user.atest001r  1 m2v1t.mappi.helsinki.fi!m2v2t.mappi.helsinki.fi atest001r     lrswipkxtecda   anyone  p

(m2v1t.mappi.helsinki.fi is the murder frontend, the murder backends are
m2v2t.mappi.helsinki.fi and m2v3t.mappi.helsinki.fi, and m2v2t was the original
backend on atest001r).

The mailbox seems to get copied over from one backend to another, but the
contents don't:

[jmmpelto at m2cn1t ~]$ sudo /usr/lib/cyrus-imapd/mbpath -C /etc/imapd.conf.m2v2t.master user.atest001r
/var/spool/imap/m2v2t/a/user/atest001r
[jmmpelto at m2cn1t ~]$ sudo ls -l /var/spool/imap/m2v2t/a/user/atest001r
total 72
-rw------- 1 cyrus mail 6004 Mar 31 16:56 cyrus.cache
-rw------- 1 cyrus mail  182 Mar 31 14:08 cyrus.header
-rw------- 1 cyrus mail 1328 Mar 31 16:56 cyrus.index
-rw------- 1 cyrus mail  573 Mar 31 16:55 1.
-rw------- 1 cyrus mail  573 Mar 31 16:55 10.
-rw------- 1 cyrus mail  574 Mar 31 16:55 11.
-rw------- 1 cyrus mail  573 Mar 31 16:55 12.
-rw------- 1 cyrus mail  574 Mar 31 16:56 13.
-rw------- 1 cyrus mail  573 Mar 31 16:56 14.
-rw------- 1 cyrus mail  573 Mar 31 16:55 2.
-rw------- 1 cyrus mail  573 Mar 31 16:55 3.
-rw------- 1 cyrus mail  573 Mar 31 16:55 4.
-rw------- 1 cyrus mail  573 Mar 31 16:55 5.
-rw------- 1 cyrus mail  573 Mar 31 16:55 6.
-rw------- 1 cyrus mail  573 Mar 31 16:55 7.
-rw------- 1 cyrus mail  573 Mar 31 16:55 8.
-rw------- 1 cyrus mail  573 Mar 31 16:55 9.
[jmmpelto at m2cn1t ~]$ sudo /usr/lib/cyrus-imapd/mbpath -C /etc/imapd.conf.m2v3t.master user.atest001r
/var/spool/imap/m2v3t/a/user/atest001r
[jmmpelto at m2cn1t ~]$ sudo ls -l /var/spool/imap/m2v3t/a/user/atest001r
total 12
-rw------- 1 cyrus mail   4 Apr 30 10:24 cyrus.cache
-rw------- 1 cyrus mail 159 Apr 30 10:24 cyrus.header
-rw------- 1 cyrus mail  96 Apr 30 10:24 cyrus.index

So we end up having a copy of the mailbox on both the original and the new
backend.

If I try to rename a mailbox, everything goes perfectly:

m2v1t.mappi.helsinki.fi> info user.atest002r
{user.atest002r}:
  condstore: false
  duplicatedeliver: false
  lastpop:  
  lastupdate: 31-Mar-2009 16:56:15 +0300
  partition: m2v2t
  server: m2v2t.mappi.helsinki.fi
  sharedseen: false
  size: 8022
m2v1t.mappi.helsinki.fi> rename user.atest002r user.atest002r m2v3t.mappi.helsinki.fi
m2v1t.mappi.helsinki.fi> info user.atest002r
{user.atest002r}:
  condstore: false
  duplicatedeliver: false
  lastpop:  
  lastupdate: 30-Apr-2009 10:31:25 +0300
  partition: m2v3t
  server: m2v3t.mappi.helsinki.fi
  sharedseen: false
  size: 8022

Any ideas? I'm using Cyrus 2.3.13, Simon's RPM revision 4.


--Janne
-- 
Janne Peltonen <janne.peltonen at helsinki.fi> PGP Key ID: 0x9CFAC88B
Please consider membership of the Hospitality Club (http://www.hospitalityclub.org)

From bond0113 at yahoo.com  Thu Apr 30 04:45:26 2009
From: bond0113 at yahoo.com (mami bond)
Date: Thu, 30 Apr 2009 01:45:26 -0700 (PDT)
Subject: unixhierarchysep: yes/no doesn't work as expected
In-Reply-To: <4a5881460904290954l202c42b3v2d3bd27b2a75d011@mail.gmail.com>
References: <463682.8682.qm@web36201.mail.mud.yahoo.com>
	<4a5881460904290954l202c42b3v2d3bd27b2a75d011@mail.gmail.com>
Message-ID: <837441.32665.qm@web36208.mail.mud.yahoo.com>

Thanks for the reply,
I tried first using "#unixhierarchysep: no" and no mailbox was created in Cyrus "cyradm -u cyrus localhost" totally empty and also empty in the folder "/var/spool/cyrus/mail/letter/user/username". That's why I changed to YES -> "unixhierarchysep: yes" and it works with the user/username.

Where is the issue making work with "unixhierarchysep: yes -> user/username" and not "unixhierarchysep: no -> user.username"

Please, help
Thank you




________________________________
De : Reinaldo de Carvalho <reinaldoc at gmail.com>
? : mami bond <bond0113 at yahoo.com>
Envoy? le : Mercredi, 29 Avril 2009, 19h54mn 29s
Objet : Re: Hello

On Wed, Apr 29, 2009 at 10:26 AM, mami bond <bond0113 at yahoo.com> wrote:
> First time writing e-mail for asking help.
> I try to configure cyrus+web-cyradm on Ubuntu. everything want well, except
> the web-cyradm.
> For month I am fighting this issue and in vain.
> using  "unixhierarchysep: yes" I can create accounts using Web-cyradm
> (version: web-cyradm-svn-0.5.5)
> in the cyrus username have slash and that ok. According the Cyrus changing
> "unixhierarchysep: no" should change slah to dot and it works but I cannot
> create user account anymore from Web-cyradm. the path is empty
> "/var/spool/cyrus/main/letter/user/username".
> I would like to use "unixhierarchysep: no" and able to create account with
> web-cyradm.
>
> How can I solve this issue?
>
> Thank you
>

You can't change unixhierarchysep after create "first" mailbox.



-- 
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090430/782477c8/attachment.html 

From bawood at umich.edu  Thu Apr 30 11:05:14 2009
From: bawood at umich.edu (Brian Awood)
Date: Thu, 30 Apr 2009 11:05:14 -0400
Subject: Bug? xfermailbox seems to be broken but using rename to xfer a
	mailbox works just fine.
In-Reply-To: <20090430075550.GB8861@helsinki.fi>
References: <20090430075550.GB8861@helsinki.fi>
Message-ID: <200904301105.15120.bawood@umich.edu>


On Thursday 30 April 2009 @ 03:55, Janne Peltonen wrote:
> Hi!
>
> Am I doing something wrong? If I try to do an xfer from one backend
> to another on a user, like so,
>
>   xfer user.atest001r m2v3t.mappi.helsinki.fi
>
> I get
>
> Apr 30 10:24:45 m2cn1t m2v1t/imap[3793]: could not dump mailbox in
> m2v2t.mappi.helsinki.fi (unknown error) Apr 30 10:24:45 m2cn1t
> m2v1t/imap[3793]: Could not move mailbox: user.atest001r,
> dump_mailbox() failed
>
> in my murder frontend log, the command (in cyradm) never returns,
> and the mailbox list on the frontend ends gets corrupted, like
> this:
>
> user.atest001r  1 m2v1t.mappi.helsinki.fi!m2v2t.mappi.helsinki.fi
> atest001r     lrswipkxtecda   anyone  p
>
> (m2v1t.mappi.helsinki.fi is the murder frontend, the murder
> backends are m2v2t.mappi.helsinki.fi and m2v3t.mappi.helsinki.fi,
> and m2v2t was the original backend on atest001r).

It looks like error was logged from your frontend host m2v1t, so I 
would guess that the xfer command was issued to it, which isn't 
correct.  You have to connect to the backend where the mailbox is 
stored and give the xfer command there.  But the fact that your 
frontend database gets corrupted like and that you didn't get a 
better error (like "mailbox not local"), definitely seems like a bug.  
What is you mupdate_config set to on your frontend?  

Brian

From wes at umich.edu  Thu Apr 30 11:37:05 2009
From: wes at umich.edu (Wesley Craig)
Date: Thu, 30 Apr 2009 11:37:05 -0400
Subject: Bug? xfermailbox seems to be broken but using rename to xfer a
	mailbox works just fine.
In-Reply-To: <200904301105.15120.bawood@umich.edu>
References: <20090430075550.GB8861@helsinki.fi>
	<200904301105.15120.bawood@umich.edu>
Message-ID: <0E732C24-E638-4FA8-B516-E52874B48DA0@umich.edu>

On 30 Apr 2009, at 11:05, Brian Awood wrote:
> It looks like error was logged from your frontend host m2v1t, so I
> would guess that the xfer command was issued to it, which isn't
> correct.  You have to connect to the backend where the mailbox is
> stored and give the xfer command there.  But the fact that your
> frontend database gets corrupted like and that you didn't get a
> better error (like "mailbox not local"), definitely seems like a bug.
> What is you mupdate_config set to on your frontend?

There's no test in cmd_xfer for MBTYPE_REMOTE (unlike cmd_rename).   
That's either a bug in that it ought to notice and report an error,  
or it ought to notice and proxy the xfer to the hosting backend.  I  
think the trend is for the latter solution, tho the former solution  
is a fine stop-gap.

:wes

From wes at umich.edu  Thu Apr 30 11:40:09 2009
From: wes at umich.edu (Wesley Craig)
Date: Thu, 30 Apr 2009 11:40:09 -0400
Subject: strange pop protocol?
In-Reply-To: <1241075287.12164.9.camel@devel.office.ancoso-development.de>
References: <1241007121.7992.51.camel@devel.office.ancoso-development.de>
	<1241075287.12164.9.camel@devel.office.ancoso-development.de>
Message-ID: <A5027436-F68B-48A7-8F03-03CF99CA8F2B@umich.edu>

On 30 Apr 2009, at 03:08, Stefan Palme wrote:
> As far as I understood, a POP3 server must ALWAYS send a response
> (+OK or -ERR). But in my case the server does not response at all.
>
> Any hints why/how this can happen? I see this behaviour very
> often on the wire. Maybe it is worth mentioning that the client
> in this case is MS Exchange.

Is the pop server hung?  Perhaps the mailbox is corrupted, so message  
two is causing the pop server to hang, crash, whatever.

:wes

From kleiner77 at gmx.net  Thu Apr 30 13:10:18 2009
From: kleiner77 at gmx.net (Stefan Palme)
Date: Thu, 30 Apr 2009 19:10:18 +0200 (CEST)
Subject: strange pop protocol?
Message-ID: <1071.83.221.68.220.1241111418.squirrel@webmail2.hora-obscura.de>

On Thu, April 30, 2009 17:40, Wesley Craig wrote:
> On 30 Apr 2009, at 03:08, Stefan Palme wrote:
>> As far as I understood, a POP3 server must ALWAYS send a response (+OK
or -ERR). But in my case the server does not response at all.
>>
>> Any hints why/how this can happen? I see this behaviour very
>> often on the wire. Maybe it is worth mentioning that the client in this
case is MS Exchange.
>
> Is the pop server hung?  Perhaps the mailbox is corrupted, so message
two is causing the pop server to hang, crash, whatever.

No, the pop server does not hang. After this "QUIT" message the client
disappears. The next connection from the same client (or from any other
client) works normally - client can fetch at least one email.

Often everything looks "normal" - i.e. for each command there is a normal
response from the server. But sometimes the server does not respond at all
in the middle of a session (without beeing hung or something).

Error log does not show any errors, so I guess the mailbox data is not
corrupted. I have some error messages of the kind "mailbox locked" (can
not look up the exact message right now), but this does not seem to be
related to the other problem (because of different times when this
happens). Or maybe this IS a possible reason somehow?

Thanks and regards
-stefan-






From wes at umich.edu  Thu Apr 30 14:15:48 2009
From: wes at umich.edu (Wesley Craig)
Date: Thu, 30 Apr 2009 14:15:48 -0400
Subject: strange pop protocol?
In-Reply-To: <1071.83.221.68.220.1241111418.squirrel@webmail2.hora-obscura.de>
References: <1071.83.221.68.220.1241111418.squirrel@webmail2.hora-obscura.de>
Message-ID: <72F7BC39-CAA8-4CCC-8DFF-EF09CEBDA86A@umich.edu>

On 30 Apr 2009, at 13:10, Stefan Palme wrote:
> No, the pop server does not hang. After this "QUIT" message the client
> disappears. The next connection from the same client (or from any  
> other
> client) works normally - client can fetch at least one email.

Have you enabled cyrus telemetry for this user?  I'd probably look at  
that.  The behavior you're describing doesn't make a ton of sense.

:wes

From kleiner77 at gmx.net  Thu Apr 30 14:48:57 2009
From: kleiner77 at gmx.net (kleiner77 at gmx.net)
Date: Thu, 30 Apr 2009 20:48:57 +0200
Subject: strange pop protocol?
In-Reply-To: <72F7BC39-CAA8-4CCC-8DFF-EF09CEBDA86A@umich.edu>
References: <1071.83.221.68.220.1241111418.squirrel@webmail2.hora-obscura.de>
	<72F7BC39-CAA8-4CCC-8DFF-EF09CEBDA86A@umich.edu>
Message-ID: <1241117337.6561.1.camel@drops.kapott.org>

On Thu, 2009-04-30 at 14:15 -0400, Wesley Craig wrote:
> On 30 Apr 2009, at 13:10, Stefan Palme wrote:
> > No, the pop server does not hang. After this "QUIT" message the client
> > disappears. The next connection from the same client (or from any  
> > other
> > client) works normally - client can fetch at least one email.
> 
> Have you enabled cyrus telemetry for this user?  I'd probably look at  
> that.  

Never heard about "telemetry" until now, but just have found how
to enable will. Will take a look at the generated logs...

> The behavior you're describing doesn't make a ton of sense.

Yep :-)

Regards
-stefan-