how many users can Cyrus server support? -- is 5.5 msg/sec slow?

[???]

Hi,

Please look at the LMTP source code.
While lmtpd accepting a new message, it stores the message as a tmp file under disk for later parsing.
To get the better performance, 
I changed the temp file handling to use solaris temp_fs(a memory file system).
And my RAID level was 5, as long as I remember rightly. ;-)

My performance test was done after following actions:
	(1) Hardware + OS : SPARC (450MHz x 1) + solaris 7
	(2) mount a mailstore as UFS
	(3) mount a temp file system (size=10MB)
	(4) modification of lmtpd to use temp_fs to store incoming mails.
	(5) mailbox db as skiplist
	(6) duplication suppress : no
	(7) mailbox creation : I used modified postfix imap client to create large # of mailboxes.
	(8) preparation of lmtp client : get the postfix source and change the smtp client
	     I prefer to use postfix, because it uses async IO scheme, to avoid the 
		client from being performance bottleneck.
	     I made about 10 lmtp connections in parallel.
In addition, I made an experiment with HTTP using my customized httpd(original was thttpd).
when a client POSTs a mail via HTTP, httpd calls cyrus IMAP store API directly.
It was also able to handle over 50 msgs/sec via HTTP. 


-----Original Message-----
From: Su Li [mailto:sli at rim.net]
Sent: Friday, September 27, 2002 10:50 AM
To: 'Christian Schulte'
Cc: 'Lawrence Greenfield'; info-cyrus at andrew.cmu.edu
Subject: RE: how many users can Cyrus server support? -- is 5.5 msg/sec slow?


Hi Christian,

I think you are right. I don't think the bottle neck is Cyrus IMAP. Because
when I move syslog to the network the performance get 2 times better and
when I change to a disk striped by 3 disks, I 2 times faster. Right now I
can get 10 msg/sec. If I can get 15-20 msg/sec, I will be very happy.

Anyways, I got a lot of good suggestions. And now I know Cyrus IMAP can do
so much better. I'll keep working on it.

I'll keep you post, as I got some improvement.

Su


-----Original Message-----
From: Christian Schulte [mailto:cs at schulte.it]
Sent: September 26, 2002 4:20 PM
To: Su Li
Cc: 'Lawrence Greenfield'; info-cyrus at andrew.cmu.edu
Subject: Re: how many users can Cyrus server support? -- is 5.5 msg/sec
sl ow?


Su Li wrote:

>Thanks Larry,
>
>But I did "For Sendmail 8.10 - 8.12.3, use the cyrusv2.mc file as a
template
>to create a Sendmail configuration file."
>  
>
And this did not increase your performance at all ? If I changed 
sendmail to deliver directly to the lmtp socket (by socket I do not mean 
a network socket, I mean the lmtp unix socket!)  I got a tremendous 
performance increase and I'm totally happy on my Intel PIII with 1.1 GHz 
and 512 MB of RAM on a RAID5 SCSI array!!! And that all with Solaris and 
Solaris' very bad performing UFS filesystem! For such a small machine 
the performance is very very well and I do not understand your problems 
any more! Maybe you should start thinking about changing your OS to some 
real BSD unix like FreeBSD, NetBSD, OpenBSD or more commercially 
Solaris! I'm running all that on a Solaris 8 for the intel architecture 
and I can't see any performance problems at all (except the UFS 
performance if writing to the disk but that has nothing to do with cyrus 
or sendmail. That of course is Solaris specific). I'm monitoring the 
whole cyrus installation via snmp _every_ day and I do not have any user 
who talks about performance issues and all are absolutely happy with 
theire email accounts. Many of them are using IMAP (those who know what 
that is, are using it happily ;-)) and even my personal account resides 
on that machine. I have three sendmail installations. One on the same 
machine delivering through the unix lmtp socket and two others on 
different machines delivering through the lmtp network socket and all 
that does really perform quite well. I'm not _the_ poweruser but what on 
earth are you planning for an installation ? Do you really think that 
you run into performance problems with 25 msg/sec ? Do you _really_ 
think that 25 msg/sec is so bad ? 25 msg/sec times 60 are 1500 msg/min 
times 60 are 90000 msg/hour times 24 are 2160000 msg/day ! Do you really 
think that your machine will have to handle over 2 million messages 
_every_ day  ? Isn't that 2 million message think more a peak value on 
high load situations than the approximately everyday situation ? I think 
more and more that your problems do not relate to cyrus. Maybe sendmail 
or even more possible the whole installation (OS, filesystem, the 
software cyrus depends on) is performing bad and you are trying to find 
a solution for a problem which relates to a compülete other think than 
cyrus ? Can you really isolate your problems to cyrus ?