Receive Email Even Slower!? -- RE: IMAP select is so slow!?

[Su Li]

Thanks for the tool "strace -T -p". It is very useful.

I use skiplist for seen.db. I got much better select speed. About 11-20
messages per second. Thanks. 

But  each user only has about 50 emails. I need to fill 500 emails and test
again. When I try sending a lot a email using my test tool,  I found Cyrus
is only getting 1 email per second from  sendmail(I just watch the imapd.log
to see the speed of receiving). I run two instances of my test tool, I got
the following error:

Sep  7 14:26:33 web9 lmtpd[5567]: DBERROR db4: 119 lockers
Sep  7 14:26:33 web9 lmtpd[5553]: DBERROR db4: 120 lockers
Sep  7 14:26:33 web9 lmtpd[5720]: DBERROR db4: 124 lockers
Sep  7 14:26:33 web9 lmtpd[5323]: DBERROR db4: 122 lockers
Sep  7 14:26:33 web9 lmtpd[5323]: DBERROR db4: Aborting locker 800000b7

Should I change duplicated to skiplist as well? Or any thing I can change to
speed up recieving mail from sendmail. 

Thanks for every body's suggestion. I really got a lot from the news group.

Su





-----Original Message-----
From: Lawrence Greenfield [mailto:leg+ at andrew.cmu.edu]
Sent: September 6, 2002 2:22 PM
To: 'info-cyrus at lists.andrew.cmu.edu'; Su Li
Subject: Re: IMAP select is so slow!? -- do I need to change database


   From: Su Li <sli at rim.net>
   Date: Fri, 6 Sep 2002 14:12:15 -0400 
[...]
   I wonder why cyrus imap can only do 3-4 select in one second.

You might want to benchmark the imapd process you are connecting to
with strace.

strace -T -p <pid>
or
strace -t -p <pid>
or
strace -c -p <pid>

will all report different information that might be helpful. (I
suspect the -T would be the most helpful.)

I suspect your problem might be with the seen state database, if it is
a flat file and you are running all of these SELECTs as the same user
(thus they have a very large seen state database).

Larry

   I have installed cyrus-imapd-2.1.8, db-4.0.14, cyrus-sasl-2.1.7. The
machine
   is a dual CPU red head Linux 7.2 with 2 G memory. I have 15,000 user's on
my
   machine and each user have 100 emails.  

   My configurations are:

   imap:
   ./configure --with-auth=unix --without-krb4 --without-ucdsnmp
   --with-bdb-incdir=/usr/local/BerkeleyDB.4.0/include
   --with-bdb-libdir=/usr/local/BerkeleyDB.4.0/lib --with-dblib=berkeley
   --with-dbpath=/usr/local/BerkeleyDB.4.0 --with-lock=fcntl
   --with-mboxlist-db=skiplist --without-notify --disable-sieve
   --enable-fulldirhash --with-idle=no

   SASL
   env CPPFLAGS="-I/usr/local/BerkeleyDB.4.0/include" \
   LDFLAGS="-L/usr/local/BerkeleyDB.4.0/lib
-R/usr/local/BerkeleyDB.4.0/lib"
   ./configure --disable-krb4 --disable-gssapi --with-dblib=Berkeley 

   Berkley DB
   ../dist/configure --disable-libtool-lock --without-PACKAGE

   I got some suggestion from Rob Siemborski and Henrique Holschuh. Thanks.
I
   don't get "oening too many files" error now. I moved my mailbox.db to a
   seperate disk. It did improved a little -- about 0.5-0.7 mail per second
   faster. I already got RAID subsystem on my Linux. 

   I am thinking, is there any thing I can do on my imap configuration so
that
   I can speed up my "select"? 

   I am also thing moving mailbox.db to a LDAP or another database. But I
not
   sure how much faster I can get on my "select"?

   Please give me some more suggestions.

   Thanks,


   Su 

   Research In Motion
   (519)888-7465 ext 3041