very slow syncing, any ideas?

[Andrew Morgan]

On Thu, 19 Oct 2006, Michael Loftis wrote:

> --On October 19, 2006 11:25:07 PM +0200 Marten Lehmann <lehmann at cnm.de> 
> wrote:
>
>> Hello,
>> 
>>> Uhm... LA of 30 is very high.  What OS?  I assume Linux, vmstat 5 will
>>> tell you where you're hitting the wall, but unless you've got an 8 CPU
>>> machine LA 30 is rather quite high.  Linux LA is a measurement of
>>> processes blocked on I/O, processes running and processes waiting to run
>>> on a CPU.
>> 
>> yes, Linux (2.6.9, RHEL4):
>
> Looking at taht i'd say you're VERY badly CPU bound.  a simple dd/cp doesn't 
> do anything to the mail but IMAP ops will require some CPU work....Cyrus also 
> will probably be forcing syncs but your I/O load doesn't look that high (my 
> mfe's run more I/O and they're not storing any mail, just logs and temporary 
> files for virus/spam scanning heh, and they only have a little IDE HDD each)
>
> the numebrs in the procs->r column indicate you've got a lot of processes 
> vying for CPU time.  the b column indicates you've got a little bit of 
> blocking going on.  Processes in the 'D' state may show up in ps or top 
> output (doubtfully top unless you change the default sort).
>
> I'd guess you're being CPU bound and cyrus is probably running (usually does 
> by default) at a bit lower priority than your interactive cp operations.  2.6 
> kernel scheduler does a lot of weird stuff if it determines you're 
> interactive, and then tends to give you priority over daemons.  it's very 
> non-deterministic, but I don't think you're running into this.  quite likely 
> some inefficiencies in the cyrus code (or intentional backoffs?) but mostly 
> just being CPU bound.  I know our MBEs can generally scp or rsync quite a bit 
> faster than they can perform a XFER/RENAME command that moves between mail 
> store servers (IE MURDER host to MURDER host).

No clue if it matters in this imapsync case, but if you are using the 2.6 
kernel, you should set the I/O scheduler to "deadline" rather than the 
default "anticipatory".  You can set this in the boot loader, such as the 
following for lilo.conf:

   append="elevator=deadline"

or you can change the scheduler on the fly by:

   echo "deadline" > /sys/block/sdb/queue/scheduler

Replace "sdb" with the block device you store your cyrus mail spool on.

 	Andy