slow populating a mailbox

[Ross Boylan]

On Mon, Sep 25, 2006 at 12:04:20PM +0200, Kjetil Torgrim Homme wrote:
> On Sun, 2006-09-24 at 23:17 -0700, Ross Boylan wrote:
> > I've been using the UW-IMAP mailutil to copy messages from my regular
> > Unix mailboxes (for mutt) to IMAP.  The messages are all going to a
> > single folder for a single user.
> > 
> > My impression is that this is rather slow, about 20 minutes for 100MG
> > worth of messages.  There is virtually no CPU use while this happens.
> 
> number of messages is much more important than the size of them.

I did a few more runs, and I'm getting about 18 messages/second (e.g.,
15,000 messages / 14 minutes; 75MG). Which doesn't sound so terrible.
My seen database, for example, is active.

> 
> we've recently relocated a bunch of mailboxes using imapsync, we got a
> speed of about 300 KiB/s for our users.  users which had mostly large
> files with attachments went faster, traditional Unix users went slower,
> perhaps a factor of two either way.
> 
> > The source traditional mailbox and destination IMAP server are on the
> > same machine, using the same hard disk (no, it's not a "real" server).
> 
> that's not improving matters, of course.
> 
> > Second, where should I look to diagnose or solve this problem?
> 
> you could try oprofile to get systemwide performance numbers.
> 
oprofile (thanks--I wasn't aware of this tool) shows

CPU: P4 / Xeon with 2 hyper-threads, speed 3000.14 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (manda
tory) count 100000
GLOBAL_POWER_E...|
  samples|      %|
------------------
  2414674 55.8774 no-vmlinux
   985527 22.8059 libc-client.so.2002edebian.1
   227336  5.2607 libqt-mt.so.3.3.6
   204198  4.7253 libc-2.3.6.so
   116551  2.6971 imapd
    82752  1.9149 libz.so.1.2.3
    41630  0.9634 libkhtml.so.4.2.0
    31830  0.7366 libasound.so.2.0.0
    30574  0.7075 oprofiled
I'm not at all sure what that means, but it seems to show that a lot
of the time is inside the kernel, and libc-client, which I think is
part of the UW mailutils, is #2 on the list.

Ross