cyrus imap slow on Fedora 13&14

[Simon Matter]

> Hi,
>
> It is not squirrell or cyrus-imapd issue IMO. I installed cyrus-imapd on
> different Fedora distributions. It works great on Fedora <=12. With
> Fedora 13&14 started problems. I didn't notice performance issues on
> F13&14 until I tried to read large (5000+ msg) folders. F12 with cyrus
> and squirrel works very well with folders with 20000+ msg even on very
> slow (I tested it with VMware Player!!) machine. When I upgraded F12 to
> F13, time reading same imap folders increased dramatically.
> When I installed cyrus-imapd (2.4.6) from sources on the same F13
> installation, cyrus-imapd and squirrel tandem works again perfectly.

Thanks for clarifying. We've been runnning the cyrus-imapd/squirrelmail
combination for years without any real performance issues. We also ran it
with and without the up-imapproxy and what most people don't believe is
that it didn't really change much. The usage/load pattern changed
(different memory, cpu + I/O usage) but performance was always good, even
on folders with 50000+ messages. That was with ~1500 users, ~800
concurrent webmail users during the day and with ~500Gb used mailstorage.

Regards,
Simon

>
> Best regards,
>
> DS
>
> On 12/29/2010 12:38 AM, Bron Gondwana wrote:
>> On Tue, Dec 28, 2010 at 12:40:31PM +0000, Andre Felipe Machado wrote:
>>
>>> Hello,
>>> We actually used pidstat and iwatch to log disk activity of a given
>>> webmail
>>> client  upon our cyrus murder/aggregator.
>>>
>>> The access pattern was ...err...  "unwise", when compared with
>>> thunderbird, for
>>> example.
>>>
>>> The webmail client open account, then opens all folders, then lists all
>>> messages, for all folders, then reads entire content from all of them,
>>> one by
>>> one, then javascript order the entire list, then split the list in
>>> pages to
>>> render at screen...
>>> It can at least be configured to not read entire content from all of
>>> them, only
>>> headers, but still one by one...
>>> All these transactions caused disk writes (a "heavy" fs operation) to
>>> cache,
>>> index and header cyrus account files, and webservers and imap frontends
>>> load.
>>> When you have MANY msg into an inbox, this could be tragic. When you
>>> have many
>>> such simultaneous users, becomes a performance disaster.
>>> I suspect that other webmail clients have similar "unwise" access
>>> patterns.
>>>
>> Now that's just insane.  I'm pretty sure Squirrelmail doesn't do that -
>> or at
>> least it didn't back when I was working on improving its IMAP client
>> library.
>>
>>
>>> So, one should *aggressively* fine tune filesystem for small files
>>> writes/updates.
>>>
>> Why are reads causing a lot of writes?  The only thing that they should
>> be
>> triggering writes of is \Seen flags IFF the messages were not already
>> seen.
>>
>>
>>> The webmail clients could be redesigned to use wise imap command
>>> patterns. Could
>>> cyrus project publish some hints at site?
>>>
>> It might be worth publishing a "here's the most efficient way to do
>> stuff with
>> Cyrus" document.  The only thing which has really changed since Cyrus
>> 2.3 and
>> earlier is that "EXPUNGE" is now a very cheap operation.  We used to set
>> a flag
>> "medeleted" on messages deleted through our web interface, because a
>> very common
>> action was walking through the list of messages clicking "Delete and
>> Next" in the
>> web interface - and expunging every single time caused a lot of rewrite
>> IO.
>>
>> These days with 2.4, that was actually LESS efficient, because you got a
>> record
>> rewrite (plus modseq bump) for the flag, and then another one for the
>> expunge
>> later.  While with a straight expunge, you just got a single record
>> rewrite and
>> modseq bump.
>>
>> ----
>>
>> But there's nothing Cyrus can do about a client as insane as the one
>> you're
>> talking about above.  Slurping the ENTIRE state of the IMAP server at
>> every
>> connection.  Well, that's one way to make sure you can do all the
>> sorting
>> yourself with perfect information... I guess.
>>
>> Bron.
>> ----
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>
>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>