Problems with idle pop3 connections locking mailboxes

[Michael Sims]

I'm having a strange problem with pop3d connections hanging that I was hoping to get
some feedback on (system vitals are at the bottom of this message).  This is usually
reported to me by end users that use both POP3 and our webmail interface (which uses
IMAP).  In the webmail interface they attempt to delete a message, and which time it
is marked for deletion, but the expunge fails with the error "Mailbox is locked by
POP server".  This has been happening more frequently in the past few weeks.  I can
always fix the problem by restarting Cyrus, but of course I'd rather not have to
start doing that regularly.

Today I had a user report this problem and I did some investigating.  Here is what
I'm seeing:

1. Using lsof I find a pop3d process that has this users's cyrus header files open,
as well as one particular message.
2. The output of netstat shows that this process has a pop3 TCP connection in
ESTABLISHED mode to the user's computer.
3. Checking /var/log/imapd.log reveals that the offending pop3d process accepted a
login from this user over two days ago.
4. Running strace on the offending process gives the following output:

[root at mail root]# strace -p23604
Process 23604 attached - interrupt to quit
write(1, "/mYYyaenY9KvRwAahzCYCiPwGiWHtz7A"..., 584

5. I've installed Ethereal and have been sniffing the traffic to this particular IP.
Every 2 minutes on the dot my server sends a TCP Keep-Alive to this IP address and a
split second later receives an ACK.  For both packets the sequence number is zero,
as well as the length.

I've had this user reboot their PC, yet the connection remains ESTABLISHED and
Ethereal continues to capture traffic every 2 minutes.  My logs show that this user
has connected to the server many times since the original connection with a
completely different IP address.  I find it very difficult to believe that there is
anything on the user's computer that is still listening to the connection.

I've read the thread in the archive "idle pop3d never times out" [1] which seems
similar to what I'm seeing.  In this thread it seems the poster never found a
solution and had to resort to periodically killing old pop3d processes.  I really
hope I don't have to start doing that, because this hasn't been that much of a
problem before...in fact I ran various versions of Cyrus for months without this
ever coming up, but now it seems to come up every other week.

I'm running RHEL 2.1 ES (kernel 2.4.9-e.27smp) and ouput from cyradm ver is:

name       : Cyrus IMAPD
version    : v2.1.16 2003/11/19 16:45:28
vendor     : Project Cyrus
support-url: http://asg.web.cmu.edu/cyrus
os         : Linux
os-version : 2.4.9-e.27smp
environment: Cyrus SASL 2.1.15
             Sleepycat Software: Berkeley DB 3.3.11: (July 12, 2001)
             Built w/OpenSSL 0.9.6b [engine] 9 Jul 2001
             Running w/OpenSSL 0.9.6b [engine] 9 Jul 2001
             CMU Sieve 2.2
             TCP Wrappers
             mmap = shared
             lock = fcntl
             nonblock = fcntl
             auth = unix
             idle = idled
             mailboxes.db = skiplist
             seen.db = skiplist
             subs.db = flat
             deliver.db = skiplist
             tls_sessions.db = db3-nosync

Does anyone have any ideas/suggestions?  Could this be an OS problem?  I upgraded to
RHEL 2.1 about 3 months ago, and I don't recall this ever happening before that
upgrade.  Is it possible that the kernel believes a connection is present when there
really isn't one?  Any help would be greatly appreciated...

[1]  http://marc.theaimsgroup.com/?l=info-cyrus&m=101880199104191&w=2

___________________________________________
Michael Sims
Project Analyst - Information Technology
Crye-Leike Realtors
Office: (901)758-5648  Pager: (901)769-3722
___________________________________________

---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html