pop3d proxy
Rob Siemborski
rjs3 at andrew.cmu.edu
Tue Mar 16 11:23:59 EST 2004
What is the pop3proxy process doing during all of this?
Getting the actual times the syscalls take would be helpful too. (-T to
strace)
On Mon, 15 Mar 2004, Michael J Farina wrote:
> Ok that worked well. Thanks. It seems to be hanging on a socket read. It
> looks up the address from /etc/host then wait for the connections to start.
> then it flies really fast. here is a blurb.
>
>
> read(3, "# Begin /etc/hosts (network card"..., 4096) = 231
> close(3) = 0
> munmap(0x40365000, 4096) = 0
> socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
> connect(3, {sa_family=AF_INET, sin_port=htons(110),
> sin_addr=inet_addr("63.xxx.xxx.29")}, 16) = 0
> getpeername(3, {sa_family=AF_INET, sin_port=htons(110),
> sin_addr=inet_addr("63.xxx.xxx.29")}, [16]) = 0
> getsockname(3, {sa_family=AF_INET, sin_port=htons(32775),
> sin_addr=inet_addr("63.xxx.xxx.6")}, [16]) = 0
> brk(0x8066000) = 0x8066000
> uname({sys="Linux", node="poptester", ...}) = 0
> brk(0x8068000) = 0x8068000
> read(3,
>
> Kind of sits right there for a while then it takes off like a run away train
> for a few then stops at that point again.
>
>
>
>
> ----- Original Message -----
> From: "Ken Murchison" <ken at oceana.com>
> To: "Michael J Farina" <mfarina at cvzoom.net>
> Cc: <info-cyrus at lists.andrew.cmu.edu>
> Sent: Monday, March 15, 2004 4:08 PM
> Subject: Re: pop3d proxy
>
>
> > Michael J Farina wrote:
> >
> > > I have the backend server's hostname in /etc/hosts that should make the
> > > lookup fast and pop3test doesn't seem to take a password on the command
> line
> > > for rapid fire logins.
> >
> > -w <passwd>
> >
> > > I don't see how I could strace the pop3proxy. I would
> > > be glad to do so if you have any suggestions.
> >
> >
> >
> > > ----- Original Message -----
> > > From: "Ken Murchison" <ken at oceana.com>
> > > To: "Michael J Farina" <mfarina at cvzoom.net>
> > > Cc: <info-cyrus at lists.andrew.cmu.edu>
> > > Sent: Monday, March 15, 2004 3:20 PM
> > > Subject: Re: pop3d proxy
> > >
> > >
> > >
> > >>Michael J Farina wrote:
> > >>
> > >>
> > >>>I am still chasing this pop3 problem. I have now tried PLAIN+TLS for
> > >
> > > proxy
> > >
> > >>>authentication. I see the same issue of pop3 proxy being slow. With
> more
> > >>>investigation I see that I can open alot of connections on the frontend
> > >>>server but they don't see to get passed to the backend. I have tried
> > >>>starting master with a -l 1024 and in my cyrus.conf I have preforked 50
> > >>>pop3d and maxchild=-1. Nothing seems to make the connection counts
> > >
> > > greater.
> > >
> > >>>Could I be hitting some max connection per IP limit that I can't see
> > >
> > > when I
> > >
> > >>>connect directly to the backend?
> > >>
> > >>I still think this is network related. Is it possible that DNS is slow?
> > >> Can you strace/truss the process to see what it is doing? What
> > >>happens if you use pop3test against the proxy, specifying the number of
> > >>logins with the -n option.
> > >>
> > >>
> > >>>
> > >>>----- Original Message -----
> > >>>From: "Michael J Farina" <mfarina at cvzoom.net>
> > >>>To: "Ken Murchison" <ken at oceana.com>
> > >>>Cc: <info-cyrus at lists.andrew.cmu.edu>
> > >>>Sent: Friday, March 12, 2004 1:13 PM
> > >>>Subject: Re: pop3d proxy
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>>It seems that the delay is in the connection. I have played with the
> > >>>
> > >>>backlog
> > >>>
> > >>>
> > >>>>queue, the max child, and prefork settings without being able to make
> it
> > >>>
> > >>>any
> > >>>
> > >>>
> > >>>>faster. I am logging in a doing a list command. The mailboxes are all
> > >>>
> > >>>empty
> > >>>
> > >>>
> > >>>>so I am not thinking that once the connection is authed it is slow. It
> > >
> > > is
> > >
> > >>>>just getting the frontend to open enough connections to the backend. I
> > >>>
> > >>>can't
> > >>>
> > >>>
> > >>>>seem to nail down what the problem is. If I connect the stress test to
> > >
> > > the
> > >
> > >>>>backend it is very fast less than a second a connection. If I can
> > >
> > > provide
> > >
> > >>>>anymore info let me know. I am looking to have over 150,000 boxes they
> > >
> > > are
> > >
> > >>>>all pop now but I want to offer IMAP.
> > >>>>
> > >>>>--Mike
> > >>>>
> > >>>>
> > >>>>----- Original Message -----
> > >>>>From: "Ken Murchison" <ken at oceana.com>
> > >>>>To: "Michael J Farina" <mfarina at cvzoom.net>
> > >>>>Cc: <info-cyrus at lists.andrew.cmu.edu>
> > >>>>Sent: Friday, March 12, 2004 9:54 AM
> > >>>>Subject: Re: pop3d proxy
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>>Michael J Farina wrote:
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>>Ken,
> > >>>>>>
> > >>>>>> Is there a way to do PLAIN from the frontends to the backends?
> The
> > >>>>>>servers I am testing are on a isolated network no network traffic
> > >>>
> > >>>other
> > >>>
> > >>>
> > >>>>than
> > >>>>
> > >>>>
> > >>>>>>the mail servers. Any help would be appreciated.
> sasl_maximum_layer:0
> > >>>>
> > >>>>didn't
> > >>>>
> > >>>>
> > >>>>>>seem to change the pop3 proxy timing. The server are also dual 1.4s
> > >>>
> > >>>with
> > >>>
> > >>>
> > >>>>2
> > >>>>
> > >>>>
> > >>>>>>gigs of ram if that helps at all.
> > >>>>>
> > >>>>>You can do PLAIN, but the frontends will also need to use TLS, since
> > >>>>>PLAIN won't be advertised by the backends unless a security layer is
> in
> > >>>>>place.
> > >>>>>
> > >>>>>Are you saying that there is a delay for every command, or for every
> > >>>>>login? Is your stress test only doing one command per login?
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>>----- Original Message -----
> > >>>>>>From: "Ken Murchison" <ken at oceana.com>
> > >>>>>>To: "Michael J Farina" <mfarina at cvzoom.net>
> > >>>>>>Cc: <info-cyrus at lists.andrew.cmu.edu>
> > >>>>>>Sent: Thursday, March 11, 2004 4:58 PM
> > >>>>>>Subject: Re: pop3d proxy
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>Michael J Farina wrote:
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>I am setting up a cyrus imap murder for a large number of
> accounts.
> > >>>
> > >>>In
> > >>>
> > >>>
> > >>>>>>my
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>stress testing I have noticed that the pop3 proxy has quite a
> delay.
> > >>>>>>
> > >>>>>>When I
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>connect directly to a backend I can check mail at a very fast rate
> > >>>>
> > >>>>about
> > >>>>
> > >>>>
> > >>>>>>.5
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>secs a mailbox. When I connect to the proxy the wait time
> increases
> > >
> > > a
> > >
> > >>>>>>ton to
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>about 5 secs. I am thinking I must have a configuration problem. I
> > >>>
> > >>>will
> > >>>
> > >>>
> > >>>>>>>>include my imapd.conf and my cyrus.conf I am using cyrus 2.2.3.
> > >>>>>>>
> > >>>>>>>Once you authenticate and the mailbox is opened on the backend, the
> > >>>>>>>proxy just acts as a bitpipe between the client and backend, so
> there
> > >>>>>>>should be very little overhead.
> > >>>>>>>
> > >>>>>>>You *might* be seeing a slowdown because the frontend is
> > >>>
> > >>>authenticating
> > >>>
> > >>>
> > >>>>>>>to the backend using DIGEST-MD5 with a security layer or TLS+PLAIN.
> > >>>
> > >>>In
> > >>>
> > >>>
> > >>>>>>>either case you have encrypt/decrypt overhead.
> > >>>>>>>
> > >>>>>>>If using DIGEST-MD5, you could try setting sasl_maximum_layer:0 on
> > >
> > > the
> > >
> > >>>>>>>backend, which will prevent the security layer from being
> negotiated.
> > >>>>>>>
> > >>>>>>>Otherwise, the only other delay that I could think of would be
> > >
> > > network
> > >
> > >>>>>>>related.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>Frontend cyrus.conf
> > >>>>>>>>
> > >>>>>>>>START {
> > >>>>>>>>mboxlist cmd="ctl_cyrusdb -r"
> > >>>>>>>>}
> > >>>>>>>>
> > >>>>>>>># UNIX sockets start with a slash and are put into
> /var/imap/sockets
> > >>>>>>>>SERVICES {
> > >>>>>>>># mupdate database service - must prefork atleast 1
> > >>>>>>>>mupdate cmd="/usr/cyrus/bin/mupdate -m" listen=3905
> prefork=1
> > >>>>>>>>
> > >>>>>>>># add or remove based on preferences
> > >>>>>>>>imap cmd="proxyd" listen="imap" prefork=5
> > >>>>>>>>pop3 cmd="pop3proxyd" listen="pop3" prefork=5 maxchild=-1
> > >>>>>>>>sieve cmd="timsieved" listen="sieve" prefork=0
> > >>>>>>>>
> > >>>>>>>>lmtp cmd="lmtpproxyd" listen="/var/imap/socket/lmtp"
> > >>>>>>
> > >>>>>>prefork=0
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>}
> > >>>>>>>>
> > >>>>>>>>EVENTS {
> > >>>>>>>>checkpoint cmd="ctl_cyrusdb -c" period=5
> > >>>>>>>>}
> > >>>>>>>>
> > >>>>>>>>Frontend imapd.conf
> > >>>>>>>>
> > >>>>>>>>configdirectory: /var/imap
> > >>>>>>>>partition-default: /tmp
> > >>>>>>>>admins: cyrus
> > >>>>>>>>sasl_pwcheck_method: saslauthd
> > >>>>>>>>allowplaintext: 1
> > >>>>>>>>mupdate_admin: murder
> > >>>>>>>>mupdate_server: frontend
> > >>>>>>>>mupdate_password: murder pass
> > >>>>>>>>proxy_authname: proxy
> > >>>>>>>>proxy_password: proxy pass
> > >>>>>>>>unixhierarchysep: 1
> > >>>>>>>>sasl_mech_list: PLAIN
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>Backend cyrus.conf
> > >>>>>>>>
> > >>>>>>>>START {
> > >>>>>>>>recover cmd="ctl_cyrusdb -r"
> > >>>>>>>>mupdatepush cmd="ctl_mboxlist -m"
> > >>>>>>>>}
> > >>>>>>>>
> > >>>>>>>># UNIX sockets start with a slash and are put into
> /var/imap/sockets
> > >>>>>>>>SERVICES {
> > >>>>>>>># add or remove based on preferences
> > >>>>>>>>imap cmd="imapd" listen="imap" prefork=5
> > >>>>>>>>pop3 cmd="pop3d" listen="pop3" prefork=0 maxchild=-1
> > >>>>>>>>
> > >>>>>>>># at least one LMTP is required for delivery
> > >>>>>>>>lmtp cmd="lmtpd" listen="lmtp" prefork=2
> > >>>>>>>>
> > >>>>>>>>}
> > >>>>>>>>EVENTS {
> > >>>>>>>># this is required
> > >>>>>>>>checkpoint cmd="ctl_cyrusdb -c" period=5
> > >>>>>>>>}
> > >>>>>>>>
> > >>>>>>>>Backend imapd.conf
> > >>>>>>>>
> > >>>>>>>>configdirectory: /var/imap
> > >>>>>>>>partition-default: /var/spool/imap
> > >>>>>>>>admins: cyrus
> > >>>>>>>>mupdate_server: frontend
> > >>>>>>>>mupdate_authname: murder
> > >>>>>>>>mupdate_password: murder pass
> > >>>>>>>>
> > >>>>>>>>allowplaintext: yes
> > >>>>>>>>sasl_pwcheck_method: saslauthd
> > >>>>>>>>sasl_mech_list: PLAIN DIGEST-MD5
> > >>>>>>>>proxyservers: proxy
> > >>>>>>>>unixhierarchysep: 1
> > >>>>>>>>defaultacl: anyone rsd
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>---
> > >>>>>>>>Home Page: http://asg.web.cmu.edu/cyrus
> > >>>>>>>>Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
> > >>>>>>>>List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>--
> > >>>>>>>Kenneth Murchison Oceana Matrix Ltd.
> > >>>>>>>Software Engineer 21 Princeton Place
> > >>>>>>>716-662-8973 x26 Orchard Park, NY 14127
> > >>>>>>>--PGP Public Key-- http://www.oceana.com/~ken/ksm.pgp
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>---
> > >>>>>>Home Page: http://asg.web.cmu.edu/cyrus
> > >>>>>>Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
> > >>>>>>List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>--
> > >>>>>Kenneth Murchison Oceana Matrix Ltd.
> > >>>>>Software Engineer 21 Princeton Place
> > >>>>>716-662-8973 x26 Orchard Park, NY 14127
> > >>>>>--PGP Public Key-- http://www.oceana.com/~ken/ksm.pgp
> > >>>>>---
> > >>>>>Home Page: http://asg.web.cmu.edu/cyrus
> > >>>>>Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
> > >>>>>List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>
> > >>--
> > >>Kenneth Murchison Oceana Matrix Ltd.
> > >>Software Engineer 21 Princeton Place
> > >>716-662-8973 x26 Orchard Park, NY 14127
> > >>--PGP Public Key-- http://www.oceana.com/~ken/ksm.pgp
> > >>---
> > >>Home Page: http://asg.web.cmu.edu/cyrus
> > >>Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
> > >>List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> >
> >
> > --
> > Kenneth Murchison Oceana Matrix Ltd.
> > Software Engineer 21 Princeton Place
> > 716-662-8973 x26 Orchard Park, NY 14127
> > --PGP Public Key-- http://www.oceana.com/~ken/ksm.pgp
> >
> >
> >
> >
>
>
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456
Research Systems Programmer * /usr/contributed Gatekeeper
---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
More information about the Info-cyrus
mailing list