PostgreSQL backend: a waste of time?

[Nicola Ranaldo]

I use cyrus since version 2, on Linux Slackware and True64 from 4.0f to
5.1a. There are no big problems on Intel architecture, but every upgrade on
alpha was a war! For example since version 2.1.5 to compile it you have to
manually define HAVE_GETADDRINFO and in mpool.c you would apply:

(Thanks to Davide Bottalico, my great coworker!)
128c128
< #define ROUNDUP(num) (((num) + 15) & 0xFFFFFFFFFFFFFFF0)
---
> #define ROUNDUP(num) (((num) + 15) & 0xFFFFFFF0)
164c164
<     p->ptr = (void *)ROUNDUP((unsigned long)p->ptr + size);
---
>     p->ptr = (void *)ROUNDUP((unsigned int)p->ptr + size);

64bits!!!

Due to continous imapd-freeze in our last upgrade (2.1.9 bdb 4.0.14) I
decided to move to a definitively stable backend (as said in a previous
message PostgreSQL is very stable on True64). This is not so easy as it
seems and could create two main problems, if you will be patient I'll show
them:

1) A single connection to PostgreSQL allows you to exec as many transactions
as you want, but they cannot be nested or interleaved, and they must be
executed in sequential order (CONNECT, BEGIN SELECT, INSERT etc. COMMIT,
BEGIN ... COMMIT... DISCONNECT). To have indipendent transactions to the
same DB the same process should open as many connections as the number of
transactions. This could waste resources!
But it seems cyrus transactions are isolated in little atomic context and do
not involve different DBs , so I can have only a unique connection to
PostgreSQL for every master child that serves the process for all of its
life, all dbfile become different tables of the only PostgreSQL db!
This *now* works fine, my code logs all these problems, and on about
50megabytes of cyrus.log I had no problems.
If cyrus code will change to exec parallel transactions the actual code will
be rewritten (if it will be safe and useful).

2) I cannot shutdown a BerkeleyDB, so a fatal error in mboxlist_init is
really fatal! But I can shutdown a PostgreSQL process for mantenaince
pourposes, or it's unavailable because it has reached max number of
concurrent
connections. In both cases master childs exit with fatal error, and when
PostgreSQL will be available again there will be no listener on sockets!

 in imapd.c I read:

/*
 * run once when process is forked;
 * MUST NOT exit directly; must return with non-zero error code
 */
int service_init(int argc, char **argv, char **envp)

but mboxlist_init() is a void! it calls directly fatal() and so exit().

Master will not check exit code of its children and so does not reset the
number of ready workers, so if another connection arrives no process will
serve it!
So if PostgreSQL goes down i have to restart cyrus!

I know how to patch this in master.c but in case of fatal error, it will
bring up a huge number of sequential forks until PostgreSQL is available
again. So other changes will be necessary, but i think that's not up to me.

Nicola Ranaldo


> I have the same point of view, could you please share with me your
> experiences with cyrus, and a more detailed information about your
project?
>
> Nicola Ranaldo wrote:
>
> >Due to our historical problems using BerkeleyDB4 over True64Unix I'm
coding
> >a
> >PostgreSQL backend.