upgrade 2.0.16 to 2.1.16 - ctl_cyrusdb hangs

[Paul Boven]

Hi Sam,

Sam Smith wrote:
> The upgrade seemed to go OK, but when I start the master process it 
> starts ctl_cyrusdb. I get the following messages:
> 
> Dec 17 14:16:05 newimap.ece.gatech.edu master[9932]: [ID 392559 
> local6.debug] about to exec /usr/cyrus/bin/ctl_cyrusdb
> Dec 17 14:16:05 newimap.ece.gatech.edu ctl_cyrusdb[9932]: [ID 702911 
> local6.notice] recovering cyrus databases
> 
> and that's all. ctl_cyrusdb never seems to finish recovering, imapd 
> never starts, etc.
> 
> We do have over 100G of email... Should ctl_cyrusdb take hours?

Take a look at your top output, and see if any IO or CPU activity is 
happening. If not, take a truss (Solaris) or strace (?) (Linux) of the 
ctl_cyrusdb process. Something that has been plagueing me is that 
ctl_cyrusdb runs out of filedescriptor because it opens the same file 
about 250 times. So if your truss output shows poll(), and EMFILE, that 
might be your problem.
If, and only if that is the case, you might be helped by removing the 
deliver-db, the tls_sessions db and all the snapshots. However, make 
sure you make backups first! I haven't yet had this problem again on 
2.1.16, but I did the first time I started 2.1.16 using my 'old' databases.
Does anyone else experience the 'out of file descriptors (EMFILE)' problem?

Regards, Paul Boven.