howto: set up a cyrus development environment

[ellie timoney]

Hi,

These instructions are mostly for Conrad, but they might be useful to
the greater list.  We should also clean these up and format them on the
wiki.

They are based on debian 8.0, cause that's what I was setting up as I
wrote it.  Other Linux distros will probably be similar in the broad
ideas but perhaps different in the specifics.  If you already have a
preferred distro, use that (I assume you know how to use its package
management system).  If you don't already have a preferred distro, maybe
consider using debian.

While writing this, I set it up in a virtual machine so that the
packages/etc don't cause any conflicts with my actual development
environment, which is debian 7.8.

For whatever reason, on debian 8.0 a handful of cassandane tests are
erroring (not failing) for the cyrus-imapd-2.5 branch, and rather more
are erroring for the master branch.  This isn't a problem for me on
debian 7.8, but I'm not yet sure of the cause (maybe I've just missed a
step in configuring cass, who knows).

Anyway, on with the instructions.

-- ellie

------------------------------------------------------------------


Setting up access to the cyrus-imapd git repository:

1. You'll need a public key.  If you don't already have a
~/.ssh/id_rsa.pub, then create one with ssh-keygen(1)

2. Login to Phabricator, and go to the SSH Keys settings panel:
https://git.cyrus.foundation/settings/panel/ssh/

3. Click "Upload public key", and paste the contents of your
~/.ssh/id_rsa.pub (n.b. NOT ~/.ssh/id_rsa!) into the public key box. 
Give it a descriptive name too.  Click Upload

4. Install git if you don't already have it:
   sudo apt-get install git

5. Clone the cyrus-imapd repository (you can get this URL from the
Diffusion app within Phabricator)
   git clone ssh://git@git.cyrus.foundation/diffusion/I/cyrus-imapd.git

Setting up dependencies:

1. Install tools for building
   sudo apt-get install build-essential autoconf automake libtool
   pkg-config bison flex valgrind 

2. Install dependencies for master branch
   sudo apt-get install libjansson-dev libxml2-dev libsqlite3-dev
   libical-dev libsasl2-dev libssl-dev libopendkim-dev libcunit1-dev
   libpcre3-dev

3. Additional dependencies for cyrus-imapd-2.5: you'll need the "-dev"
package to match whichever version of libdb you already have installed
(assuming it's probably already installed).  I needed "libdb5.3-dev" on
debian 8.0, but "libdb5.1-dev" on 7.8.

Compiling cyrus:

1.  cd /path/to/cyrus-imapd

2.  autoreconf -i -s   # generates a configure script, and its various
dependencies

3.  ./configure CFLAGS="-Wno-unused-parameter -g -O0 -Wall -Wextra
-Werror" --enable-coverage --enable-http --enable-unit-tests
--enable-replication --with-openssl=yes --enable-nntp --enable-murder
--enable-idled --prefix=/usr/cyrus

4.  make lex-fix   # you need this if compile fails with errors from
sieve/sieve.c

5.  make

6.  make check



Setting up cassandane:

1. Clone the cassandane repository (you can get the URL from the
Diffusion app within Phabricator)
  git clone ssh://git@git.cyrus.foundation/diffusion/C/cassandane.git

2. Install dependencies
  sudo apt-get install libtest-unit-perl libconfig-inifiles-perl
  libdatetime-perl libbsd-resource-perl libxml-generator-perl
  libencode-imaputf7-perl libio-stringy-perl libnews-nntpclient-perl
  libfile-chdir-perl

3. cd /path/to/cassandane

4. make

5. Copy cassandane.ini.example to cassandane.ini, and then edit
cassandane.ini to set up your cassandane environment.  Assuming you
configure cyrus with --prefix=/usr/cyrus (as above), then the defaults
are mostly fine, just set "destdir" to "/var/tmp/cyrus"

6. Create a "cyrus" user and matching group
   sudo adduser --system --group cyrus

7. Give your user account access to sudo as cyrus
  sudo visudo
  # add a line like:
    username ALL = (cyrus) NOPASSWD: ALL
  # where "username" is your own username

8. Make the destdir directory, as the cyrus user
  sudo -u cyrus mkdir /var/tmp/cass

Building cyrus-imapd for cassandane:

1.  cd /path/to/cyrus-imapd

2.  (compile as above)

3.  make DESTDIR=/var/tmp/cyrus install

Running cassandane tests:

1.  cd /path/to/cassandane

2.  ./testrunner.pl -f pretty -j 8   # read the script to see other
options



Setting up Arcanist:

Arcanist is a tool for managing workflow (code review, etc), which sits
between git and Phabricator

1. Install arcanist: https://git.cyrus.foundation/w/installing_arcanist/

2. Set up arcanist: https://git.cyrus.foundation/w/setting_up_arcanist/

3. Get familiar with the arcanist workflow:
https://git.cyrus.foundation/w/arcanist_workflow/



Setting up syslog:

A lot of cyrus's debugging information gets logged with syslog, so
you'll want to be able to capture it and find it later (especially when
debugging cassandane tests)

1.  Find the correct place to edit syslog config for your system (for
me, I needed to create a /etc/rsyslog.d/cyrus.conf)

2.  Add lines like:
  local6.*        /var/log/imapd.log
  auth.debug      /var/log/auth.log

3.  Restart the rsyslog service
  sudo /etc/init.d/rsyslog restart

4.  Arrange to rotate the imapd.log so it doesn't get stupendously large
  sudo vim /etc/logrotate.conf
  # add lines like:

/var/log/imapd.log
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}