Disable creation of hardlinks in message store
Ken Murchison
ken at oceana.com
Thu Jun 2 10:34:10 EDT 2005
Simon Matter wrote:
>>I've tried to get rid of hardlinked files in our cyrus spools to make
>>partial mailbox restores easier and more safe. My first idea was to set
>>"singleinstancestore: no" in imapd.conf and make all payload files
>>independant from each other. Unfortunately I realized later that while
>>incoming messages are stored in independant files, the IMAP COPY command
>>still creates hardlinked message files. Of course there is nothing wrong
>>with singleinstancestore because the manpage clearly states that only
>>delivery via lmtp/nntp is affected.
>>I have now looked at the code and found that mailbox_copyfile() has a
>>nolink parameter which controls the copy/link behaviour. My idea was to
>>create a new config option to disable hardlinks completely.
>>
>>My questions:
>>- is such an option a very bad idea?
>>- does such a patch already exist somewhere?
>>- what's the best name of a new config option?
>>- is there a chance to have such a patch accepted into the distribution?
>
>
> I'd like to include the following patch into my cyrus-imapd rpm packages
> to address the issue mentioned above. While testing it on a test box it
> seemed to work very well. Do the cyrus developers see any possible problem
> with it?
Actually, your patch affects ALL mailbox_copyfile(), which means that
messages won't even be hardlinked from the stage./ to the destination
mailbox. I don't think we want to do this, since this will hurt
performance for all messages deliveries (even single recipient messages).
IMO, singleinstancestore *should* also govern IMAP COPY (does anyone
object to this assessment?), so I'd propose the following patch:
Index: imap/append.c
===================================================================
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/append.c,v
retrieving revision 1.107
diff -u -r1.107 append.c
--- imap/append.c 22 May 2004 03:45:48 -0000 1.107
+++ imap/append.c 2 Jun 2005 14:27:39 -0000
@@ -801,7 +801,8 @@
int append_copy(struct mailbox *mailbox,
struct appendstate *as,
int nummsg,
- struct copymsg *copymsg)
+ struct copymsg *copymsg,
+ int nolink)
{
struct mailbox *append_mailbox = &as->m;
int msg;
@@ -845,7 +846,7 @@
mailbox_message_get_fname(mailbox, copymsg[msg].uid, fnamebuf,
sizeof(fnamebuf));
/* Link/copy message file */
- r = mailbox_copyfile(fnamebuf, fname, 0);
+ r = mailbox_copyfile(fnamebuf, fname, nolink);
if (r) goto fail;
/* Write out cache info, copy other info */
Index: imap/append.h
===================================================================
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/append.h,v
retrieving revision 1.26
diff -u -r1.26 append.h
--- imap/append.h 22 Jan 2004 21:17:07 -0000 1.26
+++ imap/append.h 2 Jun 2005 14:27:39 -0000
@@ -137,7 +137,7 @@
extern int append_copy(struct mailbox *mailbox,
struct appendstate *append_mailbox,
- int nummsg, struct copymsg *copymsg);
+ int nummsg, struct copymsg *copymsg, int nolink);
extern int append_collectnews(struct appendstate *mailbox,
const char *group, unsigned long feeduid);
Index: imap/imapd.c
===================================================================
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/imapd.c,v
retrieving revision 1.492
diff -u -r1.492 imapd.c
--- imap/imapd.c 27 May 2005 14:57:41 -0000 1.492
+++ imap/imapd.c 2 Jun 2005 14:27:39 -0000
@@ -3605,7 +3605,7 @@
imapd_userid, mailboxname);
if (!r) {
r = index_copy(imapd_mailbox, sequence, usinguid, mailboxname,
- ©uid);
+ ©uid,
!config_getswitch(IMAPOPT_SINGLEINSTANCESTORE))
;
}
index_check(imapd_mailbox, usinguid, 0);
Index: imap/imapd.h
===================================================================
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/imapd.h,v
retrieving revision 1.61
diff -u -r1.61 imapd.h
--- imap/imapd.h 22 Jun 2004 21:36:18 -0000 1.61
+++ imap/imapd.h 2 Jun 2005 14:27:39 -0000
@@ -249,7 +249,7 @@
extern int index_thread(struct mailbox *mailbox, int algorithm,
struct searchargs *searchargs, int usinguid);
extern int index_copy(struct mailbox *mailbox, char *sequence,
- int usinguid, char *name, char **copyuidp);
+ int usinguid, char *name, char **copyuidp, int
nolink);
extern int index_status(struct mailbox *mailbox, char *name,
int statusitems);
Index: imap/index.c
===================================================================
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/index.c,v
retrieving revision 1.217
diff -u -r1.217 index.c
--- imap/index.c 27 May 2005 14:57:55 -0000 1.217
+++ imap/index.c 2 Jun 2005 14:27:39 -0000
@@ -1155,7 +1155,8 @@
char *sequence,
int usinguid,
char *name,
- char **copyuidp)
+ char **copyuidp,
+ int nolink)
{
static struct copyargs copyargs;
int i;
@@ -1188,7 +1189,7 @@
docopyuid = (append_mailbox.m.myrights & ACL_READ);
r = append_copy(mailbox, &append_mailbox, copyargs.nummsg,
- copyargs.copymsg);
+ copyargs.copymsg, nolink);
if (!r) append_commit(&append_mailbox, totalsize,
&uidvalidity, &startuid, &num);
if (!r && docopyuid) {
Index: lib/imapoptions
===================================================================
RCS file: /afs/andrew/system/cvs/src/cyrus/lib/imapoptions,v
retrieving revision 1.32
diff -u -r1.32 imapoptions
--- lib/imapoptions 11 Apr 2005 06:57:31 -0000 1.32
+++ lib/imapoptions 2 Jun 2005 14:27:40 -0000
@@ -776,9 +776,9 @@
directories: ~user/.sieve. */
{ "singleinstancestore", 1, SWITCH }
-/* If enabled, lmtpd and nntpd attempt to only write one copy of a
message per
- partition and create hard links, resulting in a potentially large
- disk savings. */
+/* If enabled, imapd, lmtpd and nntpd attempt to only write one copy
+ of a message per partition and create hard links, resulting in a
+ potentially large disk savings. */
{ "skiplist_unsafe", 0, SWITCH }
/* If enabled, this option forces the skiplist cyrusdb backend to
--
Kenneth Murchison Oceana Matrix Ltd.
Software Engineer 21 Princeton Place
716-662-8973 x26 Orchard Park, NY 14127
--PGP Public Key-- http://www.oceana.com/~ken/ksm.pgp
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
More information about the Info-cyrus
mailing list