Xapian partition definition when using archiving?

Nic Bernstein nic at onlight.com
Wed Dec 20 09:05:10 EST 2017


Bron,
Thanks for the swift reply.  So if I understand this correctly, the 
"archivesearchpartition-default" is named such because it's for the 
archive location of Xapian search data, not because it's Xapian search 
data from an Archive partition.  Is that correct?  In other words, this 
is just another circumstance where seemingly obvious partition names 
(like default-default) get us into documentation trouble.  Right?

Thanks again,
     -nic

On 12/20/2017 05:39 AM, Bron Gondwana wrote:
> Hi Nic,
>
> The Xapian partitions are entirely separate from archive partitions.  
> The indexing code will find the file in the correct location if it 
> needs to read it.
>
> Here's an example from one of our servers:
>
> defaultpartition: default
> defaultsearchtier: temp
> partition-default: /mnt/ssd21d2/sloti21d2t40/store254/spool
> tempsearchpartition-default: /tmpfs-search/sloti21d2t40
> metasearchpartition-default: /mnt/ssd21d2/sloti21d2t40/store254/search
> datasearchpartition-default: /mnt/i21d2search/sloti21d2t40/store254/search
> archivesearchpartition-default: 
> /mnt/i21d2search/sloti21d2t40/store254/search-archive
> archivepartition-default: 
> /mnt/i21d2t40/sloti21d2t40/store254/spool-archive
>
> (clearly auto-generated!)
>
> Bron.
>
>
> On Wed, 20 Dec 2017, at 07:32, Nic Bernstein wrote:
>> Bron, et al.,
>> We're about to set up a whole new bunch of partitions in support of 
>> Xapian indexing, for a 2.5.10-to-3.0.4 upgrade, and then will be 
>> introducing archive functionality, too.  How do archive partitions 
>> and Xapian partitions interact?
>>
>> For example, the server currently has the following in imapd.conf:
>>
>>     defaultpartition: default
>>     partition-default: /var/mailstores/default
>>     metapartition-default: /var/imapmeta/default
>>     partition-1: /var/mailstores/1
>>     partition-2: /var/mailstores/2
>>     partition-3: /var/mailstores/3
>>     partition-4: /var/mailstores/4
>>     ...
>>     partition-29: /var/mailstores/29
>>     partition-30: /var/mailstores/30
>>     partition-100: /var/mailstores/100
>>     partition-temp: /var/mailstores/temp
>>     ...
>>     # non-default metapartitions
>>     metapartition-1: /var/imapmeta/1
>>     metapartition-2: /var/imapmeta/2
>>     metapartition-3: /var/imapmeta/3
>>     metapartition-4: /var/imapmeta/4
>>     ...
>>     metapartition-29: /var/imapmeta/29
>>     metapartition-30: /var/imapmeta/30
>>     metapartition-100: /var/imapmeta/100
>>     metapartition-temp: /var/imapmeta/temp
>>
>> Going by the documentation, which I wrote with help from you good 
>> folk at Fastmail, the Archive partition scheme might look something 
>> like this:
>> https://www.cyrusimap.org/imap/reference/admin/locations/archive-partitions.html
>>
>>     archivepartition-default: /var/mailarchives/default
>>     archivepartition-1: /var/mailarchives/1
>>     archivepartition-2: /var/mailarchives/2
>>     archivepartition-3: /var/mailarchives/3
>>     archiveartition-4: /var/mailarchives/4
>>     ...
>>     archivepartition-29: /var/mailarchives/29
>>     archivepartition-30: /var/mailarchives/30
>>     archivepartition-100: /var/mailarchives/100
>>     archivepartition-temp: /var/mailarchives/temp
>>
>> And the Xapian partition structure to mate with this would look 
>> something like this (again, from the docs):
>> https://www.cyrusimap.org/imap/developer/install-xapian.html
>>
>>     defaultpartition: default
>>     partition-default: /var/mailstores/default
>>     search_engine:xapian
>>     search_index_headers: no
>>     search_batchsize: 8192
>>     defaultsearchtier: t1
>>     1searchtier: t1
>>     2searchtier: t1
>>     3searchtier: t1
>>     4searchtier: t1
>>     ...
>>     29searchtier: t1
>>     30searchtier: t1
>>     100searchtier: t1
>>     tempsearchtier: t1
>>     ...
>>     t1searchpartition-default: /var/search/default
>>     t1searchpartition-1: /var/search/1
>>     t1searchpartition-2: /var/search/2
>>     t1searchpartition-3: /var/search/3
>>     t1searchpartition-4: /var/search/4
>>     ...
>>     t1searchpartition-29: /var/search/29
>>     t1searchpartition-30: /var/search/30
>>     t1searchpartition-100: /var/search/100
>>     t1searchpartition-temp: /var/search/temp
>>
>> First question, since there's no examples to work from; Is this 
>> Xapian layout correct?
>>
>> Do I need to define & create Xapian partitions for the metadata 
>> partitions, as is indirectly implied in Bron's original email on this 
>> topic:
>> https://lists.tartarus.org/pipermail/xapian-discuss/2014-October/009112.html
>>
>> Also, how do these Xapian and Archive, interact?  Do I need to add a 
>> separate Xapian partition for each Archive partition, or will the 
>> Archive partition be treated like a child of the non-Archive 
>> partition?  (again, implied but not directly addressed in that email).
>>
>> Any guidance gladly accepted, and whatever I learn will be repackaged 
>> into more complete documentation on same.
>>
>> Cheers,
>>     -nic
>>
>> -- 
>> Nic Bernsteinnic at onlight.com <mailto:nic at onlight.com>
>> Onlight, Inc.www.onlight.com <http://www.onlight.com>
>> 6525 W Bluemound Road, Suite 24           v. 414.272.4477
>> Milwaukee, Wisconsin  53213-4073
>>
>> Email had 1 attachment:
>>
>>  *
>>     |nic.vcf|
>>       1k (text/x-vcard)
>>
>
> --
>   Bron Gondwana
>   brong at fastmail.fm
>
>

-- 
Nic Bernstein                             nic at onlight.com
Onlight Inc.                              www.onlight.com
6525 W Bluemound Rd., Ste 24	          v. 414.272.4477
Milwaukee, Wisconsin  53213-4073	  f. 414.290.0335

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20171220/2b5fc36a/attachment-0001.html>


More information about the Cyrus-devel mailing list