Xapian partition definition when using archiving?

[Bron Gondwana]

Totally!  The name "archive" is overused.  It could be called something
else easily enough.
Bron.


On Thu, 21 Dec 2017, at 01:05, Nic Bernstein wrote:
> Bron,
>  Thanks for the swift reply.  So if I understand this correctly, the
>  "archivesearchpartition-default" is named such because it's for the
>  archive location of Xapian search data, not because it's Xapian
>  search data from an Archive partition.  Is that correct?  In other
>  words, this is just another circumstance where seemingly obvious
>  partition names (like default-default) get us into documentation
>  trouble.  Right?> 
>  Thanks again,
>      -nic
> 
> 
> On 12/20/2017 05:39 AM, Bron Gondwana wrote:
>> Hi Nic,
>> 
>> The Xapian partitions are entirely separate from archive partitions.
>> The indexing code will find the file in the correct location if it
>> needs to read it.>> 
>> Here's an example from one of our servers:
>> 
>> defaultpartition: default
>> defaultsearchtier: temp
>> partition-default: /mnt/ssd21d2/sloti21d2t40/store254/spool
>> tempsearchpartition-default: /tmpfs-search/sloti21d2t40
>> metasearchpartition-default:
>> /mnt/ssd21d2/sloti21d2t40/store254/search>> datasearchpartition-default:
>> /mnt/i21d2search/sloti21d2t40/store254/search>> archivesearchpartition-default: /mnt/i21d2search/sloti21d2t40/store254/search-
>> archive>> archivepartition-default: /mnt/i21d2t40/sloti21d2t40/store254/spool-
>> archive>> 
>> (clearly auto-generated!)
>> 
>> Bron.
>> 
>> 
>> On Wed, 20 Dec 2017, at 07:32, Nic Bernstein wrote:
>>> Bron, et al.,
>>> We're about to set up a whole new bunch of partitions in support of
>>> Xapian indexing, for a 2.5.10-to-3.0.4 upgrade, and then will be
>>> introducing archive functionality, too.  How do archive partitions
>>> and Xapian partitions interact?>>> 
>>> For example, the server currently has the following in imapd.conf:
>>>
>>>> defaultpartition: default partition-default:
>>>> /var/mailstores/default metapartition-default:
>>>> /var/imapmeta/default partition-1: /var/mailstores/1 partition-2:
>>>> /var/mailstores/2 partition-3: /var/mailstores/3 partition-4:
>>>> /var/mailstores/4 ... partition-29: /var/mailstores/29 partition-
>>>> 30: /var/mailstores/30 partition-100: /var/mailstores/100 partition-
>>>> temp: /var/mailstores/temp ... # non-default metapartitions metapartition-
>>>> 1: /var/imapmeta/1 metapartition-2: /var/imapmeta/2 metapartition-
>>>> 3: /var/imapmeta/3 metapartition-4: /var/imapmeta/4 ... metapartition-
>>>> 29: /var/imapmeta/29 metapartition-30: /var/imapmeta/30 metapartition-
>>>> 100: /var/imapmeta/100 metapartition-temp: /var/imapmeta/temp

>>>>>>> Going by the documentation, which I wrote with help from you good
>>> folk at Fastmail, the Archive partition scheme might look something
>>> like this:>>> https://www.cyrusimap.org/imap/reference/admin/locations/archive-partitions.html>>>
>>>> archivepartition-default: /var/mailarchives/default archivepartition-
>>>> 1: /var/mailarchives/1 archivepartition-2: /var/mailarchives/2 archivepartition-
>>>> 3: /var/mailarchives/3 archiveartition-4: /var/mailarchives/4 ...
>>>> archivepartition-29: /var/mailarchives/29 archivepartition-30:
>>>> /var/mailarchives/30 archivepartition-100: /var/mailarchives/100
>>>> archivepartition-temp: /var/mailarchives/temp

>>>>>>> And the Xapian partition structure to mate with this would look
>>> something like this (again, from the docs):>>> https://www.cyrusimap.org/imap/developer/install-xapian.html
>>>
>>>> defaultpartition: default partition-default:
>>>> /var/mailstores/default search_engine: xapian search_index_headers:
>>>> no search_batchsize: 8192 defaultsearchtier: t1 1searchtier: t1
>>>> 2searchtier: t1 3searchtier: t1 4searchtier: t1 ... 29searchtier:
>>>> t1 30searchtier: t1 100searchtier: t1 tempsearchtier: t1 ... t1searchpartition-
>>>> default: /var/search/default t1searchpartition-1: /var/search/1 t1searchpartition-
>>>> 2: /var/search/2 t1searchpartition-3: /var/search/3 t1searchpartition-
>>>> 4: /var/search/4 ... t1searchpartition-29: /var/search/29 t1searchpartition-
>>>> 30: /var/search/30 t1searchpartition-100: /var/search/100 t1searchpartition-
>>>> temp: /var/search/temp
>>>>>>> First question, since there's no examples to work from; Is this
>>> Xapian layout correct?>>> 
>>> Do I need to define & create Xapian partitions for the metadata
>>> partitions, as is indirectly implied in Bron's original email on
>>> this topic:>>>     https://lists.tartarus.org/pipermail/xapian-discuss/2014-October/009112.html>>> 
>>> Also, how do these Xapian and Archive, interact?  Do I need to add a
>>> separate Xapian partition for each Archive partition, or will the
>>> Archive partition be treated like a child of the non-Archive
>>> partition?  (again, implied but not directly addressed in that
>>> email).>>> 
>>> Any guidance gladly accepted, and whatever I learn will be
>>> repackaged into more complete documentation on same.>>> 
>>> Cheers,
>>>     -nic
>>>
>>> -- Nic Bernstein                             nic at onlight.com
>>> Onlight, Inc.                             www.onlight.com 6525 W
>>> Bluemound Road, Suite 24           v. 414.272.4477 Milwaukee,
>>> Wisconsin  53213-4073

>>>>>> Email had 1 attachment:


>>>  * nic.vcf
>>>   1k (text/x-vcard)
>> 
>> --
>>   Bron Gondwana
>>   brong at fastmail.fm
>> 
>> 
>
> -- Nic Bernstein                             nic at onlight.com Onlight
> Inc.                              www.onlight.com 6525 W Bluemound
> Rd., Ste 24           v. 414.272.4477 Milwaukee, Wisconsin  53213-4073
> f. 414.290.0335
>
--
  Bron Gondwana
  brong at fastmail.fm


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20171221/67ffdd26/attachment.html>