Cyrus Sieve futures

Bron Gondwana brong at
Sat Feb 18 17:54:38 EST 2017

Sorry, I've been traveling and didn't get back to this.

Ken has been doing most of the work on sieve recently. Realistically,
I'd say we don't have much time to work on it right now either. The best
thing would be to integrate anything into our qa build chain, since
otherwise the testing only holds until someone touches the code again.


On Fri, 10 Feb 2017, at 09:42, Anatoli via Cyrus-devel wrote:

> Bron,
>  That's good. Do you see this change (safeguarding Sieve with seccomp)
>  necessary? Would someone from your team help with its implementation?
>  To be realistic, most likely I won't be able to find enough time to
>  refactor Sieve alone (i.e. to adapt it to the described
>  architecture), but I could help with the architecture/logic, the
>  "results" file format, seccomp part, processes, passing the data to
>  and from and related aspects.
>  You said in one issue at github that you have plans to replace PCRE
>  with RE2. This one, plus Ken's work, I believe, could have an
>  opportunity to refactor Sieve for the seccomp change without
>  dedicating it significant extra resources.
>  If you find it appropriate we could start another discussion (in
>  github?) to outline the implementation details.
> *From:* Cyrus Devel

> *Sent:* Wednesday, February 08, 2017 01:48

> *To:* Cyrus Devel

> *Subject:* Re: Cyrus Sieve futures

>> I'm pretty sure there's no IO handling inside the sieve bytecode
>> processing, though it makes callbacks to perform the actual actions -
>> so the impure bits are very clearly defined.


>> On Wed, 8 Feb 2017, at 13:05, Anatoli via Cyrus-devel wrote:

>>> Bron, do you mean Sieve is separated from the rest of the Cyrus code
>>> or that inside Sieve code there's separation for pure data
>>> processing (without syscalls) and I/O & other resources handling,
>>> processes/threads management, etc.? I my previous mail I was meaning
>>> the later.

>>> If it's already done, that's great and everything is ready to define
>>> the format for the "results" file and proceed with the seccomp
>>> implementation, if all agree that this is a needed change.

>>> *From:* Bron Gondwana Via Cyrus-devel

>>> *Sent:* Tuesday, February 07, 2017 07:01

>>> *To:* Cyrus-devel

>>> *Subject:* Re: Cyrus Sieve futures

>>> Actually, that's pretty much already done, the separation.  Sieve,
>>> more than any other part of the Cyrus code, is very decoupled.  It
>>> used to be a separate CVS repository once upon a time.

>>> Bron.



>>> On Tue, 7 Feb 2017, at 18:36, Anatoli via Cyrus-devel wrote:

>>>> Hi Ken,


>>>> I don't have any special uses for Sieve apart from the most basic
>>>> ones, but I would like to ask you if you see it feasible, as part
>>>> of this work, to separate the "numbers crunching" (e.g. rules
>>>> matching, transformations, etc.) code from the I/O code (that is
>>>> responsible for manipulating files, communicating with other
>>>> processes, etc.)?

>>>> My idea is to isolate and safeguard Sieve's "numbers crunching"
>>>> code with[1] seccomp[2], which basically only allows to execute
>>>> userland code (i.e. no syscalls, etc.) and read/write to already-
>>>> open file descriptors. This should reduce currently broad attack
>>>> surface to some really exotic bugs in the Linux kernel (up to now
>>>> just one significant bug for strict seccomp bypass was discovered
>>>> (CVE-2009-0835 in 2.6 kernel 8 years ago), soon after this
>>>> functionality was implemented).

>>>> One of possible architectures could be described this way (similar
>>>> to Postfix architecture[3]):
>>>>  1. Sieve daemon receives an incoming data for processing and
>>>>     stores it in a memory buffer (without touching it)
>>>>  2. It creates a new "results" file and opens it for writing
>>>>  3. It forks a process
>>>>  4. The newly created child process switches to seccomp
>>>>  5. It processes the data and writes the results (in a special
>>>>     format) to the "results" file, then exits
>>>>  6. Sieve daemon detects the exit of the child process and reads
>>>>     the "results" file to perform the requested actions, then
>>>>     deletes the file
>>>>  7. If the child is killed (as the result of seccomp restrictions
>>>>     violation) or something is wrong with the format of the
>>>>     "results" file, Sieve daemon quarantines the data and writes an
>>>>     error to the logs

>>>> The format for the "results" file should be simple and well
>>>> defined, and the code to interpret it should be carefully written.
>>>> This could be started as a mere adaptation for the new architecture
>>>> of the current actions processing logic and be progressively
>>>> improved later. This would be much easier than making the entire
>>>> Sieve code base and the libraries it uses (e.g. PCRE) reasonably
>>>> safe (PCRE alone is a huge bag ofvulnerabilities[4], including lots
>>>> of RCEs).


>>>> Another question (just wondering if it's in your (or other devs)
>>>> plans and its feasibility): is it practically possible to implement
>>>> for Sieve something like "run the rules on X folder (+ subfolders)"
>>>> same way as local rules in most MUAs could be applied to already
>>>> stored mails? I find lack of this feature as the only (but notable)
>>>> downside to Sieve vs local rules.

>>>> Regards,

>>>> Anatoli


>>>> *From:* Ken Murchison Via Cyrus-devel

>>>> *Sent:* Monday, February 06, 2017 19:34

>>>> *To:* Cyrus-devel

>>>> *Subject:* Cyrus Sieve futures

>>>>> All, 


>>>>> I'm in the process of rewriting the Sieve parser and adding new
>>>>> extensions for what will become part of Cyrus v3.1.  We currently
>>>>> support deprecated and non-standardized extensions "imapflags"
>>>>> (standardized as "imap4flags) and "notify" (standardized as
>>>>> "enotify").  I'd like to rip out the parser and bytecode generator
>>>>> for these extensions, and leave just the bytecode executing code
>>>>> for the deprecated actions "mark", "unmark", and "denotify".

>>>>> Any existing scripts using these actions (or the older "notify"
>>>>> syntax) would continue to run.  New/updated scripts would have to
>>>>> switch to using the updated "notify" syntax and replace "mark" and
>>>>> "unmark" with "setflag"/"addflag" and "removeflag".  Does anyone
>>>>> have an issue with these changes?

>>>>> Does anyone have any requests for standard extensions that we
>>>>> don't currently support?  Note that "variables", "mailbox" and
>>>>> "*metadata" will be in Cyrus 3.0 and "ereject", "editheader", and
>>>>> "extlists" are already in what will be the 3.1 branch.

>>>>> Extensions that I'm looking at implementing (pretty much because
>>>>> they are low-hanging fruit) are "duplicate", "environment", and
>>>>> "ihave".  I may also look at "replace" and "extracttext" which
>>>>> would be useful if we add handling of calendar events to Sieve.


>>> --

>>>   Bron Gondwana

>>>   brong at




>> --

>>   Bron Gondwana

>>   brong at




  Bron Gondwana

  brong at


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the Cyrus-devel mailing list