Cyrus 3.0.3 XAPPLEPUSH

Thu Aug 24 09:51:37 EDT 2017

You don’t need to do anything with the tables. The script will populate the tables during the device registration event.

Here is my understanding on how it works, as it’s not documented:

Part 1: Registration

a. If Cyrus is properly configured, it will advertise the XAPPLEPUSH extension
b. The iOS device will recognize the availability and send a XAPPLEPUSH command, that has the following information:
	APS Version: 1 or 2 
	APS Account Id: UUID of the the account on the phone - allow for multiple accounts on the same device
	APS Device Token: Device identification for the push message
	APS: Subtopic: fixed string com.apple.mobilemail (per Apple’s source code and that’s what I’ve seen so far)
	Mailbox: INBOX for version 1 and mailbox list for version 2
c. Cyrus will generate an ApplePushService event and relay all the parameters above to the event handler, along with the user name.
d. The event handler, in the script, add/update the registration for what I call a push target: device_token+account_id. Each target has a list of subscribed mailboxes (user+mailbox).
e. Whenever iOS sends a XAPPLEPUSH, the script updates the timestamp of the registration. This timestamp can be used to delete the registration records, for devices that do not renew the subscription after a while. I will add an option in the script to do it, but the important is that the timestamp is there.
f. Cyrus will return the “aps_topic” information, that comes from imapd.conf. This will be used by the iOS device to authenticate the received push notifications.

Notes on the registration:

1.  I’m trying to figure out how frequent devices “register”, but I’m still to find a pattern here. So far I’ve noticed that devices will renew the registration at least once a day. If that confirms to be a pattern, I would say that it’s safe to remove a device that fails to register after 10 days (allow for that cruise vacation without internet access - well even cruises have internet now).
2. Multiple mailboxes id quite useless (at least I couldn’t find a way to make Mail.App react to pushes for it. We use Cyrus delivery to mailbox option (both with filters and direct addressing, aka plussed users). I can register and send a push, but nothing happens on the device.

Part 1: Events

a. Whenever an event that is being monitored (event groups) occur, the external event handler (the script) is called, with the event’s message on stdin. This is the normal Cyrus behavior for externally handling events from notifyd. The script will decide what events to act upon. Right now I’m processing:
	- MessageNew						- This is obvious
	- MessageRead (INBOX only)			- This will catch when a user reads a message on another device (iOS or otherwise)
	- MessageTrash (INBOX only)			- Same as above, as the user may trash a message without reading it (I do it all the time)
	- MessageMove (vnd.cmu.MessageMove)	- User moves message out of INBOX before reading
	- FlagsClear (for \Seen)					- Reverse of MessageRead 
b. Based on the event type, retrieve the user and mailbox information.
c. Query the database for all targets (device_token+account_id) that have registered to receive push notification for that user+mailbox combination
d. Send the push request to Apple.
	- The push information payload contains  the Device_Token, Account_Id. 
	- The certificate used for the push contains the aps_topic information 
	- These are the 3 pieces that are needed for the iOS Mail.App to properly react. No mailbox information is sent. 
e. Mail.App on the iOS device will react, chime (if new INBOX item) and update the badge (increase or decrease).

Notes on Events:

1. Badge will only count unseen messages in INBOX (actually the sum of the INBOX of all defined accounts).
2. You will get a chime on badge increase but not on decrease. And only if the message is NEW (clearing \Seen does not causes a chime). That’s a nice touch of Apple’s implementation.
1. Delivery to mailboxes other than INBOX will not affect the badge. Actually I can’t get much out of it. The only thing I noticed on Mail.App is that if you are looking at the folder, it gets updated (but I’m not sure it’s reaction to the push).

I’ve tried other payloads, without success. Since there is no documentation, it’s hard to do. But I’m extremely satisfied with the set of events being monitored and they way iOS reacts to it. 

I would love to see comments from other people that have used it, specially about the events and payloads. 

And a final comment: the script approach is only good for a small shop (we have only 10 users, but with huge mailbox trees). 

Arnaldo.
Ps.I have updated the script since I’ve posted. The new version includes MessageRead/Unread (clear \Seen Flag) and sends more than one push per connection (in case the user+mailbox maps to multiple devices). Drop me a line to get a new copy.

> On Aug 24, 2017, at 9:25 AM, Sebastian Hagedorn <Hagedorn at uni-koeln.de> wrote:
> 
> Hi Arnaldo,
> 
> thank you very much! This has been very enlightening. I think I understand most of it and have already set up a testing environment with a certificate from our OS X Server. What I have not yet understood completely is how you populate the database. Where do I find the "APNS Account Id" and the "APNS Device Token" for the devices table, and what do I put in the mailboxes table?
> 
> Thanks,
> Sebastian
> 
> --On 23. August 2017 um 08:27:04 -0300 Arnaldo Viegas de Lima <arnaldo at viegasdelima.com> wrote:
> 
>> A few notes:
>> 
>> - You need a valid push certificate. You can get one from macOS Server,
>> but I do not know the limitations on the usage. My shop is really small,
>> so I think it’s not a problem.
>> 
>> - There is close to nothing documenting how this work, so much of it is
>> based on looking at the source code (Cyrus), from info on a similar
>> Dovecot plugin and on the actual Apple source code (based on Dovecot).
>> Also a lot of debugging and trial and error.
>> 
>> - Contrary to the existing (and minimal) reference in the docs, there is
>> no “—enable-apple-push-service” configure flag. The code in imapd
>> is so simple and harmless that it’s always compiled.
>> 
>> - XAPPLEPUSH requires TLS, so if TLS is not configured it will not be
>> advertised.
>> 
>> - XAPPLEPUSH is only advertised with the proper notifications settings
>> (see bellow).
>> 
>> - I used MySQL because it’s already running on the same server as Cyrus
>> and I have a nice set of functions to make scripts like this a breeze.
>> For the same reasons, it’s in PHP.
>> 
>> 
>> This is what needs to be added to imapd.conf:
>> 
>> # Apple Push Service Events
>> event_notifier: external
>> event_groups: applepushservice message flags
>> event_extra_params: timestamp vnd.cmu.unseenMessages
>> notify_external: /usr/local/bin/notifyd_apns
>> aps_topic: com.apple.mail.XServer.xxxxxxxxxxxxxxx
>> 
>> aps_topic is the common name take from the certificate. It’s sent to
>> the mobile device so that it will match the source of the push
>> notification when it arrives.
>> 
>> 
>> The MySQL database configuration is also attached.
>> 
>> For the certificate you need to enable mail services and export the
>> “mobilemail” certificate and key (as pkcs12) from the keychain. Then
>> use openssl to remove key and certificate as well as to unencrypt the
>> key. Combine both in a single PEM file.
>> 
>> Enjoy!