Personalized Subscriber Emails

To mark the end of 2016 I sent out an email to all those subscribing to be notified about news about MailMate (notice the “Keep Me Posted” text field available on this page). This is the first time I’ve done this even though it’s almost 2 years ago that the first people registered for these emails.

I created the emails using MailMate with the use of some still undocumented and incomplete features. I created a draft in MailMate with some template values and then used a bundle command to create a personalized email for each subscriber in my database. The personalized parts were the recipient address, the month/year of subscribing to the emails, and the optional comment/request registered with the address. The last part was only included if I had a reply/comment to that comment. Yes, some of those replies are coming with almost 2 years of a delay. There are probably better (and certainly faster) ways to create such emails, but it was an interesting learning experience.

You can see the raw message as I wrote it here and you can see a generated one here. It’s mainly a review of what is new in MailMate in 2016.

I also wrote a private “thank you” email to all of the MailMate patrons in a similar way. If you are a registered user then you can learn more about being a patron using the “MailMate ▸ Registration ▸ Become a MailMate Patron…” menu item.

It was interesting going through all the subscriber comments which were mostly feature requests. In particular, some things were requested more often than others. These could be divided into two groups:

  • Requests I really would like to implement/do, but I keep on postponing:
    • A “simpler” 1-level threading mode, e.g., grouping all emails belonging to the same thread, optionally based on the subject of the messages.
    • Message list with multiline entries (previewing message text).
    • Correspondence view (multiple emails in a single message view).
    • Documentation for the creation of custom bundles.
  • Requests I’m not going to implement:
    • A WYSIWYG HTML editor.
    • Local (“On My Mac”) mailboxes and/or POP3 support.
    • Native Exchange support.

Somewhere in between is variations of “make MailMate prettier”, but that’s subjective and I don’t intentionally make MailMate ugly. I do think it got a little prettier in 2016, but I admit that I’m usually more concerned about the look (and adherence to standards) of the emails that MailMate sends (something which seems to be a bit unorthodox among email client developers).

Finally, there are always requests for MailMate 2.0 and this was my standard answer: “Don’t put too much into version numbers. Version 1.9.6 is more than I originally expected 2.0 to be. Also, MailMate is iteratively updated. Bumping to 2.0 would just mean version 3.0 would be getting closer (which would be a paid upgrade).”

The release notes is always the best resource for figuring out exactly what has changed in MailMate.

Happy New Year!

MailMate 1.9.6 Released

MailMate 1.9.6 is primarily a maintenance release. Here are the most notable items of the much longer and detailed release notes:

  • New: “Edit ▸ Add Link” menu item (⌘K). Inserts Markdown code to create a link based on the current word. Also inserts a URL if it’s on the pasteboard.
  • New: URL scheme for doing toolbar-like searches: mlmt:quicksearch?string=s%20mlmt
  • New: “New Message With Attachment” and “Copy Attachment” in the context sensitive menu for an attachment.
  • New: MmSMTPFixedHostname
  • Changed: Generation of the HTML alternative for plain text (non-Markdown) messages is now handled using a dedicated script. For compatibility reasons this uses whitespace:normal and <p> instead of <div>. This should resolve various issues when corresponding with Outlook users.
  • Changed: Undoing move actions now re-selects messages if they exist in the current mailbox.
  • Fixed: Saving a search as a smart mailbox frequently failed.
  • Fixed: Various issues with HTML embedding when forwarding HTML messages with attachments.
  • Fixed: Issue with the choice of Drafts mailbox (and implicitly SMTP server) when an identity could not be derived and an explicit one had not been configured.
  • Fixed: Drag’n’drop emails to the Finder on Sierra.

MailMate 1.9.5 Released

It’s a minor version bump, but it’s not a minor update. MailMate 1.9.5 is a major update as documented in the detailed release notes. Here follows a short list of the most notable highlights:

  • Embed arbitrary HTML when replying/forwarding.
  • MailMate icon replaced (designed by Eli Schiff).
  • Toolbar icons added (designed by Eli Schiff).
  • Style outgoing messages using CSS.
  • Code styling using Pygments.
  • Math styling using ASCIIMath or TeXMath (converted to MathML which is not supported by all recipient email clients).
  • Numerous additions to the Composer preferences pane to control the HTML related features above.
  • Fixed various issues with macOS 10.12 (Sierra).
  • Support for tags and colored flags in the toolbar.
  • “Format ▸ Bold/Italic” menu items which inserts/removes Markdown syntax to emphasize text.
  • Network code now uses CFNetwork instead of OpenSSL. This implicitly means proxy support (System Preferences), IPv6 support, and TLS 1.2 support.
  • Redesigned headers view.
  • Fixed an issue with sleep mode and/or being offline. MailMate failed to send all pending messages in Drafts (until relaunching MailMate).

If you are a happy MailMate user then also note this addition:

  • Added “MailMate ▸ Registration ▸ Become a MailMate Patron…” menu item (only enabled for registered users).

MailMate 1.9.4 Released

This release includes a lot of small improvements and bug fixes. The most interesting changes are various new hidden preferences, including MmSMTPAlternativeEnabled and MmSMTPUTF8Enabled. More information in the manual and in the detailed release notes.

Is OAuth2 Support a Good Thing?

MailMate now supports the OAuth2 authentication method for Gmail (and Outlook) accounts. This is a good thing for MailMate users with Gmail accounts, but I don’t really feel good about it… More about that further below.

How does it work?

A bit simplified, it works like this: Using an embedded web browser in MailMate, the user is sent to a hardcoded Google address (using a secure connection). The user is then asked by Google to allow MailMate to access the emails of the Gmail account. If accepted then MailMate receives a special code. Using this code MailMate can then obtain a so-called access token. This access token is then used when authenticating via IMAP or SMTP. In other words, the real password is never known to1 or used by MailMate itself. It is naturally also not stored by MailMate. An access token expires, but MailMate can obtain a new one when needed without interrupting the user. The access token only provides access to emails and the user can revoke the access at any time on this page. That part is a nice feature of OAuth2.

Note that the above is just one of many ways to implement OAuth2 in a desktop email client — none of them being perfect.

What’s the problem?

The main problem is that OAuth2 requires me to register MailMate with the service provider (Google/Microsoft). If the provider stops supporting other authentication schemes (which is almost true for Google) then the provider has the power to decide which email clients are allowed to access Gmail. I’m probably too old to trust big companies, but it also reminds me of what happened to third party Twitter and (more recently) Instagram clients.

In other words, after implementing OAuth2 I’m now part of the problem. I’ve made it a little bit easier for Google/Microsoft to stop supporting other authentication schemes and if they do that they can hit the kill switch on MailMate whenever they want to. They might even have a good reason to do so since a desktop email client cannot protect its so-called client identity. An evil app can easily pretend to be MailMate when requesting access to an account.

Of course, a niche email client like MailMate doesn’t really matter in the big picture, but it’s not the only email client feeling the pressure to support OAuth2.

What about Apple? Well, iCloud has an authentication scheme similar to OAuth2 (I assume), but it can only be used by Apple’s own email clients. I’m not sure if that is better or worse…

Why support OAuth2?

Google continues to push for the adoption of OAuth2 via the XOAUTH2 protocol. In my opinion, they do that using a lot of FUD as seen in this support article, but that does not mean that OAuth2 is necessarily a bad thing to use. Especially not for something like Google for which a single password provides access to all kinds of services.

And I don’t really have a choice here. When using other authentication methods then Gmail users are often rejected. The exact behavior appears to depend on how long the Google account has existed and whether it has been accessed via IMAP in the past. In particular, I believe new Gmail accounts are rejected by default if not using OAuth2. The best user experience is simply with OAuth2 enabled.

This is what it boils down to: MailMate supports Gmail (and Outlook) and I’ll do whatever I can to make it work well. This already includes working around the highly non-standard behavior of Gmail and the many bugs of Outlook IMAP. I do recommend though that all desktop email client users consider the alternatives.

  1. Since MailMate embeds the web browser itself then this is not strictly true. This also shows that OAuth2 doesn’t provide as much security for desktop applications as it does for web services.

MailMate 1.9.3 Released

This release is mainly focused on bug fixes and stability issues. The most notable new features are manual ordering of IMAP accounts/mailboxes and OAuth2 support for Gmail and Outlook accounts. Note that OAuth2 is enabled by default. If you have any issues then please report them and note that you can disable OAuth2 in the IMAP account settings. I’ll write more about OAuth2 and why it’s enabled by default in a separate blog post.

Default Email Client on El Capitan

Note: This blog post is relevant for users of any third party email client on OS X.

A few days ago I was contacted by a MailMate user reporting that he could not switch default email client using the popup provided in the General preferences pane of MailMate. It seemed to work, but after a short while the setting was reverted to its previous state. My initial thought was that this had to be some kind of local issue with Launch Services. This is the service taking care of binding file types and URL schemes to applications. In particular, it can bind the mailto: URL scheme to an application and this is what happens when switching default email client.

I was a bit surprised when I realized it didn’t work for me either. We quickly established that it was a problem on El Capitan only and that it didn’t matter if Apple Mail or MailMate was used to switch default email client. Further experimentation revealed that the problem was most likely not related to MailMate at all.

It still seemed a bit unlikely to be a problem affecting all El Capitan users since googling for the problem didn’t reveal much. There had to be some common denominator. Therefore I searched for users not having this issue via the MailMate mailing list and via Twitter. But I didn’t find anyone.

I currently believe it is a general issue on El Capitan. I’m not quite sure why it hasn’t gotten more attention, but it’s probably a combination of the following:

  • The delayed reversion to the previously default email client.
  • Most often the switch reverts, but it appears that some times the switch does stick.
  • The first switch (clean install or after resetting the Launch Services database) seems to always work.

So, a workaround does appear to be to reset Launch Services, but it’s a bad workaround since it resets all of your custom file type and URL scheme bindings. I don’t currently know any other workarounds other than to keep on trying until it sticks. A special case is if you are trying to switch back to Apple Mail. In that case it also works to uninstall your current email client.

I’ve reported the issue to Apple (rdar://23123392) and hopefully it’ll be fixed soon. I won’t do more than that unless someone comes up with a workaround which I can use to make the popup in the MailMate preferences pane work reliably. If you want to experiment then it might be useful to use the command line hack I created for my mailing list post.

Update November 23rd: A user reports that the workaround described here worked for him. I haven’t tested it myself.

MailMate 1.9.2 Released — El Capitan Compatible

With the forthcoming release of OS X 10.11 (El Capitan) it is also a good time for a MailMate update.

This release is compatible with El Capitan, but it also includes numerous other new features, changes, and fixes. As usual, all the details can be found in the release notes.

One new feature deserves extra attention. MailMate now includes a Spotlight importer for .eml files (standard email format). This feature is enabled in the General preferences pane and it allows searching for MailMate emails using the Spotlight interface. It also allows searching emails stored elsewhere which means that a local archive of emails (stored in the .eml format) is now also searchable if MailMate is installed. (Don’t tell anyone, but this works even if MailMate is not registered.)

Note that the answers provided for frequently asked questions in my previous blog post are still valid.

MailMate 1.9 Released

Update April 2nd: MailMate 1.9.1 has been released with a few important fixes. Most importantly, the new Bundles preferences pane was not available for all users.

As explained in the previous blog post I’ve been working on making MailMate a 64 bit application. This change is now complete and it involved numerous optimizations making MailMate faster while also making it use much less memory.

The migration to 64 bit is far from the only thing I’ve been working on. As always, the release notes are ridiculously long. Note that some of the listed features were also available in earlier releases as experimental 2.0 features enabled in the General preferences pane.

The following are just some of the most important highlights:

  • Bundles preferences pane which allows you to enable/disable integration with various applications. Currently, 19 bundles exist providing support for BBEdit, BusyCal, Due, EagleFiler, Evernote, OmniFocus, TextMate, Todoist, and more.
  • Rules pane in the mailbox editor for any mailbox.
  • Toolbar search field activated using ⌥⌘F. Note the search field menu and its options.
  • Correspondent/Identity added as special header transformations. This is, e.g., used for the new “Correspondent” column available for the messages outline.
  • Each tag can be assigned an emoji and a new tags column can be used to display them. Note that ⌘⌃+space opens an emoji keyboard on Yosemite.
  • Enable composer header fields by holding down ⌥ when hitting Return. For example, hit ⌥↩ to add a “Cc” header when focus is in the “To” header.
  • Numerous other new features, changes, and fixes. You really do need to read the release notes to know it all.
  • MailMate should now work with even more buggy IMAP servers than it already did. Remember, good alternatives do exist.

Enjoy the new release and rest assured that I’ll continue to work on improving MailMate.

Here are answers to what I expect to be the most frequently asked questions (with a few replays from previous posts):

  • Any new graphics in version 1.9?

    Only a few details, but I do expect some improvements for version 2.0. In particular since some of you were so kind to contribute to the somewhat discreet mini crowd funding campaign.

  • When is MailMate 2.0 released?

    I don’t know, but it probably matters more to me than to you. I need it for marketing purposes. You need it for the features. The development of MailMate is an open process and you can use the betas if you like (enabled in the Software Update preferences pane). If feeling adventurous, you can even fetch cutting edge test releases by holding down ⌥ when clicking “Check Now”.

  • When are you going to support feature X?

    I have no idea, but if you want to be notified about major new releases of MailMate then you can now subscribe. If you do that then you also get a chance to state your “most wanted feature”.

  • Are you going out of business soon?


MailMate 64 Bit Beta

You might not have noticed this, but MailMate is still a 32 bit application. The main reason for that is not that it is technically hard to switch to 64 bit. The main reason is that memory usage would increase considerably if doing it naively. MailMate indexes a lot of data using data structures with a lot of address pointers. In fact, the first experimental 64 bit build of MailMate used almost twice the memory of the 32 bit build.

It is quite rare that users “complain” about MailMate being a 64 bit app, but there are many reasons that staying 32 bit is not a good idea. One of them is the fact that if MailMate is the only 32 bit app on your system then it uses a lot more memory than you might realize. This is because a 32 bit app triggers the system to load 32 bit system frameworks. On my system, MailMate, Spotify, and SpiderOak are the only 32 bit apps (Google Chrome very recently switched to 64 bit)1. Having MailMate on that list is becoming a bit of an embarrassment.

Now, you might have noticed the lack of beta/test releases of MailMate in the past month. This is because I’ve been working on a 64 bit release of MailMate. Most of that work has been focused on optimizations regarding both speed and memory. It’s hard to measure this in general, but on my system MailMate now launches 2-3 times faster than the 32 bit release and it uses less memory.

The 64 bit releases are on a separate auto-update system and I’ll keep it that way until I’m relatively sure it’s at least as stable as the latest 32 bit release. You can help me reach that milestone by switching to 64 bit now. Initially, it’ll require a manual install of this download. If needed, you can switch back to the 32 bit release.

Although the 64 bit release primarily features optimizations, it also includes a few new features, changes, and fixes. Here are some of the most interesting:

  • New: “Default Account” setting for new messages in the Composer preferences pane.
  • New: Completion of email addresses works for any contacts with a company name (and email address).
  • New: Added perform command to the AppleScript API. Its argument is a list of strings identical to what would be used to define a custom key binding. The frontmost window of MailMate is the target even if MailMate is not in focus (activated). Example: tell application "MailMate" to perform {"toggleFlag:"}
  • New: “Distortion Mode” now also works for HTML messages and attachment descriptions.

If you are new to MailMate then don’t miss the FAQ part of the previous blog post. Related to that, I really appreciate that the (somewhat low-key) mini crowd funding campaign has reached 25% of my initial goal.

  1. You can use the Activity Monitor in OS X if you want to see which applications are running in 32 bit mode (switch to the Memory pane on Yosemite). Click on the header of the column to sort the list.