Getting email

No mail. In three whole days. Weird. I wonder if it's Thanksgiving over in the States. Not even any football related banter. Is this thing even on ?

Then I realised precisely why I was sitting alone in an island of blissful isolation, devoid of all email communications and staring at an Inbox in a perpetual state of 'Zero'.

I had forgotten to configure inbound email.

When I was testing, I used mbsync to synchronise emails from my ISP which worked well (fast, reliable, well documented) with bi-directional sync between IMAP and my local Maildir.

Note: For Linux types, the 'mbsync' utility is contained within the 'isync' package.

My needs at work are slightly different though. We use IMAP but are encouraged to download and work locally. This corporate edict is implicitly enforced by configuring users' IMAP folders with a measly 1GB quota.

Using mu4e, I needed to fetch new messages from the IMAP server, transfer them to ~/Maildir and then delete them from the IMAP server. Although mbsync has 'push', 'pull' and 'expunge' options, it wasn't entirely clear (to me) if removal of messages from the server was possible. However, the getmail utility written in Python met the requirements exactly and was easy to configure.

Previously, I used Thunderbird to get new messages from IMAP, filter them at source and transfer them directly to various 'Local Folders'. The filtering was pretty basic - typically mailing lists and routing corporate communications to /dev/null.

I tended to configure mailing list folders with a retention period of 30 days and retain 'Starred' messages for ever. Essentially, this meant mailing list traffic didn't clutter my precious Inbox and would automatically expire. I then had a full month to scan the mailing lists and mark any interesting messages for further reading or more unlikely in my case, action.

Anything left over simply came to Inbox. This was normally email directed (To: or CC:'ed to me) so this system worked pretty well.

Sending email

Sending email is easy - mu4e sensibly uses Gnus message mode to send messages. I also built the latest version of msmtp from source for future flexibility (multiple account support) and it seemed marginally quicker than talking directly to the SMTP server.

Reading email

Initially, I didn't like the mu4e initial screen. Where's my Inbox ? Where's my new messages ? I need to see my new emails and start working, dammit !

Then, to my horror, I realised mu4e doesn't have an Inbox per se - just a list of email messages that sit in ~/Maildir. Unread messages reside in '/new' and read messages live in '/cur'.

I was immediately annoyed at a plethora of irrelevant mailing list messages and corporate communications littering my Inbox - sorry littering my 'List of unread messages'.

God - this is terrible. I can see that the only solution here is for me to shave that yak again and configure procmail or use Sieve and Dovecot merely to mimic what I had working fine in Thunderbird out of the box.

Then I saw a post from Dirk (mu4e's author) on the mu4e mailing list.

'mu4e doesn't really have folders - instead "All Mail", "Inbox", "Important", "Sent Mail" etc. are queries - so the same messages can be present in more than one of those'.

Then it struck me like a bolt of lightning. He was absolutely right. I don't need a 'Corporate' folder. I don't need a stupid 'Oracle/MailingLists/dba-wizards' folder with a 3 level hierarchy. I don't even need an 'Inbox'.

What I need is a set of queries to mine the database. Yes - a set of structured text files is in fact a database. mu4e calls these queries 'bookmarks' and provides some useful ones out of the box.

• bu (unread messages)
• bw (messages sent in the last week)
• bp (message with attachments)

Then I would need some ad-hoc or stored queries (e.g. large messages). For example, if I wanted to find that excellent email from Frank about table partitioning he sent out last year:

• contact:oracle-dba-wizards from:frank date:2015 partitioning

The 'Inbox' processing now changes slightly. I needed to read each unread message and quickly decide what to do with it:

• Delete it. 'Blue Nova with lights on in car park 710 East'
• Act on it. Do something. Or reply. Then (optionally) delete it.
• Mark it for (later) 'Action'. Absolute last resort. Obviously.
• Archive it. Something potentially interesting but not now.

Then I remembered that this tied in nicely with an post by Ben Maughan on his excellent Pragmatic Emacs blog which made me question the need for a 'ToDo' folder. Or any folders, in fact.

'I also had folders for putting emails in and I would occasionally have a painful cleanout of my towering inbox, agonising over which folder to put an email in, or whether I should create a new folder for it'.

Then something else struck me. Joking aside, it was quite pleasant and liberating when I had no incoming email. The fact I didn't have a cluttered Inbox presented by default, staring me in the face was great. I had to make a conscious effort to get new mail ('U') and I then had to make a conscious effort to read it 'b u').

Initially, I explicitly disabled periodic automatic fetching of new email (as I had configured in Thunderbird) so I could verify getmail was fetching (and deleting) the correct number of messages from the server and attachments were being handled correctly.

However, I decided to stick with manual email processing initiated by me when I was ready to process email. Notifications of 'new email' are incredibly hard to ignore and a needless context switch if you are busy concentrating on something else (watching Shetland ponies dancing on YouTube).

Further thoughts

Now I am wondering what the point of all my historic folders are; archives by year, mailing lists, personal, sent etc. Previously, in Thunderbird, they were logical groupings and I viewed annual archives as 'read-only' but now in the new scheme, they are merely entries in a database and strictly speaking every single message belongs in '~/Maildir' for simplicity. With one file per message, there is no longer any advantage in logical folders.

So yes, I must be the only person in the world who intentionally went from 'Inbox Zero' to 'Inbox 47,339' and didn't care.

Also, inevitably, I am now being increasingly tempted by the lure of org-mode. In particular, Ben's quote struck a chord with me.

'your inbox is not a todo list'

This is so true and something I have been abusing for years. An email message doesn't have a start date, an end date, a category, a priority or a current status whereas org-mode supports all of those elements.

In addition, org-mode capture takes this further. You can capture anything from any source; an email message, a Web site, a phone call, an instant message, a news article, a blog post, anything.

Configuration

A lot of people conclude and help the reader by including their 'gnus', 'mu4e', 'msmtp' and 'getmail' configuration files but mine are simply variants on the many excellent, annotated examples out there and I'd only repeat an idiotic mistake and post something crucial and security related in clear text on the Internet.

I just changed my Thunderbird setup to move all messages from the IMAP server to local folders and then apply the various filters locally as opposed to applying filters on the server. This means that all messages are visible even when disconnected so I will be able to do email housekeeping in airport lounges.

I tested each new message filter in turn and everything worked fine. Unfortunately I discovered that, contrary to the documentation, Thunderbird 1.5.0.5 does not appear to automatically apply all filters to new messages in 'Local Folders-Inbox'. Sigh.

Then I was cheered up to discover this humble blog is the 'Featured Blog' for Siebel on WordPress.com.

Well when I say 'Featured', maybe the only blog mentioning 'Siebel' on WordPress would be closer to the truth.

My life is complete. Almost.

My old company used Exchange Server. I used to use Outlook for mail.

My new company uses IMAP. So now I can use Thunderbird for mail (news and RSS).

No more wasted time puzzling over why filter rules suddenly stop working.

No more wasted time messing around with proprietary PST files.

Fast, flexible, powerful search capability.

All my Outlook data imported seamlessly.

Small change, massive improvement.

Well I was finally forced to join the masses and download Firefox and ditch Internet Explorer. Yes - my nickname is Johnny Come Lately.

I had been aware of the Firefox browser for a while but my previous experience was with a very early version of Mozilla that had a few problems with a couple of the Web sites I was using at the time.

The final straw came that pushed me over the edge was when my children downloaded some invasive spyware that commandeered the Web browser and implanted a 'Web Search' toolbar. As you can not de-configure this toolbar without completely removing the product, I found myself uninstalling it, my kids would reinstall it for some whizzy colourful cursors. Repeat ad-infinitum. Life is simply too short.

The immediate benefit of Firefox is the tabbed browsing which lets you keep multiple Web sites open and visible at the same time. After you have used this feature for a while, it is indispensable and you wonder how you ever lived without it.

Then I experimented with RSS feeds from the BBC and elsewhere. Instead of having to periodically check a range of Web sites for any new content, I wanted automatic notification of when new content was available. However, while the 'Live bookmarks' feature in Firefox was pretty good, I really wanted something like a pseudo newsgroup that would list new articles on a Website just like articles in a newsgroup. So, I downloaded Thunderbird which seamlessly and painlessly converted all my email messages and contacts from Outlook Express and I was finished.

I have been using Firefox for a week now and so far, the only Web sites that I have encountered problems with is Sony's SonicStage download page (which insists on Microsoft Internet Explorer 5.0) and Blueyonder's selfcare home page (which is gloriously blank). However, this probably says more about the authors of these Web sites rather than any deficiencies in Firefox.