Background
If you didn’t know, Google recently changed the way it groups emails into threads, or as it calls them, conversations. Previously Gmail used the subject and sender information to group emails. This new method relies on explicit linking through the References header, which is much better.
Lots of our network equipment sends email notifications without the References header, and this confuses conversation view when the emails have the same subject. This causes problems especially when an appliance sends emails about different events all with the same subject line. It makes it hard to group the separate events.
My Solution
All we have to do is somehow add a References header with a random value (random because if a References header exists but is unique, the message won’t be threaded even if it has the same subject and is from the same sender as another message). I decided to use part of the Message-ID header as my source of random data (not being able to generate random data the way this solution works). Message-ID is a unique id for each message. This suits my purpose as only when the References header matches up with a previously sent message’s corresponding Message-ID header will threading occur.
If you use an on-premises SMTP server you may be in luck, because you might be able to process the messages before they are sent.
At work we use an onsite Postfix SMTP server for handling the sending emails from photocopiers, network appliances, web filters etc. None of them use the References header which is painful. Using the header_checks config statement in /etc/postfix/main.cf, we can easily add this in.
According to RFC 2076 the References header has the following purpose:
In email: reference to other related messages, in Usenet News: reference to replied-to-articles.
According to RFC 822, the References header is a phrase or msg-id. A msg-id is something that looks like an email address surrounded by angled brackets, eg: <12345.12345@somedomain.com.au>. What I decided to do was to take the value of the Message-ID header and mash it up a bit then use that as the value for the new References header. The resulting config is reproduced below:
In the file /etc/postfix/main.cf:
... header_checks=pcre:/etc/postfix/header_checks ...
In the file /etc/postfix/header_checks (which you may have to create):
/^Message-ID:\s+<([0-9])\.([0-9])@mydomain.com.au>/ PREPEND References: <$2$1@unique.dummy.id.com.au>
This statement starts with a regular expression that looks for a Message-ID header and captures the required information out of it. Then there is the PREPEND command which prepends the following text to the header. The prepended text is itself a regex which pieces together the parts extracted in the first regex. The effect of this is to insert a dummy References header.
So far this is working in testing as expected and will be moved to the production SMTP server soon.
Limitations
The solution I present above has a few limitations:
- if the Message-ID header doesn’t exist this won’t work
- it doesn’t check for the existence of a References header before making a modification. In my case I know the References header doesn’t exist so I am ok for now. This may break in the future
- I can’t limit the application of this modification by sender or any other value
- If the Message-ID header is not made up of two strings of digits separated by a full stop, this won’t work.
Other ways this could work is a Postfix filter or milter. I am not sure about them, but they warrant further investigation. When my solution breaks, I will have a look into it.
Fin
I hope this helps someone who, like me, likes conversation view in Gmail, but has to deal with many emails a day that get grouped into conversations when they really shouldn’t.
Jacob Degeling 