Outbound Email Security – Part 1 – SPF

Email, as a technology, has been around for longer than “The Web”. While one often hears about security for the web (in the form of encryption, tor, etc), rarely do people discuss email security. As with any technology there are many areas to “secure”, however this series will focus on outbound email security as it relates to anti-spam and anti-phishing. There are three key technologies in play, SPF, DKIM, and DMAC, so this will be a three part series. We’re going to be covering both the “How” (to set it up) along with the very important “Why” (you should set it up). Today we’re starting with Sender Policy Framework, or SPF.

Email follows a lot of the same logic as postal mail and in fact shares some of the same flaws. The first flaw that both mail systems have is related to what is known as the “

return address” in postal mail. There is absolutely nothing to stop anyone from dropping a letter in the mail addressed to your friend, and putting a return address of “1600 Pennsylvania Ave“. Is your friend actually getting a letter from one B. Obama? Of course not, you’re just having a little fun with them.

Relating this to email is simple, it’s forging the outgoing email address. In the analogy above, it would be equivalent to sending your friend an email from [email protected]. Email was invented (the basic concept we know today) in 1971, when the internet was just ARPANET and only a few connected computers were online. As such there was no authentication for sending an email and that continued until today.

So in comes SPF, which is a DNS record for your domain that anyone can lookup. It helps address the above problem by providing a list of authorized sending locations. In our real world analogy this would be the same as verifying all mail sent by me was stamped by the San Francisco post office. Granted that’s not quite as narrow as SPF gets, since it’s by IP address (or effectively by each unique email sending server), but I’ve stretched this analogy as far as I can go.

The SPF record is, as mentioned, fairly simple. A valid record looks like:

v=spf1 include:aspmx.googlemail.com ipv4:10.12.14.31 mx -all

This translates to: “Ask aspmx.googlemail.com what their SPF record is. The IP address ‘10.12.14.31’ is valid. My ‘mx’ records (mail servers) are also valid.” If you’d like to see what some real world samples look like, we’ll use SPF Surveyor for: snowulf.com, Reddit.com, Microsoft.com, Slickdeals.net and Gizmodo.com

If you looked at all of these, you’ll notice that Slickdeals and Gizmodo get big angry red error messages. In both of these domains cases, their SPF records use more than 10 DNS lookups (that’d be the A/MX/PTR/Include items). When RFC4408 (the spec for SPF) was written pre-2006, they thought 10 DNS lookups per domain would be enough. Unfortunately in the modern “cloud” world with email being outsourced, this isn’t enough for a lot of companies — and this leads us to our biggest caveat. You need to be very careful about your “includes”.

In the case of Gizmodo.com they include “_spf.gawker.com” which turns around and includes “_netblocks.gawker.com”, “servers.mcsv.net”, “_spf.google.com”, “gawker.drhinternet.net” and “mailgun.org”. Now mcsv, google and mailgun all turn around and include more. As you can see, even optimizing Gizmodo’s record set by dropping some redundant “MX” and “PTR” entries, they still have too many includes per spec. The only to “fix” the problem is not send email through as many systems. If you work in IT in the corporate world, this is a real challenge.

So what happens if you go over spec? It depends on the receiving mail server. Some people implement the spec exactly and drop any request beyond the specced 10. Some may be more flexible and allow the entire “invalid” record. Still others just throw up their hands and throw out the entire SPF record.

This could mean that email doesn’t get delivered to some recipients, while it works for others, but it’s a toss up. Most mail servers place very little value on SPF records, especially since most mail administrators use the neutral “?all” or softfail “~all” actions in their records. If you spend the time to build a proper SPF record, you should use the hardfail of “-all”. There are also some interesting interactions with invalid SPF records when we talk about DMARC, but we’ll talk about that later in this series.

For most (barring large companies), having a proper SPF record is easy. There are even SPF wizards to help build a proper record. As they are easy, I was curious as to how many people actually used them. Being that they are public records, I checked 98,103 domains of which 39,462 (or 40.2%) had detectable SPF records. For an optional email mechanism that isn’t bad, but everyone who sends email should “do their part” to help reduce spam and email spoofing by implementing SPF.

In Part 2 of this series, we’ll talk about DKIM.