User:David MacQuigg/Sender ID

See also: Email authentication for a general overview and terminology.

Definition: Email authentication method that verifies the domain name in a "purported" sender address.

Sender ID is an email authentication method that seeks to correlate the domain name in a "purported" sender address with the IP address of an SMTP client currently connected and waiting for confirmation that a message was received. The purported address is determined by looking at the headers on the message data. After receiving the message data, but before the client is disconnected, the server does a DNS query for a Sender ID record on the domain name.

If the IP address is listed in the Sender ID record, the authentication result is PASS, and the message may be processed in accordance with the reputation assigned to the domain. If the authentication result is FAIL, the message may be immediately rejected. Often, however, the result is neither PASS nor FAIL, but unclear. This is a result of the many domains that don't publish Sender ID records, or that have records giving unclear results.

Sender Policy Framework (SPF) is an email authentication method that uses the source IP address in a TCP connection to verify the domain name in the envelope Return Address with every message. A Pass result provides strong assurance that the message came from a transmitter authorized by the domain owner. A Fail result allows immediate rejection of the message, while the transmitter is still connected, and before transferring any data.

The domain in the Return Address is verified by doing a DNS query for an SPF record under the domain name. If the IP address is listed in that record, the result is Pass. Thus SPF security depends on the security of Internet IP addresses and of the Domain Name System.

SPF prevents the "replay" abuse possible with signature methods like DKIM, because the transmitter agent (verified domain owner) can be held responsible for high-volume replication of a message.

Limitations

SPF seeks to authenticate the domain name of an originator, usually the MSA, with an IP address from the last agent in a chain of trust, usually the Transmitter. This works well because there is a close relationship between the MSA and the Transmitter. Often they are the same agent.

When a Forwarder is involved, however, the source IP address on the "last hop" to the MDA is no longer correlated to the originator's domain name. An SPF check by the MDA will likely fail.

|--- Sender's Network ---|           |--------- Recipient's Network --------|
                                /
Author ==> MSA/Transmitter --> / --> Receiver/Forwarder ~~> MDA ==> Recipient
                    /         /        /
                   /       Border     /
                  /                  /
                  ------ DNS -------  

The first solution to this problem was a campaign to get Forwarders to re-write the Return Address on forwarded messages, so that it would have the Forwarder's domain name. (See step 5 in the example below.) This failed because Forwarders were not willing to install SRS software and take on responsibility for bounced messages. Also, it required changing a long-standing interpretation of the Return Address as being that of the original sender.

The current solution is to emphasize proper setup within the Recipient's network. SPF checks should be done only at the Border. Recipients should whitelist their Forwarders at their MDAs. Unfortunately, the confusion over this issue has persisted so long that it has led to the second big problem with SPF, the "chicken-and-egg" standoff.

Senders are unwilling to publish robust SPF records ending in "-all" due to the fear that their mail will be rejected by mis-configured forwarding setups. The common practice among nearly all domains publishing SPF records is to end them in ?all. This says to treat all other IP addresses not as Fail, but Neutral. This takes away the major motivation for Receivers to do SPF checks. Without large numbers of Receivers taking these records seriously, senders find that they can be sloppy with their SPF records, or not publish them at all, and it doesn't matter.

How it works

This is a brief explanation of an authentication using an SPF record with just a few of the more common terms. See RFC-4408 for the other terms and full details of SPF.

Senders that wish to use SPF should publish a TXT record and an identical SPF record under their domain name in DNS. Typical DNS records might look like:

example.com.  1800  IN  TXT  "v=spf1 ip4:192.168.33.32/28 mx a:colo.example.net ?all"
example.com.  1800  IN  SPF  "v=spf1 ip4:192.168.33.32/28 mx a:colo.example.net ?all"

These records authorize a number of IP addresses to transmit email for example.com. These addresses include a block of 16 starting at 192.168.33.32, plus any addresses found via the MX records for that same domain (i.e. the incoming mail servers), plus any addresses found in A records for the domain colo.example.net. The ?all term at the end means to treat all other addresses as "neutral", meaning the same as if no SPF records were published.

1) The sender (typically the Mail Submission Agent for example.com) checks the envelope return address to make sure the domain part is "example.com", then relays the message to the Transmitter.

2) The Receiver extracts the domain part of the return address, and does a DNS query (preferably for the SPF record) under that domain.

  MAIL FROM:<author@example.com>

3) The terms in the SPF record are tested one at a time, left to right, and the first definite result (pass or fail) is accepted. The prefixes (+, -, ?) mean (pass, fail, neutral), with + being the default.

4) The Receiver adds an authentication header to the message:

  Received-SPF: Pass (mx1.example.org: domain of
   myname@example.com designates 192.168.33.35 as permitted sender)
      receiver=mx1.example.org; client-ip=192.168.33.35;
      envelope-from=<myname@example.com>; helo=foo.example.com;

5) If the message is forwarded. the forwarder is expected to rewrite the envelope return address so that an SPF check won't fail when the message is forwarded to the next agent. This is done using a related protocol, the Sender Rewriting Scheme (SRS). In this scheme, the forwarder's domain name is substituted for the domain part of the original, and the local part is a concatenation of various strings joined with = characters. These strings include authentication codes and the original return address.

 original:   MAIL FROM:<author@example.com>
 rewritten:  MAIL FROM:<SRS0+yf09=Cw=example.com=author@forwarder.com>

Explanatory notes

TXT is the code for a text record in DNS. Text records can store any text strings the domain owner wants to publish. SPF is a new record type, a TXT record just for SPF data. It will be necessary to publish both records until all DNS software is able to recognize the new type.

1800 seconds is the lifetime of the DNS record after it has left the server on which it was published. This allows frequently used records to be read from a cache, instead of doing a fresh query every time.

Everything between the double quotes is part of the text string, but not the quotes themselves. The string consists of a number of space-delimited terms. "v=spf1" means this is a record for version 1 of SPF.

The syntax for SPF includes a number of key=value pairs (or key:value). This syntax takes a little more space, but it allows for future growth of SPF by simply adding new key items.

The /28 in the ip4 terms is CIDR notation for a block of 16 addresses. The number 28 is the number of bits in a "mask", which is a bit pattern used in testing address ranges. Comparisons using a mask are faster than a general integer range test, but the blocks are limited to sizes that are a power of two (1, 2, 4, 8, 16, ...). CIDR notation is commonly used in routers, where speed is critical.

Bibliography

RFC-4406 (2006), "Sender ID: Authenticating E-Mail", J. Lyon, M. Wong, http://tools.ietf.org/html/rfc4406. Microsoft's variation on SPF.
RFC-4405 (2006), "SMTP Service Extension for Indicating the Responsible Submitter of an E-Mail Message", E. Allman, H. Katz, http://tools.ietf.org/html/rfc4405. Solution to forwarding problem.
RFC-4407 (2006), "Purported Responsible Address in E-Mail Messages", J. Lyon, http://tools.ietf.org/html/rfc4407. Key difference with SPF.

• [Sender ID Home Page] [r]: Add brief definition or description - Microsoft website
  

• [RFC-4408] [r]: Add brief definition or description - "Sender Policy Framework (SPF) for Authorizing Use of Domains in E-Mail, Version 1", M. Wong, W. Schlitt, 2006.
• [Sender Policy Framework] [r]: Add brief definition or description - project website
• [Sender Rewriting Scheme] [r]: Add brief definition or description - website for SRS