Request for Comments: 4408 W. Schlitt
Category: Experimental April 2006
Sender Policy Framework (SPF) for
Authorizing Use of Domains in E-Mail, Version 1
Status of This Memo
This memo defines an Experimental Protocol for the Internet
community. It does not specify an Internet standard of any kind.
Discussion and suggestions for improvement are requested.
Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2006).
IESG Note
The following documents (RFC 4405, RFC 4406, RFC 4407, and RFC 4408)
are published simultaneously as Experimental RFCs, although there is
no general technical consensus and efforts to reconcile the two
approaches have failed. As such, these documents have not received
full IETF review and are published "AS-IS" to document the different
approaches as they were considered in the MARID working group.
The IESG takes no position about which approach is to be preferred
and cautions the reader that there are serious open issues for each
approach and concerns about using them in tandem. The IESG believes
that documenting the different approaches does less harm than not
documenting them.
Note that the Sender ID experiment may use DNS records that may have
been created for the current SPF experiment or earlier versions in
this set of experiments. Depending on the content of the record,
this may mean that Sender-ID heuristics would be applied incorrectly
to a message. Depending on the actions associated by the recipient
with those heuristics, the message may not be delivered or may be
discarded on receipt.
Participants relying on Sender ID experiment DNS records are warned
that they may lose valid messages in this set of circumstances.
aParticipants publishing SPF experiment DNS records should consider
the advice given in section 3.4 of RFC 4406 and may wish to publish
both v=spf1 and spf2.0 records to avoid the conflict.
Participants in the Sender-ID experiment need to be aware that the
way Resent-* header fields are used will result in failure to receive
legitimate email when interacting with standards-compliant systems
(specifically automatic forwarders which comply with the standards by
not adding Resent-* headers, and systems which comply with RFC 822
but have not yet implemented RFC 2822 Resent-* semantics). It would
be inappropriate to advance Sender-ID on the standards track without
resolving this interoperability problem.
The community is invited to observe the success or failure of the two
approaches during the two years following publication, in order that
a community consensus can be reached in the future.
Abstract
E-mail on the Internet can be forged in a number of ways. In
particular, existing protocols place no restriction on what a sending
host can use as the reverse-path of a message or the domain given on
the SMTP HELO/EHLO commands. This document describes version 1 of
the Sender Policy Framework (SPF) protocol, whereby a domain may
explicitly authorize the hosts that are allowed to use its domain
name, and a receiving host may check such authorization.
Table of Contents
1. Introduction ....................................................4
1.1. Protocol Status ............................................4
1.2. Terminology ................................................5
2. Operation .......................................................5
2.1. The HELO Identity ..........................................5
2.2. The MAIL FROM Identity .....................................5
2.3. Publishing Authorization ...................................6
2.4. Checking Authorization .....................................6
2.5. Interpreting the Result ....................................7
2.5.1. None ................................................8
2.5.2. Neutral .............................................8
2.5.3. Pass ................................................8
2.5.4. Fail ................................................8
2.5.5. SoftFail ............................................9
2.5.6. TempError ...........................................9
2.5.7. PermError ...........................................9
3. SPF Records .....................................................9
3.1. Publishing ................................................10
3.1.1. DNS Resource Record Types ..........................10
3.1.2. Multiple DNS Records ...............................11
3.1.3. Multiple Strings in a Single DNS record ............11
3.1.4. Record Size ........................................11
3.1.5. Wildcard Records ...................................11
4. The check_host() Function ......................................12
4.1. Arguments .................................................12
4.2. Results ...................................................13
4.3. Initial Processing ........................................13
4.4. Record Lookup .............................................13
4.5. Selecting Records .........................................13
4.6. Record Evaluation .........................................14
4.6.1. Term Evaluation ....................................14
4.6.2. Mechanisms .........................................15
4.6.3. Modifiers ..........................................15
4.7. Default Result ............................................16
4.8. Domain Specification ......................................16
5. Mechanism Definitions ..........................................16
5.1. "all" .....................................................17
5.2. "include" .................................................18
5.3. "a" .......................................................19
5.4. "mx" ......................................................20
5.5. "ptr" .....................................................20
5.6. "ip4" and "ip6" ...........................................21
5.7. "exists" ..................................................22
6. Modifier Definitions ...........................................22
6.1. redirect: Redirected Query ................................23
6.2. exp: Explanation ..........................................23
7. The Received-SPF Header Field ..................................25
8. Macros .........................................................27
8.1. Macro Definitions .........................................27
8.2. Expansion Examples ........................................30
9. Implications ...................................................31
9.1. Sending Domains ...........................................31
9.2. Mailing Lists .............................................32
9.3. Forwarding Services and Aliases ...........................32
9.4. Mail Services .............................................34
9.5. MTA Relays ................................................34
10. Security Considerations .......................................35
10.1. Processing Limits ........................................35
10.2. SPF-Authorized E-Mail May Contain Other False
Identities ...............................................37
10.3. Spoofed DNS and IP Data ..................................37
10.4. Cross-User Forgery .......................................37
10.5. Untrusted Information Sources ............................38
10.6. Privacy Exposure .........................................38
11. Contributors and Acknowledgements .............................38
12. IANA Considerations ...........................................39
12.1. The SPF DNS Record Type ..................................39
12.2. The Received-SPF Mail Header Field .......................39
13. References ....................................................39
13.1. Normative References .....................................39
13.2. Informative References ...................................40
Appendix A. Collected ABNF .......................................42
Appendix B. Extended Examples ....................................44
B.1. Simple Examples ..........................................44
B.2. Multiple Domain Example ..................................45
B.3. DNSBL Style Example ......................................46
B.4. Multiple Requirements Example ............................46
1. Introduction
The current E-Mail infrastructure has the property that any host
injecting mail into the mail system can identify itself as any domain
name it wants. Hosts can do this at a variety of levels: in
particular, the session, the envelope, and the mail headers.
Although this feature is desirable in some circumstances, it is a
major obstacle to reducing Unsolicited Bulk E-Mail (UBE, aka spam).
Furthermore, many domain name holders are understandably concerned
about the ease with which other entities may make use of their domain
names, often with malicious intent.
This document defines a protocol by which domain owners may authorize
hosts to use their domain name in the "MAIL FROM" or "HELO" identity.
Compliant domain holders publish Sender Policy Framework (SPF)
records specifying which hosts are permitted to use their names, and
compliant mail receivers use the published SPF records to test the
authorization of sending Mail Transfer Agents (MTAs) using a given
"HELO" or "MAIL FROM" identity during a mail transaction.
An additional benefit to mail receivers is that after the use of an
identity is verified, local policy decisions about the mail can be
made based on the sender’s domain, rather than the host’s IP address.
This is advantageous because reputation of domain names is likely to
be more accurate than reputation of host IP addresses. Furthermore,
if a claimed identity fails verification, local policy can take
stronger action against such E-Mail, such as rejecting it.
1.1. Protocol Status
SPF has been in development since the summer of 2003 and has seen
deployment beyond the developers beginning in December 2003. The
design of SPF slowly evolved until the spring of 2004 and has since
stabilized. There have been quite a number of forms of SPF, some
written up as documents, some submitted as Internet Drafts, and many
discussed and debated in development forums.
The goal of this document is to clearly document the protocol defined
by earlier draft specifications of SPF as used in existing
implementations. This conception of SPF is sometimes called "SPF
Classic". It is understood that particular implementations and
deployments may differ from, and build upon, this work. It is hoped
that we have nonetheless captured the common understanding of SPF
version 1.
1.2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
This document is concerned with the portion of a mail message
commonly called "envelope sender", "return path", "reverse path",
"bounce address", "2821 FROM", or "MAIL FROM". Since these terms are
either not well defined or often used casually, this document defines
the "MAIL FROM" identity in Section 2.2. Note that other terms that
may superficially look like the common terms, such as "reverse-path",
are used only with the defined meanings from normative documents.
2. Operation
2.1. The HELO Identity
The "HELO" identity derives from either the SMTP HELO or EHLO command
(see [RFC2821]). These commands supply the SMTP client (sending
host) for the SMTP session. Note that requirements for the domain
presented in the EHLO or HELO command are not always clear to the
sending party, and SPF clients must be prepared for the "HELO"
identity to be malformed or an IP address literal. At the time of
this writing, many legitimate E-Mails are delivered with invalid HELO
domains.
It is RECOMMENDED that SPF clients not only check the "MAIL FROM"
identity, but also separately check the "HELO" identity by applying
the check_host() function (Section 4) to the "HELO" identity as the
<sender>.
2.2. The MAIL FROM Identity
The "MAIL FROM" identity derives from the SMTP MAIL command (see
[RFC2821]). This command supplies the "reverse-path" for a message,
which generally consists of the sender mailbox, and is the mailbox to
which notification messages are to be sent if there are problems
delivering the message.
[RFC2821] allows the reverse-path to be null (see Section 4.5.5 in
RFC 2821). In this case, there is no explicit sender mailbox, and
such a message can be assumed to be a notification message from the
mail system itself. When the reverse-path is null, this document
defines the "MAIL FROM" identity to be the mailbox composed of the
localpart "postmaster" and the "HELO" identity (which may or may not
have been checked separately before).
SPF clients MUST check the "MAIL FROM" identity. SPF clients check
the "MAIL FROM" identity by applying the check_host() function to the
"MAIL FROM" identity as the <sender>.
2.3. Publishing Authorization
An SPF-compliant domain MUST publish a valid SPF record as described
in Section 3. This record authorizes the use of the domain name in
the "HELO" and "MAIL FROM" identities by the MTAs it specifies.
If domain owners choose to publish SPF records, it is RECOMMENDED
that they end in "-all", or redirect to other records that do, so
that a definitive determination of authorization can be made.
Domain holders may publish SPF records that explicitly authorize no
hosts if mail should never originate using that domain.
When changing SPF records, care must be taken to ensure that there is
a transition period so that the old policy remains valid until all
legitimate E-Mail has been checked.
2.4. Checking Authorization
A mail receiver can perform a set of SPF checks for each mail message
it receives. An SPF check tests the authorization of a client host
to emit mail with a given identity. Typically, such checks are done
by a receiving MTA, but can be performed elsewhere in the mail
processing chain so long as the required information is available and
reliable. At least the "MAIL FROM" identity MUST be checked, but it
is RECOMMENDED that the "HELO" identity also be checked beforehand.
Without explicit approval of the domain owner, checking other
identities against SPF version 1 records is NOT RECOMMENDED because
there are cases that are known to give incorrect results. For
example, almost all mailing lists rewrite the "MAIL FROM" identity
(see Section 9.2), but some do not change any other identities in the
message. The scenario described in Section 9.3, sub-section 1.2, is
another example. Documents that define other identities should
define the method for explicit approval.
It is possible that mail receivers will use the SPF check as part of
a larger set of tests on incoming mail. The results of other tests
may influence whether or not a particular SPF check is performed.
For example, finding the sending host’s IP address on a local white
list may cause all other tests to be skipped and all mail from that
host to be accepted.
When a mail receiver decides to perform an SPF check, it MUST use a
correctly-implemented check_host() function (Section 4) evaluated
with the correct parameters. Although the test as a whole is
optional, once it has been decided to perform a test it must be
performed as specified so that the correct semantics are preserved
between publisher and receiver.
To make the test, the mail receiver MUST evaluate the check_host()
function with the arguments set as follows:
<ip> - the IP address of the SMTP client that is emitting the
mail, either IPv4 or IPv6.
<domain> - the domain portion of the "MAIL FROM" or "HELO" identity.
<sender> - the "MAIL FROM" or "HELO" identity.
Note that the <domain> argument may not be a well-formed domain name.
For example, if the reverse-path was null, then the EHLO/HELO domain
is used, with its associated problems (see Section 2.1). In these
cases, check_host() is defined in Section 4.3 to return a "None"
result.
Although invalid, malformed, or non-existent domains cause SPF checks
to return "None" because no SPF record can be found, it has long been
the policy of many MTAs to reject E-Mail from such domains,
especially in the case of invalid "MAIL FROM". In order to prevent
the circumvention of SPF records, rejecting E-Mail from invalid
domains should be considered.
Implementations must take care to correctly extract the <domain> from
the data given with the SMTP MAIL FROM command as many MTAs will
still accept such things as source routes (see [RFC2821], Appendix
C), the %-hack (see [RFC1123]), and bang paths (see [RFC1983]).
These archaic features have been maliciously used to bypass security
systems.
2.5. Interpreting the Result
This section describes how software that performs the authorization
should interpret the results of the check_host() function. The
authorization check SHOULD be performed during the processing of the
SMTP transaction that sends the mail. This allows errors to be
returned directly to the sending MTA by way of SMTP replies.
Performing the authorization after the SMTP transaction has finished
may cause problems, such as the following: (1) It may be difficult to
accurately extract the required information from potentially
deceptive headers; (2) legitimate E-Mail may fail because the
sender’s policy may have since changed.
Generating non-delivery notifications to forged identities that have
failed the authorization check is generally abusive and against the
explicit wishes of the identity owner.
2.5.1. None
A result of "None" means that no records were published by the domain
or that no checkable sender domain could be determined from the given
identity. The checking software cannot ascertain whether or not the
client host is authorized.
2.5.2. Neutral
The domain owner has explicitly stated that he cannot or does not
want to assert whether or not the IP address is authorized. A
"Neutral" result MUST be treated exactly like the "None" result; the
distinction exists only for informational purposes. Treating
"Neutral" more harshly than "None" would discourage domain owners
from testing the use of SPF records (see Section 9.1).
2.5.3. Pass
A "Pass" result means that the client is authorized to inject mail
with the given identity. The domain can now, in the sense of
reputation, be considered responsible for sending the message.
Further policy checks can now proceed with confidence in the
legitimate use of the identity.
2.5.4. Fail
A "Fail" result is an explicit statement that the client is not
authorized to use the domain in the given identity. The checking
software can choose to mark the mail based on this or to reject the
mail outright.
If the checking software chooses to reject the mail during the SMTP
transaction, then it SHOULD use an SMTP reply code of 550 (see
[RFC2821]) and, if supported, the 5.7.1 Delivery Status Notification
(DSN) code (see [RFC3464]), in addition to an appropriate reply text.
The check_host() function may return either a default explanation
string or one from the domain that published the SPF records (see
Section 6.2). If the information does not originate with the
checking software, it should be made clear that the text is provided
by the sender’s domain. For example:
550-5.7.1 SPF MAIL FROM check failed:
550-5.7.1 The domain example.com explains:
550 5.7.1 Please see http://www.example.com/mailpolicy.html
2.5.5. SoftFail
A "SoftFail" result should be treated as somewhere between a "Fail"
and a "Neutral". The domain believes the host is not authorized but
is not willing to make that strong of a statement. Receiving
software SHOULD NOT reject the message based solely on this result,
but MAY subject the message to closer scrutiny than normal.
The domain owner wants to discourage the use of this host and thus
desires limited feedback when a "SoftFail" result occurs. For
example, the recipient’s Mail User Agent (MUA) could highlight the
"SoftFail" status, or the receiving MTA could give the sender a
message using a technique called "greylisting" whereby the MTA can
issue an SMTP reply code of 451 (4.3.0 DSN code) with a note the
first time the message is received, but accept it the second time.
2.5.6. TempError
A "TempError" result means that the SPF client encountered a
transient error while performing the check. Checking software can
choose to accept or temporarily reject the message. If the message
is rejected during the SMTP transaction for this reason, the software
SHOULD use an SMTP reply code of 451 and, if supported, the 4.4.3 DSN
code.
2.5.7. PermError
A "PermError" result means that the domain’s published records could
not be correctly interpreted. This signals an error condition that
requires manual intervention to be resolved, as opposed to the
TempError result. Be aware that if the domain owner uses macros
(Section 8), it is possible that this result is due to the checked
identities having an unexpected format.
3. SPF Records
An SPF record is a DNS Resource Record (RR) that declares which hosts
are, and are not, authorized to use a domain name for the "HELO" and
"MAIL FROM" identities. Loosely, the record partitions all hosts
into permitted and not-permitted sets (though some hosts might fall
into neither category).
The SPF record is a single string of text. An example record is the
following:
v=spf1 +mx a:colo.example.com/28 -all
This record has a version of "spf1" and three directives: "+mx",
"a:colo.example.com/28" (the + is implied), and "-all".
3.1. Publishing
Domain owners wishing to be SPF compliant must publish SPF records
for the hosts that are used in the "MAIL FROM" and "HELO" identities.
The SPF records are placed in the DNS tree at the host name it
pertains to, not a subdomain under it, such as is done with SRV
records. This is the same whether the TXT or SPF RR type (see
Section 3.1.1) is used.
The example above in Section 3 might be published via these lines in
a domain zone file:
example.com. TXT "v=spf1 +mx a:colo.example.com/28 -all"
smtp-out.example.com. TXT "v=spf1 a -all"
When publishing via TXT records, beware of other TXT records
published there for other purposes. They may cause problems with
size limits (see Section 3.1.4).
3.1.1. DNS Resource Record Types
This document defines a new DNS RR of type SPF, code 99. The format
of this type is identical to the TXT RR [RFC1035]. For either type,
the character content of the record is encoded as [US-ASCII].
It is recognized that the current practice (using a TXT record) is
not optimal, but it is necessary because there are a number of DNS
server and resolver implementations in common use that cannot handle
the new RR type. The two-record-type scheme provides a forward path
to the better solution of using an RR type reserved for this purpose.
An SPF-compliant domain name SHOULD have SPF records of both RR
types. A compliant domain name MUST have a record of at least one
type. If a domain has records of both types, they MUST have
identical content. For example, instead of publishing just one
record as in Section 3.1 above, it is better to publish:
example.com. IN TXT "v=spf1 +mx a:colo.example.com/28 -all"
example.com. IN SPF "v=spf1 +mx a:colo.example.com/28 -all"
Example RRs in this document are shown with the TXT record type;
however, they could be published with the SPF type or with both
types.
3.1.2. Multiple DNS Records
A domain name MUST NOT have multiple records that would cause an
authorization check to select more than one record. See Section 4.5
for the selection rules.
3.1.3. Multiple Strings in a Single DNS record
As defined in [RFC1035] sections 3.3.14 and 3.3, a single text DNS
record (either TXT or SPF RR types) can be composed of more than one
string. If a published record contains multiple strings, then the
record MUST be treated as if those strings are concatenated together
without adding spaces. For example:
IN TXT "v=spf1 .... first" "second string..."
MUST be treated as equivalent to
IN TXT "v=spf1 .... firstsecond string..."
SPF or TXT records containing multiple strings are useful in