the server SHOULD include an entity containing an explanation of the
error situation, and whether it is a temporary or permanent
condition. These status codes are applicable to any request method.
User agents SHOULD display any included entity to the user.
If the client is sending data, a server implementation using TCP
SHOULD be careful to ensure that the client acknowledges receipt of
the packet(s) containing the response, before the server closes the
input connection. If the client continues sending data to the server
after the close, the server's TCP stack will send a reset packet to
the client, which may erase the client's unacknowledged input buffers
before they can be read and interpreted by the HTTP application.
10.4.1 400 Bad Request
The request could not be understood by the server due to malformed
syntax. The client SHOULD NOT repeat the request without
modifications.
10.4.2 401 Unauthorized
The request requires user authentication. The response MUST include a
WWW-Authenticate header field (section 14.47) containing a challenge
applicable to the requested resource. The client MAY repeat the
request with a suitable Authorization header field (section 14.8). If
the request already included Authorization credentials, then the 401
response indicates that authorization has been refused for those
credentials. If the 401 response contains the same challenge as the
prior response, and the user agent has already attempted
authentication at least once, then the user SHOULD be presented the
entity that was given in the response, since that entity might
include relevant diagnostic information. HTTP access authentication
is explained in "HTTP Authentication: Basic and Digest Access
Authentication" [43].
10.4.3 402 Payment Required
This code is reserved for future use.
10.4.4 403 Forbidden
The server understood the request, but is refusing to fulfill it.
Authorization will not help and the request SHOULD NOT be repeated.
If the request method was not HEAD and the server wishes to make
public why the request has not been fulfilled, it SHOULD describe the
reason for the refusal in the entity. If the server does not wish to
make this information available to the client, the status code 404
(Not Found) can be used instead.
10.4.5 404 Not Found
The server has not found anything matching the Request-URI. No
indication is given of whether the condition is temporary or
permanent. The 410 (Gone) status code SHOULD be used if the server
knows, through some internally configurable mechanism, that an old
resource is permanently unavailable and has no forwarding address.
This status code is commonly used when the server does not wish to
reveal exactly why the request has been refused, or when no other
response is applicable.
10.4.6 405 Method Not Allowed
The method specified in the Request-Line is not allowed for the
resource identified by the Request-URI. The response MUST include an
Allow header containing a list of valid methods for the requested
resource.
10.4.7 406 Not Acceptable
The resource identified by the request is only capable of generating
response entities which have content characteristics not acceptable
according to the accept headers sent in the request.
Unless it was a HEAD request, the response SHOULD include an entity
containing a list of available entity characteristics and location(s)
from which the user or user agent can choose the one most
appropriate. The entity format is specified by the media type given
in the Content-Type header field. Depending upon the format and the
capabilities of the user agent, selection of the most appropriate
choice MAY be performed automatically. However, this specification
does not define any standard for such automatic selection.
Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.
If the response could be unacceptable, a user agent SHOULD
temporarily stop receipt of more data and query the user for a
decision on further actions.
10.4.8 407 Proxy Authentication Required
This code is similar to 401 (Unauthorized), but indicates that the
client must first authenticate itself with the proxy. The proxy MUST
return a Proxy-Authenticate header field (section 14.33) containing a
challenge applicable to the proxy for the requested resource. The
client MAY repeat the request with a suitable Proxy-Authorization
header field (section 14.34). HTTP access authentication is explained
in "HTTP Authentication: Basic and Digest Access Authentication"
[43].
10.4.9 408 Request Timeout
The client did not produce a request within the time that the server
was prepared to wait. The client MAY repeat the request without
modifications at any later time.
10.4.10 409 Conflict
The request could not be completed due to a conflict with the current
state of the resource. This code is only allowed in situations where
it is expected that the user might be able to resolve the conflict
and resubmit the request. The response body SHOULD include enough
information for the user to recognize the source of the conflict.
Ideally, the response entity would include enough information for the
user or user agent to fix the problem; however, that might not be
possible and is not required.
Conflicts are most likely to occur in response to a PUT request. For
example, if versioning were being used and the entity being PUT
included changes to a resource which conflict with those made by an
earlier (third-party) request, the server might use the 409 response
to indicate that it can't complete the request. In this case, the
response entity would likely contain a list of the differences
between the two versions in a format defined by the response
Content-Type.
10.4.11 410 Gone
The requested resource is no longer available at the server and no
forwarding address is known. This condition is expected to be
considered permanent. Clients with link editing capabilities SHOULD
delete references to the Request-URI after user approval. If the
server does not know, or has no facility to determine, whether or not
the condition is permanent, the status code 404 (Not Found) SHOULD be
used instead. This response is cacheable unless indicated otherwise.
The 410 response is primarily intended to assist the task of web
maintenance by notifying the recipient that the resource is
intentionally unavailable and that the server owners desire that
remote links to that resource be removed. Such an event is common for
limited-time, promotional services and for resources belonging to
individuals no longer working at the server's site. It is not
necessary to mark all permanently unavailable resources as "gone" or
to keep the mark for any length of time -- that is left to the
discretion of the server owner.
10.4.12 411 Length Required
The server refuses to accept the request without a defined Content-
Length. The client MAY repeat the request if it adds a valid
Content-Length header field containing the length of the message-body
in the request message.
10.4.13 412 Precondition Failed
The precondition given in one or more of the request-header fields
evaluated to false when it was tested on the server. This response
code allows the client to place preconditions on the current resource
metainformation (header field data) and thus prevent the requested
method from being applied to a resource other than the one intended.
10.4.14 413 Request Entity Too Large
The server is refusing to process a request because the request
entity is larger than the server is willing or able to process. The
server MAY close the connection to prevent the client from continuing
the request.
If the condition is temporary, the server SHOULD include a Retry-
After header field to indicate that it is temporary and after what
time the client MAY try again.
10.4.15 414 Request-URI Too Long
The server is refusing to service the request because the Request-URI
is longer than the server is willing to interpret. This rare
condition is only likely to occur when a client has improperly
converted a POST request to a GET request with long query
information, when the client has descended into a URI "black hole" of
redirection (e.g., a redirected URI prefix that points to a suffix of
itself), or when the server is under attack by a client attempting to
exploit security holes present in some servers using fixed-length
buffers for reading or manipulating the Request-URI.
10.4.16 415 Unsupported Media Type
The server is refusing to service the request because the entity of
the request is in a format not supported by the requested resource
for the requested method.
10.4.17 416 Requested Range Not Satisfiable
A server SHOULD return a response with this status code if a request
included a Range request-header field (section 14.35), and none of
the range-specifier values in this field overlap the current extent
of the selected resource, and the request did not include an If-Range
request-header field. (For byte-ranges, this means that the first-
byte-pos of all of the byte-range-spec values were greater than the
current length of the selected resource.)
When this status code is returned for a byte-range request, the
response SHOULD include a Content-Range entity-header field
specifying the current length of the selected resource (see section
14.16). This response MUST NOT use the multipart/byteranges content-
type.
10.4.18 417 Expectation Failed
The expectation given in an Expect request-header field (see section
14.20) could not be met by this server, or, if the server is a proxy,
the server has unambiguous evidence that the request could not be met
by the next-hop server.
10.5 Server Error 5xx
Response status codes beginning with the digit "5" indicate cases in
which the server is aware that it has erred or is incapable of
performing the request. Except when responding to a HEAD request, the
server SHOULD include an entity containing an explanation of the
error situation, and whether it is a temporary or permanent
condition. User agents SHOULD display any included entity to the
user. These response codes are applicable to any request method.
10.5.1 500 Internal Server Error
The server encountered an unexpected condition which prevented it
from fulfilling the request.
10.5.2 501 Not Implemented
The server does not support the functionality required to fulfill the
request. This is the appropriate response when the server does not
recognize the request method and is not capable of supporting it for
any resource.
10.5.3 502 Bad Gateway
The server, while acting as a gateway or proxy, received an invalid
response from the upstream server it accessed in attempting to
fulfill the request.
10.5.4 503 Service Unavailable
The server is currently unable to handle the request due to a
temporary overloading or maintenance of the server. The implication
is that this is a temporary condition which will be alleviated after
some delay. If known, the length of the delay MAY be indicated in a
Retry-After header. If no Retry-After is given, the client SHOULD
handle the response as it would for a 500 response.
Note: The existence of the 503 status code does not imply that a
server must use it when becoming overloaded. Some servers may wish
to simply refuse the connection.
10.5.5 504 Gateway Timeout
The server, while acting as a gateway or proxy, did not receive a
timely response from the upstream server specified by the URI (e.g.
HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed
to access in attempting to complete the request.
Note: Note to implementors: some deployed proxies are known to
return 400 or 500 when DNS lookups time out.
10.5.6 505 HTTP Version Not Supported
The server does not support, or refuses to support, the HTTP protocol
version that was used in the request message. The server is
indicating that it is unable or unwilling to complete the request
using the same major version as the client, as described in section
3.1, other than with this error message. The response SHOULD contain
an entity describing why that version is not supported and what other
protocols are supported by that server.
11 Access Authentication
HTTP provides several OPTIONAL challenge-response authentication
mechanisms which can be used by a server to challenge a client
request and by a client to provide authentication information. The
general framework for access authentication, and the specification of
"basic" and "digest" authentication, are specified in "HTTP
Authentication: Basic and Digest Access Authentication" [43]. This
specification adopts the definitions of "challenge" and "credentials"
from that specification.
12 Content Negotiation
Most HTTP responses include an entity which contains information for
interpretation by a human user. Naturally, it is desirable to supply
the user with the "best available" entity corresponding to the
request. Unfortunately for servers and caches, not all users have the
same preferences for what is "best," and not all user agents are
equally capable of rendering all entity types. For that reason, HTTP
has provisions for several mechanisms for "content negotiation" --
the process of selecting the best representation for a given response
when there are multiple representations available.
Note: This is not called "format negotiation" because the
alternate representations may be of the same media type, but use
different capabilities of that type, be in different languages,
etc.
Any response containing an entity-body MAY be subject to negotiation,
including error responses.
There are two kinds of content negotiation which are possible in
HTTP: server-driven and agent-driven negotiation. These two kinds of
negotiation are orthogonal and thus may be used separately or in
combination. One method of combination, referred to as transparent
negotiation, occurs when a cache uses the agent-driven negotiation
information provided by the origin server in order to provide
server-driven negotiation for subsequent requests.
12.1 Server-driven Negotiation
If the selection of the best representation for a response is made by
an algorithm located at the server, it is called server-driven
negotiation. Selection is based on the available representations of
the response (the dimensions over which it can vary; e.g. language,
content-coding, etc.) and the contents of particular header fields in
the request message or on other information pertaining to the request
(such as the network address of the client).
Server-driven negotiation is advantageous when the algorithm for
selecting from among the available representations is difficult to
describe to the user agent, or when the server desires to send its
"best guess" to the client along with the first response (hoping to
avoid the round-trip delay of a subsequent request if the "best
guess" is good enough for the user). In order to improve the server's
guess, the user agent MAY include request header fields (Accept,
Accept-Language, Accept-Encoding, etc.) which describe its
preferences for such a response.
Server-driven negotiation has disadvantages:
1. It is impossible for the server to accurately determine what
might be "best" for any given user, since that would require
complete knowledge of both the capabilities of the user agent
and the intended use for the response (e.g., does the user want
to view it on screen or print it on paper?).
2. Having the user agent describe its capabilities in every
request can be both very inefficient (given that only a small
percentage of responses have multiple representations) and a
potential violation of the user's privacy.
3. It complicates the implementation of an origin server and the
algorithms for generating responses to a request.
4. It may limit a public cache's ability to use the same response
for multiple user's requests.
HTTP/1.1 includes the following request-header fields for enabling
server-driven negotiation through description of user agent
capabilities and user preferences: Accept (section 14.1), Accept-
Charset (section 14.2), Accept-Encoding (section 14.3), Accept-
Language (section 14.4), and User-Agent (section 14.43). However, an
origin server is not limited to these dimensions and MAY vary the
response based on any aspect of the request, including information
outside the request-header fields or within extension header fields
not defined by this specification.
The Vary header field can be used to express the parameters the
server uses to select a representation that is subject to server-
driven negotiation. See section 13.6 for use of the Vary header field
by caches and section 14.44 for use of the Vary header field by
servers.
12.2 Agent-driven Negotiation
With agent-driven negotiation, selection of the best representation
for a response is performed by the user agent after receiving an
initial response from the origin server. Selection is based on a list
of the available representations of the response included within the
header fields or entity-body of the initial response, with each
representation identified by its own URI. Selection from among the
representations may be performed automatically (if the user agent is
capable of doing so) or manually by the user selecting from a
generated (possibly hypertext) menu.
Agent-driven negotiation is advantageous when the response would vary
over commonly-used dimensions (such as type, language, or encoding),
when the origin server is unable to determine a user agent's
capabilities from examining the request, and generally when public
caches are used to distribute server load and reduce network usage.
Agent-driven negotiation suffers from the disadvantage of needing a
second request to obtain the best alternate representation. This
second request is only efficient when caching is used. In addition,
this specification does not define any mechanism for supporting
automatic selection, though it also does not prevent any such
mechanism from being developed as an extension and used within
HTTP/1.1.
HTTP/1.1 defines the 300 (Multiple Choices) and 406 (Not Acceptable)
status codes for enabling agent-driven negotiation when the server is
unwilling or unable to provide a varying response using server-driven
negotiation.
12.3 Transparent Negotiation
Transparent negotiation is a combination of both server-driven and
agent-driven negotiation. When a cache is supplied with a form of the
list of available representations of the response (as in agent-driven
negotiation) and the dimensions of variance are completely understood
by the cache, then the cache becomes capable of performing server-
driven negotiation on behalf of the origin server for subsequent
requests on that resource.
Transparent negotiation has the advantage of distributing the
negotiation work that would otherwise be required of the origin
server and also removing the second request delay of agent-driven
negotiation when the cache is able to correctly guess the right
response.
This specification does not define any mechanism for transparent
negotiation, though it also does not prevent any such mechanism from
being developed as an extension that could be used within HTTP/1.1.
13 Caching in HTTP
HTTP is typically used for distributed information systems, where
performance can be improved by the use of response caches. The
HTTP/1.1 protocol includes a number of elements intended to make
caching work as well as possible. Because these elements are
inextricable from other aspects of the protocol, and because they
interact with each other, it is useful to describe the basic caching
design of HTTP separately from the detailed descriptions of methods,
headers, response codes, etc.
Caching would be useless if it did not significantly improve
performance. The goal of caching in HTTP/1.1 is to eliminate the need
to send requests in many cases, and to eliminate the need to send
full responses in many other cases. The former reduces the number of
network round-trips required for many operations; we use an
"expiration" mechanism for this purpose (see section 13.2). The
latter reduces network bandwidth requirements; we use a "validation"
mechanism for this purpose (see section 13.3).
Requirements for performance, availability, and disconnected
operation require us to be able to relax the goal of semantic
transparency. The HTTP/1.1 protocol allows origin servers, caches,
and clients to explicitly reduce transparency when necessary.
However, because non-transparent operation may confuse non-expert
users, and might be incompatible with certain server applications
(such as those for ordering merchandise), the protocol requires that
transparency be relaxed
- only by an explicit protocol-level request when relaxed by
client or origin server
- only with an explicit warning to the end user when relaxed by
cache or client
Therefore, the HTTP/1.1 protocol provides these important elements:
1. Protocol features that provide full semantic transparency when
this is required by all parties.
2. Protocol features that allow an origin server or user agent to
explicitly request and control non-transparent operation.
3. Protocol features that allow a cache to attach warnings to
responses that do not preserve the requested approximation of
semantic transparency.
A basic principle is that it must be possible for the clients to
detect any potential relaxation of semantic transparency.
Note: The server, cache, or client implementor might be faced with
design decisions not explicitly discussed in this specification.
If a decision might affect semantic transparency, the implementor
ought to err on the side of maintaining transparency unless a
careful and complete analysis shows significant benefits in
breaking transparency.
13.1.1 Cache Correctness
A correct cache MUST respond to a request with the most up-to-date
response held by the cache that is appropriate to the request (see
sections 13.2.5, 13.2.6, and 13.12) which meets one of the following
conditions:
1. It has been checked for equivalence with what the origin server
would have returned by revalidating the response with the
origin server (section 13.3);
2. It is "fresh enough" (see section 13.2). In the default case,
this means it meets the least restrictive freshness requirement
of the client, origin server, and cache (see section 14.9); if
the origin server so specifies, it is the freshness requirement
of the origin server alone.
If a stored response is not "fresh enough" by the most
restrictive freshness requirement of both the client and the
origin server, in carefully considered circumstances the cache
MAY still return the response with the appropriate Warning
header (see section 13.1.5 and 14.46), unless such a response
is prohibited (e.g., by a "no-store" cache-directive, or by a
"no-cache" cache-request-directive; see section 14.9).
3. It is an appropriate 304 (Not Modified), 305 (Proxy Redirect),
or error (4xx or 5xx) response message.
If the cache can not communicate with the origin server, then a
correct cache SHOULD respond as above if the response can be
correctly served from the cache; if not it MUST return an error or
warning indicating that there was a communication failure.
If a cache receives a response (either an entire response, or a 304
(Not Modified) response) that it would normally forward to the
requesting client, and the received response is no longer fresh, the
cache SHOULD forward it to the requesting client without adding a new
Warning (but without removing any existing Warning headers). A cache
SHOULD NOT attempt to revalidate a response simply because that
response became stale in transit; this might lead to an infinite
loop. A user agent that receives a stale response without a Warning
MAY display a warning indication to the user.
13.1.2 Warnings
Whenever a cache returns a response that is neither first-hand nor
"fresh enough" (in the sense of condition 2 in section 13.1.1), it
MUST attach a warning to that effect, using a Warning general-header.
The Warning header and the currently defined warnings are described
in section 14.46. The warning allows clients to take appropriate
action.
Warnings MAY be used for other purposes, both cache-related and
otherwise. The use of a warning, rather than an error status code,
distinguish these responses from true failures.
Warnings are assigned three digit warn-codes. The first digit
indicates whether the Warning MUST or MUST NOT be deleted from a
stored cache entry after a successful revalidation:
1xx Warnings that describe the freshness or revalidation status of
the response, and so MUST be deleted after a successful
revalidation. 1XX warn-codes MAY be generated by a cache only when
validating a cached entry. It MUST NOT be generated by clients.
2xx Warnings that describe some aspect of the entity body or entity
headers that is not rectified by a revalidation (for example, a
lossy compression of the entity bodies) and which MUST NOT be
deleted after a successful revalidation.
See section 14.46 for the definitions of the codes themselves.
HTTP/1.0 caches will cache all Warnings in responses, without
deleting the ones in the first category. Warnings in responses that
are passed to HTTP/1.0 caches carry an extra warning-date field,
which prevents a future HTTP/1.1 recipient from believing an
erroneously cached Warning.
Warnings also carry a warning text. The text MAY be in any
appropriate natural language (perhaps based on the client's Accept
headers), and include an OPTIONAL indication of what character set is
used.
Multiple warnings MAY be attached to a response (either by the origin
server or by a cache), including multiple warnings with the same code
number. For example, a server might provide the same warning with
texts in both English and Basque.
When multiple warnings are attached to a response, it might not be
practical or reasonable to display all of them to the user. This
version of HTTP does not specify strict priority rules for deciding
which warnings to display and in what order, but does suggest some
heuristics.
13.1.3 Cache-control Mechanisms
The basic cache mechanisms in HTTP/1.1 (server-specified expiration
times and validators) are implicit directives to caches. In some
cases, a server or client might need to provide explicit directives
to the HTTP caches. We use the Cache-Control header for this purpose.
The Cache-Control header allows a client or server to transmit a
variety of directives in either requests or responses. These
directives typically override the default caching algorithms. As a
general rule, if there is any apparent conflict between header
values, the most restrictive interpretation is applied (that is, the
one that is most likely to preserve semantic transparency). However,
in some cases, cache-control directives are explicitly specified as
weakening the approximation of semantic transparency (for example,
"max-stale" or "public").
The cache-control directives are described in detail in section 14.9.
13.1.4 Explicit User Agent Warnings
Many user agents make it possible for users to override the basic
caching mechanisms. For example, the user agent might allow the user
to specify that cached entities (even explicitly stale ones) are
never validated. Or the user agent might habitually add "Cache-
Control: max-stale=3600" to every request. The user agent SHOULD NOT
default to either non-transparent behavior, or behavior that results
in abnormally ineffective caching, but MAY be explicitly configured
to do so by an explicit action of the user.
If the user has overridden the basic caching mechanisms, the user
agent SHOULD explicitly indicate to the user whenever this results in
the display of information that might not meet the server's
transparency requirements (in particular, if the displayed entity is
known to be stale). Since the protocol normally allows the user agent
to determine if responses are stale or not, this indication need only
be displayed when this actually happens. The indication need not be a
dialog box; it could be an icon (for example, a picture of a rotting
fish) or some other indicator.
If the user has overridden the caching mechanisms in a way that would
abnormally reduce the effectiveness of caches, the user agent SHOULD
continually indicate this state to the user (for example, by a
display of a picture of currency in flames) so that the user does not
inadvertently consume excess resources or suffer from excessive
latency.
13.1.5 Exceptions to the Rules and Warnings
In some cases, the operator of a cache MAY choose to configure it to
return stale responses even when not requested by clients. This
decision ought not be made lightly, but may be necessary for reasons
of availability or performance, especially when the cache is poorly
connected to the origin server. Whenever a cache returns a stale
response, it MUST mark it as such (using a Warning header) enabling
the client software to alert the user that there might be a potential
problem.
It also allows the user agent to take steps to obtain a first-hand or
fresh response. For this reason, a cache SHOULD NOT return a stale
response if the client explicitly requests a first-hand or fresh one,
unless it is impossible to comply for technical or policy reasons.
13.1.6 Client-controlled Behavior
While the origin server (and to a lesser extent, intermediate caches,
by their contribution to the age of a response) are the primary
source of expiration information, in some cases the client might need
to control a cache's decision about whether to return a cached
response without validating it. Clients do this using several
directives of the Cache-Control header.
A client's request MAY specify the maximum age it is willing to
accept of an unvalidated response; specifying a value of zero forces
the cache(s) to revalidate all responses. A client MAY also specify
the minimum time remaining before a response expires. Both of these
options increase constraints on the behavior of caches, and so cannot
further relax the cache's approximation of semantic transparency.
A client MAY also specify that it will accept stale responses, up to
some maximum amount of staleness. This loosens the constraints on the
caches, and so might violate the origin server's specified
constraints on semantic transparency, but might be necessary to
support disconnected operation, or high availability in the face of
poor connectivity.
13.2 Expiration Model
13.2.1 Server-Specified Expiration
HTTP caching works best when caches can entirely avoid making
requests to the origin server. The primary mechanism for avoiding
requests is for an origin server to provide an explicit expiration
time in the future, indicating that a response MAY be used to satisfy
subsequent requests. In other words, a cache can return a fresh
response without first contacting the server.
Our expectation is that servers will assign future explicit
expiration times to responses in the belief that the entity is not
likely to change, in a semantically significant way, before the
expiration time is reached. This normally preserves semantic
transparency, as long as the server's expiration times are carefully
chosen.
The expiration mechanism applies only to responses taken from a cache
and not to first-hand responses forwarded immediately to the
requesting client.
If an origin server wishes to force a semantically transparent cache
to validate every request, it MAY assign an explicit expiration time
in the past. This means that the response is always stale, and so the
cache SHOULD validate it before using it for subsequent requests. See
section 14.9.4 for a more restrictive way to force revalidation.
If an origin server wishes to force any HTTP/1.1 cache, no matter how
it is configured, to validate every request, it SHOULD use the "must-
revalidate" cache-control directive (see section 14.9).
Servers specify explicit expiration times using either the Expires
header, or the max-age directive of the Cache-Control header.
An expiration time cannot be used to force a user agent to refresh
its display or reload a resource; its semantics apply only to caching
mechanisms, and such mechanisms need only check a resource's
expiration status when a new request for that resource is initiated.
See section 13.13 for an explanation of the difference between caches
and history mechanisms.
13.2.2 Heuristic Expiration
Since origin servers do not always provide explicit expiration times,
HTTP caches typically assign heuristic expiration times, employing
algorithms that use other header values (such as the Last-Modified
time) to estimate a plausible expiration time. The HTTP/1.1
specification does not provide specific algorithms, but does impose
worst-case constraints on their results. Since heuristic expiration
times might compromise semantic transparency, they ought to used
cautiously, and we encourage origin servers to provide explicit
expiration times as much as possible.
13.2.3 Age Calculations
In order to know if a cached entry is fresh, a cache needs to know if
its age exceeds its freshness lifetime. We discuss how to calculate
the latter in section 13.2.4; this section describes how to calculate
the age of a response or cache entry.
In this discussion, we use the term "now" to mean "the current value
of the clock at the host performing the calculation." Hosts that use
HTTP, but especially hosts running origin servers and caches, SHOULD
use NTP [28] or some similar protocol to synchronize their clocks to
a globally accurate time standard.
HTTP/1.1 requires origin servers to send a Date header, if possible,
with every response, giving the time at which the response was
generated (see section 14.18). We use the term "date_value" to denote
the value of the Date header, in a form appropriate for arithmetic
operations.
HTTP/1.1 uses the Age response-header to convey the estimated age of
the response message when obtained from a cache. The Age field value
is the cache's estimate of the amount of time since the response was
generated or revalidated by the origin server.
In essence, the Age value is the sum of the time that the response
has been resident in each of the caches along the path from the
origin server, plus the amount of time it has been in transit along
network paths.
We use the term "age_value" to denote the value of the Age header, in
a form appropriate for arithmetic operations.
A response's age can be calculated in two entirely independent ways:
1. now minus date_value, if the local clock is reasonably well
synchronized to the origin server's clock. If the result is
negative, the result is replaced by zero.
2. age_value, if all of the caches along the response path
implement HTTP/1.1.
Given that we have two independent ways to compute the age of a
response when it is received, we can combine these as
corrected_received_age = max(now - date_value, age_value)
and as long as we have either nearly synchronized clocks or all-
HTTP/1.1 paths, one gets a reliable (conservative) result.
Because of network-imposed delays, some significant interval might
pass between the time that a server generates a response and the time
it is received at the next outbound cache or client. If uncorrected,
this delay could result in improperly low ages.
Because the request that resulted in the returned Age value must have
been initiated prior to that Age value's generation, we can correct
for delays imposed by the network by recording the time at which the
request was initiated. Then, when an Age value is received, it MUST
be interpreted relative to the time the request was initiated, not
the time that the response was received. This algorithm results in
conservative behavior no matter how much delay is experienced. So, we
compute:
corrected_initial_age = corrected_received_age
+ (now - request_time)
where "request_time" is the time (according to the local clock) when
the request that elicited this response was sent.
Summary of age calculation algorithm, when a cache receives a
response:
/*
* age_value
* is the value of Age: header received by the cache with
* this response.
* date_value
* is the value of the origin server's Date: header
* request_time
* is the (local) time when the cache made the request
* that resulted in this cached response
* response_time
* is the (local) time when the cache received the
* response
* now
* is the current (local) time
*/
apparent_age = max(0, response_time - date_value);
corrected_received_age = max(apparent_age, age_value);
response_delay = response_time - request_time;
corrected_initial_age = corrected_received_age + response_delay;
resident_time = now - response_time;
current_age = corrected_initial_age + resident_time;
The current_age of a cache entry is calculated by adding the amount
of time (in seconds) since the cache entry was last validated by the
origin server to the corrected_initial_age. When a response is
generated from a cache entry, the cache MUST include a single Age
header field in the response with a value equal to the cache entry's
current_age.
The presence of an Age header field in a response implies that a
response is not first-hand. However, the converse is not true, since
the lack of an Age header field in a response does not imply that the
response is first-hand unless all caches along the request path are
compliant with HTTP/1.1 (i.e., older HTTP caches did not implement
the Age header field).
13.2.4 Expiration Calculations
In order to decide whether a response is fresh or stale, we need to
compare its freshness lifetime to its age. The age is calculated as
described in section 13.2.3; this section describes how to calculate
the freshness lifetime, and to determine if a response has expired.
In the discussion below, the values can be represented in any form
appropriate for arithmetic operations.
We use the term "expires_value" to denote the value of the Expires
header. We use the term "max_age_value" to denote an appropriate
value of the number of seconds carried by the "max-age" directive of
the Cache-Control header in a response (see section 14.9.3).
The max-age directive takes priority over Expires, so if max-age is
present in a response, the calculation is simply:
freshness_lifetime = max_age_value
Otherwise, if Expires is present in the response, the calculation is:
freshness_lifetime = expires_value - date_value
Note that neither of these calculations is vulnerable to clock skew,
since all of the information comes from the origin server.
If none of Expires, Cache-Control: max-age, or Cache-Control: s-
maxage (see section 14.9.3) appears in the response, and the response
does not include other restrictions on caching, the cache MAY compute
a freshness lifetime using a heuristic. The cache MUST attach Warning
113 to any response whose age is more than 24 hours if such warning
has not already been added.
Also, if the response does have a Last-Modified time, the heuristic
expiration value SHOULD be no more than some fraction of the interval
since that time. A typical setting of this fraction might be 10%.
The calculation to determine if a response has expired is quite
simple:
response_is_fresh = (freshness_lifetime > current_age)
13.2.5 Disambiguating Expiration Values
Because expiration values are assigned optimistically, it is possible
for two caches to contain fresh values for the same resource that are
different.
If a client performing a retrieval receives a non-first-hand response
for a request that was already fresh in its own cache, and the Date
header in its existing cache entry is newer than the Date on the new
response, then the client MAY ignore the response. If so, it MAY
retry the request with a "Cache-Control: max-age=0" directive (see
section 14.9), to force a check with the origin server.
If a cache has two fresh responses for the same representation with
different validators, it MUST use the one with the more recent Date
header. This situation might arise because the cache is pooling
responses from other caches, or because a client has asked for a
reload or a revalidation of an apparently fresh cache entry.
13.2.6 Disambiguating Multiple Responses
Because a client might be receiving responses via multiple paths, so
that some responses flow through one set of caches and other
responses flow through a different set of caches, a client might
receive responses in an order different from that in which the origin
server sent them. We would like the client to use the most recently
generated response, even if older responses are still apparently
fresh.
Neither the entity tag nor the expiration value can impose an
ordering on responses, since it is possible that a later response
intentionally carries an earlier expiration time. The Date values are
ordered to a granularity of one second.
When a client tries to revalidate a cache entry, and the response it
receives contains a Date header that appears to be older than the one
for the existing entry, then the client SHOULD repeat the request
unconditionally, and include
Cache-Control: max-age=0
to force any intermediate caches to validate their copies directly
with the origin server, or
Cache-Control: no-cache
to force any intermediate caches to obtain a new copy from the origin
server.
If the Date values are equal, then the client MAY use either response
(or MAY, if it is being extremely prudent, request a new response).
Servers MUST NOT depend on clients being able to choose
deterministically between responses generated during the same second,
if their expiration times overlap.
13.3 Validation Model
When a cache has a stale entry that it would like to use as a
response to a client's request, it first has to check with the origin
server (or possibly an intermediate cache with a fresh response) to
see if its cached entry is still usable. We call this "validating"
the cache entry. Since we do not want to have to pay the overhead of
retransmitting the full response if the cached entry is good, and we
do not want to pay the overhead of an extra round trip if the cached
entry is invalid, the HTTP/1.1 protocol supports the use of
conditional methods.
The key protocol features for supporting conditional methods are
those concerned with "cache validators." When an origin server
generates a full response, it attaches some sort of validator to it,
which is kept with the cache entry. When a client (user agent or
proxy cache) makes a conditional request for a resource for which it
has a cache entry, it includes the associated validator in the
request.
The server then checks that validator against the current validator
for the entity, and, if they match (see section 13.3.3), it responds
with a special status code (usually, 304 (Not Modified)) and no
entity-body. Otherwise, it returns a full response (including
entity-body). Thus, we avoid transmitting the full response if the
validator matches, and we avoid an extra round trip if it does not
match.
In HTTP/1.1, a conditional request looks exactly the same as a normal
request for the same resource, except that it carries a special
header (which includes the validator) that implicitly turns the
method (usually, GET) into a conditional.
The protocol includes both positive and negative senses of cache-
validating conditions. That is, it is possible to request either that
a method be performed if and only if a validator matches or if and
only if no validators match.
Note: a response that lacks a validator may still be cached, and
served from cache until it expires, unless this is explicitly
prohibited by a cache-control directive. However, a cache cannot
do a conditional retrieval if it does not have a validator for the
entity, which means it will not be refreshable after it expires.
13.3.1 Last-Modified Dates
The Last-Modified entity-header field value is often used as a cache
validator. In simple terms, a cache entry is considered to be valid
if the entity has not been modified since the Last-Modified value.
13.3.2 Entity Tag Cache Validators
The ETag response-header field value, an entity tag, provides for an
"opaque" cache validator. This might allow more reliable validation
in situations where it is inconvenient to store modification dates,
where the one-second resolution of HTTP date values is not
sufficient, or where the origin server wishes to avoid certain
paradoxes that might arise from the use of modification dates.
Entity Tags are described in section 3.11. The headers used with
entity tags are described in sections 14.19, 14.24, 14.26 and 14.44.
13.3.3 Weak and Strong Validators
Since both origin servers and caches will compare two validators to
decide if they represent the same or different entities, one normally
would expect that if the entity (the entity-body or any entity-
headers) changes in any way, then the associated validator would
change as well. If this is true, then we call this validator a
"strong validator."
However, there might be cases when a server prefers to change the
validator only on semantically significant changes, and not when
insignificant aspects of the entity change. A validator that does not
always change when the resource changes is a "weak validator."
Entity tags are normally "strong validators," but the protocol
provides a mechanism to tag an entity tag as "weak." One can think of
a strong validator as one that changes whenever the bits of an entity
changes, while a weak value changes whenever the meaning of an entity
changes. Alternatively, one can think of a strong validator as part
of an identifier for a specific entity, while a weak validator is
part of an identifier for a set of semantically equivalent entities.
Note: One example of a strong validator is an integer that is
incremented in stable storage every time an entity is changed.
An entity's modification time, if represented with one-second
resolution, could be a weak validator, since it is possible that
the resource might be modified twice during a single second.
Support for weak validators is optional. However, weak validators
allow for more efficient caching of equivalent objects; for
example, a hit counter on a site is probably good enough if it is
updated every few days or weeks, and any value during that period
is likely "good enough" to be equivalent.
A "use" of a validator is either when a client generates a request
and includes the validator in a validating header field, or when a
server compares two validators.
Strong validators are usable in any context. Weak validators are only
usable in contexts that do not depend on exact equality of an entity.
For example, either kind is usable for a conditional GET of a full
entity. However, only a strong validator is usable for a sub-range
retrieval, since otherwise the client might end up with an internally
inconsistent entity.
Clients MAY issue simple (non-subrange) GET requests with either weak
validators or strong validators. Clients MUST NOT use weak validators
in other forms of request.
The only function that the HTTP/1.1 protocol defines on validators is