protocols other than their usual retrieval context (e.g., email or
USENET news).
It is beyond the scope of this specification to specify how, for each
media type, a base URI can be embedded. The appropriate syntax, when
available, is described by the data format specification associated
with each media type.
5.1.2. Base URI from the Encapsulating Entity
If no base URI is embedded, the base URI is defined by the
representation’s retrieval context. For a document that is enclosed
within another entity, such as a message or archive, the retrieval
context is that entity. Thus, the default base URI of a
representation is the base URI of the entity in which the
representation is encapsulated.
A mechanism for embedding a base URI within MIME container types
(e.g., the message and multipart types) is defined by MHTML
[RFC2557]. Protocols that do not use the MIME message header syntax,
but that do allow some form of tagged metadata to be included within
messages, may define their own syntax for defining a base URI as part
of a message.
5.1.3. Base URI from the Retrieval URI
If no base URI is embedded and the representation is not encapsulated
within some other entity, then, if a URI was used to retrieve the
representation, that URI shall be considered the base URI. Note that
if the retrieval was the result of a redirected request, the last URI
used (i.e., the URI that resulted in the actual retrieval of the
representation) is the base URI.
5.1.4. Default Base URI
If none of the conditions described above apply, then the base URI is
defined by the context of the application. As this definition is
necessarily application-dependent, failing to define a base URI by
using one of the other methods may result in the same content being
interpreted differently by different types of applications.
A sender of a representation containing relative references is
responsible for ensuring that a base URI for those references can be
established. Aside from fragment-only references, relative
references can only be used reliably in situations where the base URI
is well defined.
5.2. Relative Resolution
This section describes an algorithm for converting a URI reference
that might be relative to a given base URI into the parsed components
of the reference’s target. The components can then be recomposed, as
described in Section 5.3, to form the target URI. This algorithm
provides definitive results that can be used to test the output of
other implementations. Applications may implement relative reference
resolution by using some other algorithm, provided that the results
match what would be given by this one.
5.2.1. Pre-parse the Base URI
The base URI (Base) is established according to the procedure of
Section 5.1 and parsed into the five main components described in
Section 3. Note that only the scheme component is required to be
present in a base URI; the other components may be empty or
undefined. A component is undefined if its associated delimiter does
not appear in the URI reference; the path component is never
undefined, though it may be empty.
Normalization of the base URI, as described in Sections 6.2.2 and
6.2.3, is optional. A URI reference must be transformed to its
target URI before it can be normalized.
5.2.2. Transform References
For each URI reference (R), the following pseudocode describes an
algorithm for transforming R into its target URI (T):
-- The URI reference is parsed into the five URI components
--
(R.scheme, R.authority, R.path, R.query, R.fragment) = parse(R);
-- A non-strict parser may ignore a scheme in the reference
-- if it is identical to the base URI’s scheme.
--
if ((not strict) and (R.scheme == Base.scheme)) then
undefine(R.scheme);
endif;
if defined(R.scheme) then
T.scheme = R.scheme;
T.authority = R.authority;
T.path = remove_dot_segments(R.path);
T.query = R.query;
else
if defined(R.authority) then
T.authority = R.authority;
T.path = remove_dot_segments(R.path);
T.query = R.query;
else
if (R.path == "") then
T.path = Base.path;
if defined(R.query) then
T.query = R.query;
else
T.query = Base.query;
endif;
else
if (R.path starts-with "/") then
T.path = remove_dot_segments(R.path);
else
T.path = merge(Base.path, R.path);
T.path = remove_dot_segments(T.path);
endif;
T.query = R.query;
endif;
T.authority = Base.authority;
endif;
T.scheme = Base.scheme;
endif;
T.fragment = R.fragment;
5.2.3. Merge Paths
The pseudocode above refers to a "merge" routine for merging a
relative-path reference with the path of the base URI. This is
accomplished as follows:
o If the base URI has a defined authority component and an empty
path, then return a string consisting of "/" concatenated with the
reference’s path; otherwise,
o return a string consisting of the reference’s path component
appended to all but the last segment of the base URI’s path (i.e.,
excluding any characters after the right-most "/" in the base URI
path, or excluding the entire base URI path if it does not contain
any "/" characters).
5.2.4. Remove Dot Segments
The pseudocode also refers to a "remove_dot_segments" routine for
interpreting and removing the special "." and ".." complete path
segments from a referenced path. This is done after the path is
extracted from a reference, whether or not the path was relative, in
order to remove any invalid or extraneous dot-segments prior to
forming the target URI. Although there are many ways to accomplish
this removal process, we describe a simple method using two string
buffers.
1. The input buffer is initialized with the now-appended path
components and the output buffer is initialized to the empty
string.
2. While the input buffer is not empty, loop as follows:
A. If the input buffer begins with a prefix of "../" or "./",
then remove that prefix from the input buffer; otherwise,
B. if the input buffer begins with a prefix of "/./" or "/.",
where "." is a complete path segment, then replace that
prefix with "/" in the input buffer; otherwise,
C. if the input buffer begins with a prefix of "/../" or "/..",
where ".." is a complete path segment, then replace that
prefix with "/" in the input buffer and remove the last
segment and its preceding "/" (if any) from the output
buffer; otherwise,
D. if the input buffer consists only of "." or "..", then remove
that from the input buffer; otherwise,
E. move the first path segment in the input buffer to the end of
the output buffer, including the initial "/" character (if
any) and any subsequent characters up to, but not including,
the next "/" character or the end of the input buffer.
3. Finally, the output buffer is returned as the result of
remove_dot_segments.
Note that dot-segments are intended for use in URI references to
express an identifier relative to the hierarchy of names in the base
URI. The remove_dot_segments algorithm respects that hierarchy by
removing extra dot-segments rather than treat them as an error or
leaving them to be misinterpreted by dereference implementations.
The following illustrates how the above steps are applied for two
examples of merged paths, showing the state of the two buffers after
each step.
STEP OUTPUT BUFFER INPUT BUFFER
1 : /a/b/c/./../../g
2E: /a /b/c/./../../g
2E: /a/b /c/./../../g
2E: /a/b/c /./../../g
2B: /a/b/c /../../g
2C: /a/b /../g
2C: /a /g
2E: /a/g
STEP OUTPUT BUFFER INPUT BUFFER
1 : mid/content=5/../6
2E: mid /content=5/../6
2E: mid/content=5 /../6
2C: mid /6
2E: mid/6
Some applications may find it more efficient to implement the
remove_dot_segments algorithm by using two segment stacks rather than
strings.
Note: Beware that some older, erroneous implementations will fail
to separate a reference’s query component from its path component
prior to merging the base and reference paths, resulting in an
interoperability failure if the query component contains the
strings "/../" or "/./".
5.3. Component Recomposition
Parsed URI components can be recomposed to obtain the corresponding
URI reference string. Using pseudocode, this would be:
result = ""
if defined(scheme) then
append scheme to result;
append ":" to result;
endif;
if defined(authority) then
append "//" to result;
append authority to result;
endif;
append path to result;
if defined(query) then
append "?" to result;
append query to result;
endif;
if defined(fragment) then
append "#" to result;
append fragment to result;
endif;
return result;
Note that we are careful to preserve the distinction between a
component that is undefined, meaning that its separator was not
present in the reference, and a component that is empty, meaning that
the separator was present and was immediately followed by the next
component separator or the end of the reference.
5.4. Reference Resolution Examples
Within a representation with a well defined base URI of
http://a/b/c/d;p?q
a relative reference is transformed to its target URI as follows.
5.4.1. Normal Examples
"g:h" = "g:h"
"g" = "http://a/b/c/g"
"./g" = "http://a/b/c/g"
"g/" = "http://a/b/c/g/"
"/g" = "http://a/g"
"//g" = "http://g"
"?y" = "http://a/b/c/d;p?y"
"g?y" = "http://a/b/c/g?y"
"#s" = "http://a/b/c/d;p?q#s"
"g#s" = "http://a/b/c/g#s"
"g?y#s" = "http://a/b/c/g?y#s"
";x" = "http://a/b/c/;x"
"g;x" = "http://a/b/c/g;x"
"g;x?y#s" = "http://a/b/c/g;x?y#s"
"" = "http://a/b/c/d;p?q"
"." = "http://a/b/c/"
"./" = "http://a/b/c/"
".." = "http://a/b/"
"../" = "http://a/b/"
"../g" = "http://a/b/g"
"../.." = "http://a/"
"../../" = "http://a/"
"../../g" = "http://a/g"
5.4.2. Abnormal Examples
Although the following abnormal examples are unlikely to occur in
normal practice, all URI parsers should be capable of resolving them
consistently. Each example uses the same base as that above.
Parsers must be careful in handling cases where there are more ".."
segments in a relative-path reference than there are hierarchical
levels in the base URI’s path. Note that the ".." syntax cannot be
used to change the authority component of a URI.
"../../../g" = "http://a/g"
"../../../../g" = "http://a/g"
Similarly, parsers must remove the dot-segments "." and ".." when
they are complete components of a path, but not when they are only
part of a segment.
"/./g" = "http://a/g"
"/../g" = "http://a/g"
"g." = "http://a/b/c/g."
".g" = "http://a/b/c/.g"
"g.." = "http://a/b/c/g.."
"..g" = "http://a/b/c/..g"
Less likely are cases where the relative reference uses unnecessary
or nonsensical forms of the "." and ".." complete path segments.
"./../g" = "http://a/b/g"
"./g/." = "http://a/b/c/g/"
"g/./h" = "http://a/b/c/g/h"
"g/../h" = "http://a/b/c/h"
"g;x=1/./y" = "http://a/b/c/g;x=1/y"
"g;x=1/../y" = "http://a/b/c/y"
Some applications fail to separate the reference’s query and/or
fragment components from the path component before merging it with
the base path and removing dot-segments. This error is rarely
noticed, as typical usage of a fragment never includes the hierarchy
("/") character and the query component is not normally used within
relative references.
"g?y/./x" = "http://a/b/c/g?y/./x"
"g?y/../x" = "http://a/b/c/g?y/../x"
"g#s/./x" = "http://a/b/c/g#s/./x"
"g#s/../x" = "http://a/b/c/g#s/../x"
Some parsers allow the scheme name to be present in a relative
reference if it is the same as the base URI scheme. This is
considered to be a loophole in prior specifications of partial URI
[RFC1630]. Its use should be avoided but is allowed for backward
compatibility.
"http:g" = "http:g" ; for strict parsers
/ "http://a/b/c/g" ; for backward compatibility
6. Normalization and Comparison
One of the most common operations on URIs is simple comparison:
determining whether two URIs are equivalent without using the URIs to
access their respective resource(s). A comparison is performed every
time a response cache is accessed, a browser checks its history to
color a link, or an XML parser processes tags within a namespace.
Extensive normalization prior to comparison of URIs is often used by
spiders and indexing engines to prune a search space or to reduce
duplication of request actions and response storage.
URI comparison is performed for some particular purpose. Protocols
or implementations that compare URIs for different purposes will
often be subject to differing design trade-offs in regards to how
much effort should be spent in reducing aliased identifiers. This
section describes various methods that may be used to compare URIs,
the trade-offs between them, and the types of applications that might
use them.
6.1. Equivalence
Because URIs exist to identify resources, presumably they should be
considered equivalent when they identify the same resource. However,
this definition of equivalence is not of much practical use, as there
is no way for an implementation to compare two resources unless it
has full knowledge or control of them. For this reason,
determination of equivalence or difference of URIs is based on string
comparison, perhaps augmented by reference to additional rules
provided by URI scheme definitions. We use the terms "different" and
"equivalent" to describe the possible outcomes of such comparisons,
but there are many application-dependent versions of equivalence.
Even though it is possible to determine that two URIs are equivalent,
URI comparison is not sufficient to determine whether two URIs
identify different resources. For example, an owner of two different
domain names could decide to serve the same resource from both,
resulting in two different URIs. Therefore, comparison methods are
designed to minimize false negatives while strictly avoiding false
positives.
In testing for equivalence, applications should not directly compare
relative references; the references should be converted to their
respective target URIs before comparison. When URIs are compared to
select (or avoid) a network action, such as retrieval of a
representation, fragment components (if any) should be excluded from
the comparison.
6.2. Comparison Ladder
A variety of methods are used in practice to test URI equivalence.
These methods fall into a range, distinguished by the amount of
processing required and the degree to which the probability of false
negatives is reduced. As noted above, false negatives cannot be
eliminated. In practice, their probability can be reduced, but this
reduction requires more processing and is not cost-effective for all
applications.
If this range of comparison practices is considered as a ladder, the
following discussion will climb the ladder, starting with practices
that are cheap but have a relatively higher chance of producing false
negatives, and proceeding to those that have higher computational
cost and lower risk of false negatives.
6.2.1. Simple String Comparison
If two URIs, when considered as character strings, are identical,
then it is safe to conclude that they are equivalent. This type of
equivalence test has very low computational cost and is in wide use
in a variety of applications, particularly in the domain of parsing.
Testing strings for equivalence requires some basic precautions.
This procedure is often referred to as "bit-for-bit" or
"byte-for-byte" comparison, which is potentially misleading. Testing
strings for equality is normally based on pair comparison of the
characters that make up the strings, starting from the first and
proceeding until both strings are exhausted and all characters are
found to be equal, until a pair of characters compares unequal, or
until one of the strings is exhausted before the other.
This character comparison requires that each pair of characters be
put in comparable form. For example, should one URI be stored in a
byte array in EBCDIC encoding and the second in a Java String object
(UTF-16), bit-for-bit comparisons applied naively will produce
errors. It is better to speak of equality on a character-for-
character basis rather than on a byte-for-byte or bit-for-bit basis.
In practical terms, character-by-character comparisons should be done
codepoint-by-codepoint after conversion to a common character
encoding.
False negatives are caused by the production and use of URI aliases.
Unnecessary aliases can be reduced, regardless of the comparison
method, by consistently providing URI references in an already-
normalized form (i.e., a form identical to what would be produced
after normalization is applied, as described below).
Protocols and data formats often limit some URI comparisons to simple
string comparison, based on the theory that people and
implementations will, in their own best interest, be consistent in
providing URI references, or at least consistent enough to negate any
efficiency that might be obtained from further normalization.
6.2.2. Syntax-Based Normalization
Implementations may use logic based on the definitions provided by
this specification to reduce the probability of false negatives.
This processing is moderately higher in cost than character-for-
character string comparison. For example, an application using this
approach could reasonably consider the following two URIs equivalent:
example://a/b/c/%7Bfoo%7D
eXAMPLE://a/./b/../b/%63/%7bfoo%7d
Web user agents, such as browsers, typically apply this type of URI
normalization when determining whether a cached response is
available. Syntax-based normalization includes such techniques as
case normalization, percent-encoding normalization, and removal of
dot-segments.
6.2.2.1. Case Normalization
For all URIs, the hexadecimal digits within a percent-encoding
triplet (e.g., "%3a" versus "%3A") are case-insensitive and therefore
should be normalized to use uppercase letters for the digits A-F.
When a URI uses components of the generic syntax, the component
syntax equivalence rules always apply; namely, that the scheme and
host are case-insensitive and therefore should be normalized to
lowercase. For example, the URI <HTTP://www.EXAMPLE.com/> is
equivalent to <http://www.example.com/>. The other generic syntax
components are assumed to be case-sensitive unless specifically
defined otherwise by the scheme (see Section 6.2.3).
6.2.2.2. Percent-Encoding Normalization
The percent-encoding mechanism (Section 2.1) is a frequent source of
variance among otherwise identical URIs. In addition to the case
normalization issue noted above, some URI producers percent-encode
octets that do not require percent-encoding, resulting in URIs that
are equivalent to their non-encoded counterparts. These URIs should
be normalized by decoding any percent-encoded octet that corresponds
to an unreserved character, as described in Section 2.3.
6.2.2.3. Path Segment Normalization
The complete path segments "." and ".." are intended only for use
within relative references (Section 4.1) and are removed as part of
the reference resolution process (Section 5.2). However, some
deployed implementations incorrectly assume that reference resolution
is not necessary when the reference is already a URI and thus fail to
remove dot-segments when they occur in non-relative paths. URI
normalizers should remove dot-segments by applying the
remove_dot_segments algorithm to the path, as described in
Section 5.2.4.
6.2.3. Scheme-Based Normalization
The syntax and semantics of URIs vary from scheme to scheme, as
described by the defining specification for each scheme.
Implementations may use scheme-specific rules, at further processing
cost, to reduce the probability of false negatives. For example,
because the "http" scheme makes use of an authority component, has a
default port of "80", and defines an empty path to be equivalent to
"/", the following four URIs are equivalent:
http://example.com
http://example.com/
/example.com:/">http://example.com:/
http://example.com:80/
In general, a URI that uses the generic syntax for authority with an
empty path should be normalized to a path of "/". Likewise, an
explicit ":port", for which the port is empty or the default for the
scheme, is equivalent to one where the port and its ":" delimiter are