Request for Comments: 4236 The Measurement Factory
Category: Standards Track M. Stecher
CyberGuard Corporation
November 2005
HTTP Adaptation with Open Pluggable Edge Services (OPES)
Status of This Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2005).
Abstract
Open Pluggable Edge Services (OPES) framework documents several
application-agnostic mechanisms such as OPES tracing, OPES bypass,
and OPES callout protocol. This document extends those generic
mechanisms for Hypertext Transfer Protocol (HTTP) adaptation.
Together, application-agnostic OPES documents and this HTTP profile
constitute a complete specification for HTTP adaptation with OPES.
Table of Contents
1. Scope ...........................................................3
2. OPES Document Map ...............................................3
3. Callout Protocol ................................................4
3.1. Application Message Parts ..................................5
3.2. Application Profile Features ...............................6
3.2.1. Profile Parts .......................................6
3.2.2. Profile Structure ...................................8
3.2.3. Aux-Parts ...........................................8
3.2.4. Pause-At-Body .......................................9
3.2.5. Stop-Receiving-Body ................................10
3.2.6. Preservation-Interest-Body .........................10
3.2.7. Content-Encodings ..................................11
3.2.8. Profile Negotiation Example ........................12
3.3. Application Message Start Message .........................13
3.4. DUM Message ...............................................13
3.5. Selective Adaptation ......................................14
3.6. Hop-by-hop Headers ........................................15
3.7. Transfer Encodings ........................................15
3.8. HTTP Header Correctness ...................................16
3.8.1. Message Size Recalculation .........................16
3.8.2. Content-MD5 Header .................................17
3.9. Examples ..................................................18
4. Tracing ........................................................22
5. Bypass .........................................................24
6. IAB Considerations .............................................24
7. Security Considerations ........................................24
8. IANA Considerations ............................................24
9. Compliance .....................................................25
10. References ....................................................25
10.1. Normative References .....................................25
10.2. Informative References ...................................25
1. Scope
The Open Pluggable Edge Services (OPES) framework documents several
application-agnostic mechanisms such as OPES processor and endpoints
communications [RFC3897] or OPES callout protocol [RFC4037]. This
document extends those generic mechanisms for adaptation of a
specific application protocol, HTTP [RFC2616]. Together,
application-agnostic OPES documents and this HTTP profile constitute
a complete specification for HTTP adaptation with OPES.
The primary sections of this document specify HTTP-specific
extensions for the corresponding application-agnostic mechanisms
documented elsewhere.
2. OPES Document Map
This document belongs to a large set of OPES specifications produced
by the IETF OPES Working Group. Familiarity with the overall OPES
approach and typical scenarios is often essential when trying to
comprehend isolated OPES documents. This section provides an index
of OPES documents to assist the reader with finding "missing"
information.
o The document on "OPES Use Cases and Deployment Scenarios"
[RFC3752] describes a set of services and applications that are
considered in scope for OPES and have been used as a motivation
and guidance in designing the OPES architecture.
o The OPES architecture and common terminology are described in "An
Architecture for Open Pluggable Edge Services (OPES)" [RFC3835].
o "Policy, Authorization and Enforcement Requirements of OPES"
[RFC3838] outlines requirements and assumptions on the policy
framework, without specifying concrete authorization and
enforcement methods.
o "Security Threats and Risks for OPES" [RFC3837] provides OPES risk
analysis, without recommending specific solutions.
o "OPES Treatment of IAB Considerations" [RFC3914] addresses all
architecture-level considerations expressed by the IETF Internet
Architecture Board (IAB) when the OPES WG was chartered.
o At the core of the OPES architecture are the OPES processor and
the callout server, two network elements that communicate with
each other via an OPES Callout Protocol (OCP). The requirements
for such protocol are discussed in "Requirements for OPES Callout
Protocols" [RFC3836].
o "OPES Callout Protocol Core" [RFC4037] specifies an application
agnostic protocol core to be used for the communication between
OPES processor and callout server.
o "OPES entities and end points communications" [RFC3897] specifies
generic tracing and bypass mechanisms for OPES.
o The OCP Core and Communications documents are independent from the
application protocol being adapted by OPES entities. Their
generic mechanisms have to be complemented by application-specific
profiles. This document, HTTP adaptation with OPES, is such an
application profile for HTTP. It specifies how application-
agnostic OPES mechanisms are to be used and augmented in order to
support adaptation of HTTP messages.
o Finally, "P: Message Processing Language" [rules-p] defines a
language for specifying what OPES adaptations (e.g., translation)
must be applied to what application messages (e.g., e-mail from
bob@example.com). P language is meant for configuring application
proxies (OPES processors).
3. Callout Protocol
This section documents the HTTP profile for the OPES Callout Protocol
(OCP) Core [RFC4037]. Familiarity with OCP Core is required to
understand the HTTP profile. This section uses OCP Core conventions,
terminology, and mechanisms.
OPES processor communicates its desire to adapt HTTP messages via a
Negotiation Offer (NO) message with HTTP-specific feature identifiers
documented in Section 3.2. HTTP-specific OCP optimization mechanisms
can be negotiated at the same time. A callout server that supports
adaptation of HTTP messages has a chance to negotiate what HTTP
message parts will participate in adaptation, including negotiation
of HTTP request parts as metadata for HTTP response adaptation.
Negotiable HTTP message parts are documented in Section 3.1.
HTTP profile introduces a new parameter for the Application Message
Start (AMS) message to communicate known HTTP message length (HTTP
headers often do not convey length information reliably or at all).
This parameter is documented in Section 3.3. Section 3.4 documents a
mechanism to report HTTP message parts with Data Use Mine (DUM)
messages.
The remaining OCP sections document various OCP marshaling corner
cases such as handling of HTTP transfer encodings and 100 Continue
responses.
3.1. Application Message Parts
An HTTP message may have several well-known parts: headers, body, and
trailers. HTTP OPES processors are likely to have information about
HTTP message parts because they have to isolate and interpret HTTP
headers and find HTTP message boundaries. Callout servers may either
not care about certain parts or may benefit from reusing HTTP OPES
processor work on isolating and categorizing interesting parts.
The following is the declaration of am-part (application message
part) type using OCP Core Protocol Element Type Declaration Mnemonic
(PETDM):
am-part: extends atom;
am-parts: extends list of am-part;
Figure 1
The following six "am-part" atoms are valid values:
request-header: The start-line of an HTTP request message, all
request message headers, and the CRLF separator at the end of HTTP
headers (compare with section 4.1 of [RFC2616]).
request-body: The message body of an HTTP request message as defined
in section 4.3 of [RFC2616] but not including the trailer.
request-trailer: The entity headers of the trailer of an HTTP request
message in chunked transfer encoding. This part follows the same
syntax as the trailer defined in section 3.6.1 of [RFC2616].
response-header: The start-line of an HTTP response message, all
response message headers, and the CRLF separator at the end of
HTTP headers (compare with section 4.1 of [RFC2616]).
response-body: The message body of an HTTP response message as
defined in section 4.3 of [RFC2616] but not including the trailer.
response-trailer: The entity headers of the trailer of an HTTP
response message in chunked transfer encoding. This part follows
the same syntax as the trailer defined in section 3.6.1 of
[RFC2616].
3.2. Application Profile Features
This document defines two HTTP profiles for OCP: request and response
profiles. These two profiles are described below. Each profile has
a unique feature identifier, a list of original application message
parts, and a list of adapted application message parts:
profile ID: http://www.iana.org/assignments/opes/ocp/http/request
original request parts: request-header, request-body, request-
trailer
adapted request parts: request-header, request-body, request-
trailer
adapted response parts: response-header, response-body, response-
trailer
profile ID: http://www.iana.org/assignments/opes/ocp/http/response
original transaction parts: request-header (aux), request-body
(aux), request-trailer (aux), response-header, response-body,
response-trailer
adapted response parts: response-header, response-body, response-
trailer
The request profile contains two variants of adapted part lists: HTTP
request parts and HTTP response parts. Parts marked with an "(aux)"
suffix are auxiliary parts that can only be used if explicitly
negotiated for a profile. See Section 3.2.1 for specific rules
governing negotiation and use of am-parts.
The scope of a negotiated profile is the OCP connection (default) or
the service group specified via the SG parameter.
3.2.1. Profile Parts
An OCP agent MUST send application message parts in the order implied
by the profile parts lists above. An OCP agent receiving an out-of-
order part MAY terminate the transaction with an error.
An OPES processor MUST NOT send parts that are not listed as
"original" in the negotiated profile. A callout server MUST NOT send
parts that are not listed as "adapted" in the negotiated profile. An
OCP agent receiving an not-listed part MUST terminate the transaction
with an error. The informal rationale for the last requirement is to
reduce the number of subtle interoperability problems where an agent
thinks that the parts it is sending are understood/used by the other
agent when, in fact, they are being ignored or skipped because they
are not expected.
Some HTTP messages lack certain parts. For example, many HTTP
requests do not have bodies, and most HTTP messages do not have
trailers. An OCP agent MUST NOT send (i.e., must skip) absent
application message parts.
An OCP agent MUST send present non-auxiliary parts and it MUST send
those present auxiliary parts that were negotiated via the Aux-Parts
(Section 3.2.3) parameter. OCP agents MUST NOT send auxiliary parts
that were not negotiated via the Aux-Parts (Section 3.2.3) parameter.
An OCP agent receiving a message part in violation of the above
requirements MAY terminate the corresponding transaction with an
error.
By design, original parts not included in the adapted parts list
cannot be adapted. In other words, a callout service can only adapt
parts in the adapted parts list even though it may have access to
other parts.
In the request profile, the callout server MUST send either adapted
request parts or adapted response parts. An OPES processor receiving
adapted flow with application message parts from both lists (in
violation of the previous rule) MUST terminate the OCP transaction
with an error. Informally, the callout server sends adapted response
parts to "short-circuit" the HTTP transaction, forcing the OPES
processor to return an HTTP response without forwarding an adapted
HTTP request. This short-circuiting is useful for responding, for
example, to an HTTP request that the callout service defines as
forbidden.
Unless explicitly configured to do otherwise, an OPES processor MUST
offer all non-auxiliary original parts in Negotiation Offer (NO)
messages. See Section 3.5 for this rule rationale and examples of
harmful side-effects from selective adaptation.
3.2.2. Profile Structure
An HTTP application profile feature extends semantics of the feature
type of OCP Core while adding the following named parameters to that
type:
o Aux-Parts (Section 3.2.3)
o Pause-At-Body (Section 3.2.4)
o Stop-Receiving-Body (Section 3.2.5)
o Preservation-Interest-Body (Section 3.2.6)
o Content-Encodings (Section 3.2.7)
The definition of the HTTP profile feature structure using PETDM
follows:
HTTP-Profile: extends Feature with {
[Aux-Parts: am-parts];
[Pause-At-Body: size];
[Stop-Receiving-Body: size];
[Preservation-Interest-Body: size];
[Content-Encodings: codings];
};
Figure 2
An HTTP profile structure can be used in feature lists of Negotiation
Offer (NO) messages and as an anonymous parameter of a Negotiation
Response (NR) message. All profile parameters apply to any OCP
transaction within profile scope.
3.2.3. Aux-Parts
The Aux-Parts parameter of an HTTP response profile can be used to
negotiate the inclusion of auxiliary application message parts into
the original data flow. The parameter is a possibly empty list of
am-part tokens. An OPES processor MAY send an Aux-Parts parameter to
advertise availability of auxiliary application message parts. A
callout server MAY respond with a possibly empty subset of the parts
it needs. The callout server response defines the subset of
successfully negotiated auxiliary message parts.
When receiving a Negotiation Offer (NO) message, the callout server
MUST ignore any non-auxiliary part listed in the Aux-Parts parameter.
When sending a Negotiation Response (NR) message, the callout server
MUST NOT select any application message part that was not explicitly
listed in the negotiation offer. In case of a violation of the last
rule, the OPES processor MUST terminate the transaction.
An OPES processor MUST send each negotiated auxiliary part to the
callout server, unless the part is absent.
Example:
Aux-Parts: (request-header,request-body)
Figure 3
3.2.4. Pause-At-Body
A callout server MAY use the Pause-At-Body parameter to request a
pause in original application message body transmission before
original dataflow starts. The parameter’s value is of type "offset".
The parameter specifies the start of the non-auxiliary application
message body suffix that the sender is temporarily not interested in
seeing.
[headers][ body prefix | body suffix ][trailer]
<-- ? --><-- offset --><-- ? ---------------->
<-- equiv. DWP offset ->
Figure 4
When an OPES processor receives a Pause-At-Body parameter, it MUST
behave as if it has received a Want Data Paused (DWP) message with
the corresponding org-offset. Note that the latter offset is
different from the Pause-At-Body offset and is unknown until the size
of the HTTP message headers is known.
For example, if the Pause-At-Body value is zero, the OPES processor
should send a Paused My Data (DPM) message just before it sends the
first Data Use Mine (DUM) message with the response-body part in the
HTTP response profile. If the Pause-At-Body value is 300, the OPES
processor should send a DPM message after transmitting 300 OCTETs for
that application message part.
Example:
Pause-At-Body: 0
Figure 5
3.2.5. Stop-Receiving-Body
A callout server MAY use the Stop-Receiving-Body parameter to imply a
Want Stop Receiving Data (DWSR) message behavior before the original
dataflow starts. The parameter’s value is of type "offset". The
parameter specifies an offset into the original, non-auxiliary
message body part (request-body in request profile and response-body
in response profile).
A callout service MAY send a Stop-Receiving-Body parameter with its
negotiation response if there is a fixed offset into the message body
for all transactions of a profile for which a Want Stop Receiving
Data (DWSR) message would be sent. An OPES processor MUST behave as
if it has received a DWSR message with the corresponding offset.
Note that the latter offset is different from the Stop-Receiving-Body
offset and is unknown until the size of the HTTP message headers is
known.
For example, if the Stop-Receiving-Body value is zero in an HTTP
response profile, the OPES processor should send an Application
Message End (AME) message with result code 206 immediately after
sending the response-header message part and before starting with the
response-body message part.
Example:
Stop-Receiving-Body: 0
Figure 6
3.2.6. Preservation-Interest-Body
The Preservation-Interest-Body parameter can be used to optimize data
preservation at the OPES processor. The parameter’s value is of type
"size" and denominates a prefix size of the original, non-auxiliary
message body part (request-body in HTTP request profile and
response-body in response profile).
A callout service MAY send a Preservation-Interest-Body parameter
with its negotiation response if there is a fixed-size prefix of the
application message body for which a Data Preservation Interest (DPI)
message would be sent. An OPES processor MUST behave as if it
receives a DPI message with org-offset zero and org-size equal to the
value of the Preservation-Interest-Body parameter.
For example, if the Preservation-Interest-Body value is zero in an
HTTP response profile, the callout server must not send any Data Use
Yours (DUY) message for the response-body part; the OPES processor
may use this information to optimize its data preservation behavior
even before it makes the decision to preserve data.
Example:
Preservation-Interest-Body: 0
Figure 7
3.2.7. Content-Encodings
A callout server MAY send a Content-Encodings list to indicate its
preferences in content encodings. Encodings listed first are
preferred to other encodings. An OPES processor MAY use any content
encoding when sending application messages to a callout server.
The list of preferred content encodings does not imply lack of
support for other encodings. The OPES processor MUST NOT bypass a
service just because the actual content encoding does not match the
service’s preferences.
If an OCP agent receives an application message that it cannot handle
due to specific content encoding, the usual transaction termination
rules apply.
content-coding: extends atom;
content-codings: extends list of content-coding;
Example:
Content-Encodings: (gzip)
Figure 8
The semantics of content-coding is defined in section 3.5 of
[RFC2616].
3.2.8. Profile Negotiation Example
Example:
P: NO ({"54:http://www.iana.org/assignments/opes/ocp/http/response"
Aux-Parts: (request-header,request-body)
})
SG: 5
;
S: NR {"54:http://www.iana.org/assignments/opes/ocp/http/response"
Aux-Parts: (request-header)
Pause-At-Body: 30
Preservation-Interest-Body: 0
Content-Encodings: (gzip)
}
SG: 5
;
Figure 9
This example shows a negotiation offer made by an OPES processor for
a service group (id 5) that has already been created; the callout
server sends an adequate negotiation response.
The OPES processor offers one profile feature for HTTP response
messages. Besides the standard message parts, the OPES processor is
able to add the header and body of the original HTTP request as
auxiliary message parts.
The callout server requests the auxiliary request-header part, but is
not interested in receiving the request-body part.
The OPES processor sends at most the following message parts, in the
specified order, for all transactions in service group 5: request-
header, response-header, response-body, response-trailer. Note that
the request-body part is not included (because it is an auxiliary
part that was not explicitly requested). Some of the response parts
may not be sent if the original message lacks them.
The callout server indicates through the Preservation-Interest-Body
parameter with size zero that it will not send any DUY messages. The
OPES processor may therefore preserve no preservation for any
transaction of this profile.
By sending a Pause-At-Body value of 30, the callout server requests a
data pause. The OPES processor sends a Paused My Data (DPM) message
immediately after sending at least 30 OCTETs of the response-body
part. Thereafter, the OPES processor waits for a Want More Data
(DWM) message from the callout service.
3.3. Application Message Start Message
A new named parameter for Application Message Start (AMS) messages is
introduced.
AM-EL: size
Figure 10
AM-EL value is the size of the request-body part in the HTTP request
profile, and is the size of the response-body part in the HTTP
response profile, before any transfer codings have been applied (or
after all transfer codings have been removed). This definition is
consistent with the HTTP entity length definition.
An OCP agent that knows the exact length of the HTTP message entity
(see Section 7.2.2 "Entity Length" in [RFC2616]) at the time it sends
the AMS message, SHOULD announce this length using the AM-EL named
parameter of an AMS message. If the exact entity length is not
known, an OCP agent MUST NOT send an AM-EL parameter. Relaying
correct entity length can have significant performance advantages for
the recipient, and implementations are strongly encouraged to relay
known entity lengths. Similarly, relaying incorrect entity length
can have drastic correctness consequences for the recipient, and
implementations are urged to exercise great care when relaying entity
length.
An OPES processor receiving an AM-EL parameter SHOULD use the
parameter’s value in a Content-Length HTTP entity header when