constructing an HTTP message, provided a Content-Length HTTP entity
header is allowed for the given application message by HTTP (see
Section 3.8.1).
3.4. DUM Message
A new named parameter for Data Use Mine (DUM) messages is introduced.
AM-Part: am-part
Figure 11
An OCP agent MUST send an AM-Part parameter with every DUM message
that is a part of an OCP transaction with an HTTP profile. The AM-
Part parameter value is a single am-part token. As implied by the
syntax, a DUM message can only contain data of a single application
message part. One message part can be fragmented into any number of
DUM messages with the same AM-Part parameter.
The following example shows three DUM messages containing an abridged
HTTP response message. The response-body part is fragmented and sent
within two DUM messages.
Example:
P: DUM 88 1 0
Kept: 0
AM-Part: response-header
64:HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 51
;
P: DUM 88 1 64
Kept: 64
AM-Part: response-body
19:<html><body>This is
;
P: DUM 88 1 83
Kept: 83
AM-Part: response-body
32: a simple message.</body></html>
;
Figure 12
3.5. Selective Adaptation
The HTTP profile for OCP applies to all HTTP messages. That scope
includes HTTP messages such as 1xx (Informational) responses, POST,
CONNECT, and OPTIONS requests, as well as responses with extension
status codes and requests with extension methods. Unless
specifically configured to do otherwise, an OPES processor MUST
forward all HTTP messages for adaptation at callout servers. OPES
bypass instructions, configured HTTP message handling rules, and
OCP-negotiation with a callout server are all examples of an
acceptable "specific configuration" that provides an exception to
this rule.
While it may seem useless to attempt to adapt "control" messages such
as a 100 (Continue) response, skipping such messages by default may
lead to serious security flaws and interoperability problems. For
example, sensitive company information might be relayed via a
carefully crafted 100 Continue response; or a malicious CONNECT
request may not get logged if OPES processor does not forward these
messages to a callout service that is supposed to handle them.
By design, OPES processor implementation cannot unilaterally decide
that an HTTP message is not worth adapting. It needs a callout
server opinion, a configuration setting, or another external
information to make the decision.
3.6. Hop-by-hop Headers
HTTP defines several hop-by-hop headers (e.g., Connection) and allows
for extension headers to be specified as hop-by-hop ones (via the
Connection header mechanism). Depending on the environment and
configuration, an OPES processor MAY forward hop-by-hop headers to
callout servers and MAY use hop-by-hop headers returned by callout
servers to build an HTTP message for the next application hop.
However, see Section 3.7 for requirements specific to the Transfer-
Encoding header.
For example, a logging or statistics collection service may want to
see hop-by-hop headers sent by the previous application hop to the
OPES processor and/or hop-by-hop headers sent by the OPES processor
to the next application hop. Another service may actually handle
HTTP logic of removing and adding hop-by-hop headers. Many services
will ignore hop-by-hop headers. This specification does not define a
mechanism for distinguishing these use cases.
3.7. Transfer Encodings
HTTP messages may use transfer encodings, a hop-by-hop encoding
feature of HTTP. Adaptations that use HTTP transfer encodings have
to be explicitly negotiated. This specification does not document
such negotiations. In the absence of explicit transfer-encoding
negotiations, an OCP agent MUST NOT send transfer-encoded application
message bodies.
Informally, the above rule means that the agent or its environment
have to make sure that all transfer encodings are stripped from an
HTTP message body before it enters OCP scope. An agent MUST
terminate the OCP transaction if it has to send an application
message body but cannot remove all transfer encodings. Violations of
these rules lead to interoperability problems.
If an OCP agent receives transfer-encoded application data in
violation of the above requirement, the agent MAY terminate the
corresponding OCP transaction.
An OPES processor removing transfer encodings MUST remove the
Transfer-Encoding header before sending the header part to the
callout service. A callout server receiving a Transfer-Encoding
header MAY assume that original application data is still transfer-
encoded (and terminate the transaction). The OPES processor MUST
send a correct Transfer-Encoding header to the next HTTP recipient,
independent of what header (if any) the callout server returned.
Logging and wiretapping are the examples where negotiating acceptable
transfer encodings may be worthwhile. While a callout server may not
be able to strip an encoding, it may still want to log the entire
message "as is". In most cases, however, the callout server would
not be able to meaningfully handle unknown transfer encodings.
3.8. HTTP Header Correctness
When communicating with HTTP applications, OPES processors MUST
ensure correctness of all computable HTTP headers documented in
specifications that the processors intend to be compliant with. A
computable header is defined as a header whose value can be computed
based on the message body alone. For example, the correctness of
Content-Length and Content-MD5 headers has to be ensured by
processors claiming compliance with HTTP/1.1 ([RFC2616]).
Informally and by default, the OPES processor has to validate and
eventually recalculate, add, or remove computable HTTP headers in
order to build a compliant HTTP message from an adapted application
message returned by the callout server. If a particular OPES
processor trusts certain HTTP headers that a callout service sends,
it can use those headers "as is".
An OPES processor MAY forward a partially adapted HTTP message from a
callout server to the next callout server, without verifying HTTP
header correctness. Consequently, a callout service cannot assume
that the HTTP headers it receives are correct or final from an HTTP
point of view.
The following subsections present guidelines for the recalculation of
some HTTP headers.
3.8.1. Message Size Recalculation
By default, an OCP agent MUST NOT trust the Content-Length header
that is sent within an HTTP header message part. The message length
could be modified by a callout service without adaptation of the HTTP
message headers.
Before sending the HTTP message to the HTTP peer, the OPES processor
has to ensure correctness of the message length indication according
to section 4.4 of [RFC2616].
Besides ensuring HTTP message correctness, good OPES processors set
up the message to optimize performance, including minimizing delivery
latency. Specifically, indicating the end of a message by closing
the HTTP connection ought to be the last resort:
o If the callout server sends an AM-EL parameter with its AMS
message, the OPES processor SHOULD use this value to create a
Content-Length header to be able to keep a persistent HTTP
connection. Note that HTTP rules prohibit a Content-Length header
to be used in transfer-encoded messages.
o If AM-EL parameter or equivalent entity length information is not
available, and HTTP rules allow for chunked transfer encoding, the
OPES processor SHOULD use chunked transfer encoding. Note that
any Content-Length header has to be removed in this case.
o If the message size is not known a priori and chunked transfer
coding cannot be used, but the OPES processor can wait for the OCP
transaction to finish before forwarding the adapted HTTP message
on a persistent HTTP connection, then the processor SHOULD compute
and add a Content-Length header.
o Finally, if all optimizations are not applicable, the OPES
processor SHOULD delete any Content-Length header and forward
adapted data immediately, while indicating the message end by
closing the HTTP connection.
3.8.2. Content-MD5 Header
By default, the OPES processor MUST assume that the callout service
modifies the content in a way that the MD5 checksum of the message
body becomes invalid.
According to section 14.15 of [RFC2616], HTTP intermediaries must not
generate Content-MD5 headers. A recalculation is therefore possible
only if the OPES processor is considered authoritative for the entity
being adapted. An un-authoritative OPES processor MUST remove the
Content-MD5 header unless it detects that the HTTP message was not
modified; in this case, it MAY leave the Content-MD5 header in the
message. When such detection significantly increases message
latency, deleting the Content-MD5 header may be a better option.
3.9. Examples
This is a possible OCP message flow using an HTTP request profile.
An end-user wants to access the home page of
www.restricted.example.com, through the proxy, but access is denied
by a URL blocking service running on the callout server used by the
proxy.
OCP messages from the OPES processor are marked with "P:" and OCP
messages from the callout server are marked with "S:". The OCP
connection is not closed at the end but kept open for the next OCP
transaction.
Example:
P: CS;
S: CS;
P: SGC 11 ({"31:ocp-test.example.com/url-filter"});
P: NO ({"53:http://www.iana.org/assignments/opes/ocp/http/request"})
SG: 11
;
S: NR {"53:http://www.iana.org/assignments/opes/ocp/http/request"}
SG: 11
;
P: TS 55 11;
P: AMS 55
AM-EL: 0
;
P: DUM 55 0
Kept: 0
AM-Part: request-header
235:GET http://www.restricted.example.com/ HTTP/1.1
Accept: */*
Accept-Language: de
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; Windows NT 5.0)
Host: www.restricted.example.com
Proxy-Connection: Keep-Alive
;
P: AME 55;
S: AMS 55;
S: DUM 55 0
AM-Part: response-header
76:HTTP/1.1 403 Forbidden
Content-Type: text/html
Proxy-Connection: close
;
S: DUM 55 0
AM-Part: response-body
67:<html><body>You are not allowed to
access this page.</body></html>
;
S: AME 55;
P: TE 55;
S: TE 55;
Figure 13
The next example is a language translation of a small plain text file
that gets transferred in an HTTP response. In this example, OCP
agents negotiate a profile for the whole OCP connection. The OCP
connection remains open in the end of the OCP transaction. (Note
that NO and NR messages were rendered with an extra new line to
satisfy RFC formatting requirements.)
Example:
P: CS;
S: CS;
P: NO
({"54:http://www.iana.org/assignments/opes/ocp/http/response"});
S: NR
{"54:http://www.iana.org/assignments/opes/ocp/http/response"};
P: SGC 12 ({"44:ocp-test.example.com/translate?from=EN&to=DE"});
P: TS 89 12;
P: AMS 89
AM-EL: 86
;
P: DUM 89 0
AM-Part: response-header
65:HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 86
;
P: DUM 89 65
AM-Part: response-body
86:Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous fortune
;
P: AME 89;
S: AMS 89
AM-EL: 78
;
P: TE 89;
S: DUM 89 0
AM-Part: response-header
65:HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 78
;
S: DUM 89 63
AM-Part: response-body
80:Ob’s edler im Gemuet, die Pfeil und Schleudern
des wuetenden Geschicks erdulden
;
S: AME 89;
S: TE 89;
Figure 14
The following example shows modification of an HTML resource and
demonstrates data preservation optimization. The callout server uses
a DUY message to send back an unchanged response header part, but
because it does not know the size of the altered HTML resource at the
time it sends the AMS message, the callout server omits the AM-EL
parameter; the OPES processor is responsible for adjusting the
Content-Length header.
Example:
P: CS;
S: CS;
P: SGC 10 ({"30:ocp-test.example.com/ad-filter"});
P: NO ({"54:http://www.iana.org/assignments/opes/ocp/http/response"
Aux-Parts: (request-header,request-body)
},{"45:http://www.iana.org/assignments/opes/ocp/MIME"})
SG: 10
;
S: NR {"54:http://www.iana.org/assignments/opes/ocp/http/response"
Aux-Parts: (request-header)
Content-Encodings: (gzip)
}
SG: 10
;
P: TS 88 10;
P: AMS 88
AM-EL: 95
;
P: DUM 88 0
AM-Part: request-header
65:GET /opes/adsample.html HTTP/1.1
Host: www.martin-stecher.de
;
P: DUM 88 65
Kept: 65 64
AM-Part: response-header
64:HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 95
;
P: DUM 88 129
Kept: 65 90
AM-Part: response-body
26:<html>
<body>
This is my
;
S: AMS 88;
P: DUM 88 155
Kept: 65 158
AM-Part: response-body
68: new ad: <img src="my_ad.gif"
width=88 height=31>
</body>
</html>
;
S: DUY 88 65 64
S: DPI 88 129 2147483647;
P: AME 88;
S: DUM 88 0
AM-Part: response-body
52:<html>
<body>
This is my new ad:
</body>
</html>
;
S: DPI 88 129 0;
P: TE 88;
S: AME 88;
S: TE 88;
Figure 15
4. Tracing
[RFC3897] defines application-agnostic tracing facilities in OPES.
Compliance with this specification requires compliance with
[RFC3897]. When adapting HTTP, trace entries are supplied using HTTP
message headers. The following HTTP extension headers are defined to
carry trace entries. Their definitions are given using BNF notation
and elements defined in [RFC2616].
OPES-System = "OPES-System" ":" #trace-entry
OPES-Via = "OPES-Via" ":" #trace-entry
trace-entry = opes-agent-id *( ";" parameter )
opes-agent-id = absoluteURI
Figure 16
An OPES System MUST add its trace entry to the OPES-System header.
Other OPES agents MUST use the OPES-Via header if they add their
tracing entries. All OPES agents MUST append their entries.
Informally, OPES-System is the only required OPES tracing header
while OPES-Via provides optional tracing details; both headers
reflect the order of trace entry additions.
If an OPES-Via header is used in the original application message, an
OPES System MUST append its entry to the OPES-Via header. Otherwise,
an OPES System MAY append its entry to the OPES-Via header. If an
OPES System is using both headers, it MUST add identical trace
entries except it MAY omit some or all trace-entry parameters from
the OPES-Via header. Informally, the OPES System entries in the
OPES-Via header are used to delimit and group OPES-Via entries from
different OPES Systems without having a priory knowledge about OPES
System identifiers.
Note that all of these headers are defined using #list constructs
and, hence, a valid HTTP message may contain multiple trace entries
per header. OPES agents SHOULD use a single header-field rather than
using multiple equally-named fields to record a long trace. Using
multiple equally-named extension header-fields is illegal from HTTP’s
point of view and may not work with some of the OPES-unaware HTTP
proxies.
For example, here is an HTTP response message header after OPES
adaptations have been applied by a single OPES processor executing 10
OPES services:
Example:
HTTP/1.1 200 OK
Date: Thu, 18 Sep 2003 06:25:24 GMT
Last-Modified: Wed, 17 Sep 2003 18:24:25 GMT
Content-type: application/octet-stream
OPES-System: http://www.cdn.example.com/opes?session=ac79a749f56
OPES-Via: http://www.cdn.example.com/opes?session=ac79a749f56,
http://www.srvcs-4u.example.com/cat/?sid=123,
http://www.srvcs-4u.example.com/cat/?sid=124,
http://www.srvcs-4u.example.com/cat/?sid=125 ; mode=A
Figure 17
In the above example, the OPES processor has not included its trace
entry or its trace entry was replaced by an OPES system trace entry.
Only 3 out of 10 services are traced. The remaining services did not
include their entries or their entries were removed by OPES system or
processor. The last traced service included a "mode" parameter.
Various identifiers in trace entries will probably have no meaning to
the recipient of the message, but may be decoded by OPES System
software.
OPES entities MAY place optional tracing entries in a message trailer
(i.e., entity-headers at the end of a Chunked-Body of a chunked-
encoded message), provided trailer presence does not violate HTTP
protocol. See [RFC3897] for a definition of what tracing entries are
optional. OPES entities MUST NOT place required tracing entries in a
message trailer.
5. Bypass
An HTTP extension header is introduced to allow for OPES system
bypass as defined in [RFC3897].
OPES-Bypass = "OPES-Bypass" ":" ( "*" | 1#bypass-entry )
bypass-entry = opes-agent-id
Figure 18
This header can be added to HTTP requests to request OPES system
bypass for the listed OPES agents. The asterisk "*" character is
used to represent all possible OPES agents.
See [RFC3897] for what can be bypassed and for bypass requirements.
6. IAB Considerations
OPES treatment of IETF Internet Architecture Board (IAB)
considerations [RFC3238] are documented in "OPES Treatment of IAB
Considerations" [RFC3914].
7. Security Considerations
Application-independent security considerations are documented in
application-agnostic OPES specifications. HTTP profiles do not
introduce any HTTP-specific security considerations. However, that
does not imply that HTTP adaptations are immune from security
threats.
Specific threat examples include such adaptations as rewriting the
Request-URI of an HTTP CONNECT request or removing an HTTP hop-by-hop
Upgrade header before the HTTP proxy can act on it. As with any
adaptation, the OPES agents MUST NOT perform such actions without
HTTP client or server consent.
8. IANA Considerations
The IANA registers request and response profile features (Section
3.2) using the registration procedure outlined in the "IANA
Considerations" Section of OCP Core [RFC4037]. The corresponding
"uri" parameters for the two features are:
o http://www.iana.org/assignments/opes/ocp/http/request
o http://www.iana.org/assignments/opes/ocp/http/response
9. Compliance
Compliance with OPES mechanisms is defined in corresponding
application-agnostic specifications. HTTP profiles for these
mechanisms use corresponding compliance definitions from these
specifications, as if each profile were incorporated into the
application-agnostic specification it profiles.
10. References
10.1. Normative References
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC3897] Barbir, A., "Open Pluggable Edge Services (OPES) Entities
and End Points Communication", RFC 3897, September 2004.
[RFC4037] Rousskov, A., "Open Pluggable Edge Services (OPES) Callout
Protocol (OCP) Core", RFC 4037, March 2005.
10.2. Informative References
[RFC3835] Barbir, A., Penno, R., Chen, R., Hofmann, M., and H.
Orman, "An Architecture for Open Pluggable Edge Services
(OPES)", RFC 3835, August 2004.
[RFC3836] Beck, A., Hofmann, M., Orman, H., Penno, R., and A.
Terzis, "Requirements for Open Pluggable Edge Services
(OPES) Callout Protocols", RFC 3836, August 2004.
[RFC3837] Barbir, A., Batuner, O., Srinivas, B., Hofmann, M., and H.
Orman, "Security Threats and Risks for Open Pluggable Edge
Services (OPES)", RFC 3837, August 2004.
[RFC3752] Barbir, A., Burger, E., Chen, R., McHenry, S., Orman, H.,
and R. Penno, "Open Pluggable Edge Services (OPES) Use
Cases and Deployment Scenarios", RFC 3752, April 2004.
[RFC3838] Barbir, A., Batuner, O., Beck, A., Chan, T., and H. Orman,
"Policy, Authorization, and Enforcement Requirements of
the Open Pluggable Edge Services (OPES)", RFC 3838, August
2004.
[rules-p] Beck, A. and A. Rousskov, "P: Message Processing
Language", work in progress, October 2003.
[RFC3914] Barbir, A. and A. Rousskov, "Open Pluggable Edge Services
(OPES) Treatment of IAB Considerations", RFC 3914, October
2004.
[RFC3238] Floyd, S. and L. Daigle, "IAB Architectural and Policy
Considerations for Open Pluggable Edge Services", RFC
3238, January 2002.
Acknowledgements
The authors gratefully acknowledge the contributions of Robert
Collins (Syncretize) and Larry Masinter (Adobe). Larry Masinter
provided an early review of this document.
Authors’ Addresses
Alex Rousskov
The Measurement Factory
EMail: rousskov@measurement-factory.com
URI: http://www.measurement-factory.com/
Martin Stecher
CyberGuard Corporation
Vattmannstr. 3
Paderborn 33100
DE
EMail: martin.stecher@webwasher.com
URI: http://www.webwasher.com/
Full Copyright Statement
Copyright (C) The Internet Society (2005).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,