depending on processor settings and state.
When an OPES processor commits to keeping a data chunk, it announces
its decision and the chunk parameters via a Kept parameter of a Data
Use Mine (DUM) message. The callout server MAY "use" the chunk by
sending a Data Use Yours (DUY) message referring to the preserved
chunk. That OCP message does not have payload and, therefore, the
return trip is eliminated.
As the mapping between original and adapted data is not known to the
processor, the processor MUST keep the announced-as-preserved chunk
until the end of the corresponding transaction, unless the callout
server explicitly tells the processor that the chunk is not needed.
As implied by the above requirement, the processor cannot assume that
a data chunk is no longer needed just because the callout server sent
a Data Use Yours (DUY) message or adapted data with, for instance,
the same offset as the preserved chunk.
For simplicity, preserved data is always a contiguous chunk of
original data, described by an (offset, size) pair using a "Kept"
parameter of a Data Use Mine (DUM) message. An OPES processor may
volunteer to increase the size of the kept data. An OPES processor
may increase the offset if the callout server indicated that the kept
data is no longer needed.
Both agents may benefit from data reuse. An OPES processor has to
allocate storage to support this optimization, but a callout server
does not. On the other hand, it is the callout server that is
responsible for relieving the processor from data preservation
commitments. There is no simple way to resolve this conflict of
interest on a protocol level. Some OPES processors may allocate a
relatively small buffer for data preservation purposes and stop
preserving data when the buffer becomes full. This technique would
benefit callout services that can quickly reuse or discard kept data.
Another processor strategy would be to size the buffer based on
historical data reuse statistics. To improve chances of beneficial
cooperation, callout servers are strongly encouraged to immediately
notify OPES processors of unwanted data. The callout server that
made a decision not to send Data Use Yours (DUY) messages (for a
specific data ranges or at all) SHOULD immediately inform the OPES
processor of that decision with the corresponding Data Preservation
Interest (DPI) message(s) or other mechanisms.
8. ’Premature Dataflow Termination’ Optimizations
Many callout services adapt small portions of large messages and
would preferably not to be in the loop when that adaptation is over.
Some callout services may not seek data modification and would
preferably not send data back to the OPES processor, even if the OPES
processor is not supporting the data preservation optimization
(Section 7). By OCP design, unilateral premature dataflow
termination by a callout server would lead to termination of an OCP
transaction with an error. Thus, the two agents must cooperate to
allow for error-free premature termination.
This section documents two mechanisms for premature termination of
original or adapted dataflow. In combination, the mechanisms allow
the callout server to get out of the processing loop altogether.
8.1. Original Dataflow
There are scenarios where a callout server is not interested in the
remaining original dataflow. For example, a simple access blocking
or "this site is temporary down" callout service has to send an
adapted (generated) application message but would preferably not
receive original data from the OPES processor.
OCP Core supports premature original dataflow termination via the
Want Stop Receiving Data (DWSR) message. A callout server that does
not seek to receive additional original data (beyond a certain size)
sends a DWSR message. The OPES processor receiving a DWSR message
terminates original dataflow by sending an Application Message End
(AME) message with a 206 (partial) status code.
The following figure illustrates a typical sequence of events. The
downward lines connecting the two dataflows illustrate the
transmission delay that allows for more OCP messages to be sent while
an agent waits for the opposing agent reaction.
OPES Callout
Processor Server
DUM> <DUM
DUM> <DWSR <-- Server is ready to stop receiving
... _____/<DUM <-- Server continues as usual
DUM>______/ <DUM
AME> ... <-- Processor stops sending original data
\_____ <DUM
\______<DUM
<DUM <-- Server continues to send adapted data
...
<AME
The mechanism described in this section has no effect on the adapted
dataflow. Receiving an Application Message End (AME) message with
206 (partial) result status code from the OPES processor does not
introduce any special requirements for the adapted dataflow
termination. However, it is not possible to terminate adapted
dataflow prematurely after the original dataflow has been prematurely
terminated (see section 8.3).
8.2. Adapted Dataflow
There are scenarios where a callout service may want to stop sending
adapted data before a complete application message has been sent.
For example, a logging-only callout service has to receive all
application messages but would preferably not send copies back to the
OPES processor.
OCP Core supports premature adapted dataflow termination via a
combination of Want Stop Sending Data (DWSS) and Stop Sending Data
(DSS) messages. A callout service that seeks to stop sending data
sends a DWSS message, soliciting an OPES processor permission to
stop. While waiting for the permission, the server continues with
its usual routine.
An OPES processor receiving a Want Stop Sending Data message responds
with a Stop Sending Data (DSS) message. The processor may then pause
to wait for the callout server to terminate the adapted dataflow or
may continue sending original data while making a copy of it. Once
the server terminates the adapted dataflow, the processor is
responsible for using original data (sent or paused after sending
DSS) instead of the adapted data.
The callout server receiving a DSS message terminates the adapted
dataflow (see the Stop Sending Data (DSS) message definition for the
exact requirements and corner cases).
The following figure illustrates a typical sequence of events,
including a possible pause in original dataflow when the OPES
processor is waiting for the adapted dataflow to end. The downward
lines connecting the two dataflows illustrate the transmission delay
that allows for more OCP messages to be sent while an agent waits for
the opposing agent reaction.
OPES Callout
Processor Server
DUM> <DUM
DUM> <DWSS <-- Server is ready to stop sending
... _____/<DUM <-- Server continues as usual,
DUM>______/ <DUM waiting for DSS
DSS> ...
\_____ <DUM
possible \______<DUM
org-dataflow <AME 206 <-- Server terminates adapted dataflow
pause _____/ upon receiving the DSS message
______/
DUM> <-- Processor resumes original dataflow
DUM> to the server and starts using
... original data without adapting it
AME>
Premature adapted dataflow preservation is not trivial, as the OPES
processor relies on the callout server to provide adapted data
(modified or not) to construct the adapted application message. If
the callout server seeks to quit its job, special care must be taken
to ensure that the OPES processor can construct the complete
application message. On a logical level, this mechanism is
equivalent to switching from one callout server to another
(non-modifying) callout server in the middle of an OCP transaction.
Other than a possible pause in the original dataflow, the mechanism
described in this section has no effect on the original dataflow.
Receiving an Application Message End (AME) message with 206 (partial)
result status code from the callout server does not introduce any
special requirements for the original dataflow termination.
8.3. Getting Out of the Loop
Some adaptation services work on application message prefixes and do
not seek to be in the adaptation loop once their work is done. For
example, an ad insertion service that did its job by modifying the
first fragment of a web "page" would not seek to receive more
original data or to perform further adaptations. The ’Getting Out of
the Loop’ optimization allows a callout server to get completely out
of the application message processing loop.
The "Getting Out of the Loop" optimization is made possible by
terminating the adapted dataflow (section 8.2) and then by
terminating the original dataflow (section 8.1). The order of
termination is very important.
If the original dataflow is terminated first, the OPES processor
would not allow the adapted dataflow to be terminated prematurely, as
the processor would not be able to reconstruct the remaining portion
of the adapted application message. The processor would not know
which suffix of the remaining original data has to follow the adapted
parts. The mapping between original and adapted octets is known only
to the callout service.
An OPES processor that received a DWSS message followed by a DWSR
message MUST NOT send an AME message with a 206 (partial) status code
before sending a DSS message. Informally, this rule means that a
callout server that wants to get out of the loop fast should send a
DWSS message immediately followed by a DWSR message; the server does
not have to wait for the OPES processor’s permission to terminate
adapted dataflow before requesting that the OPES processor terminates
original dataflow.
9. Protocol Element Type Declaration Mnemonic (PETDM)
A protocol element type is a named set of syntax and semantics rules.
This section defines a simple, formal declaration mnemonic for
protocol element types, labeled PETDM. PETDM simplicity is meant to
ease type declarations in this specification. PETDM formality is
meant to improve interoperability among implementations. Two
protocol elements are supported by PETDM: message parameter values
and messages.
All OCP Core parameter and message types are declared by using PETDM.
OCP extensions SHOULD use PETDM when declaring new types.
Atom, list, structure, and message constructs are four available base
types. Their syntax and semantics rules are defined in section 3.1.
New types can be declared in PETDM to extend base types semantics by
using the following declaration templates. The new semantics rules
are meant to be attached to each declaration using prose text.
Text in angle brackets "<>" are template placeholders, to be
substituted with actual type names or parameter name tokens. Square
brackets "[]" surround optional elements such as structure members or
message payload.
o Declaring a new atomic type:
<new-type-name>: extends atom;
o Declaring a new list with old-type-name items:
<new-type-name>: extends list of <old-type-name>;
Unless it is explicitly noted otherwise, empty lists are valid and
have the semantics of an absent parameter value.
o Declaring a new structure with members:
<new-type-name>: extends structure with {
<old-type-nameA> <old-type-nameB> [<old-type-nameC>] ...;
<member-name1>: <old-type-name1>;
<member-name2>: <old-type-name2>;
[<member-name3>: <old-type-name3>];
...
};
The new structure may have anonymous members and named members.
Neither group has to exist. Note that it is always possible for
extensions to add more members to old structures without affecting
type semantics because unrecognized members are ignored by compliant
agents.
o Declaring a new message with parameters:
<new-type-name>: extends message with {
<old-type-nameA> <old-type-nameB> [<old-type-nameC>] ...;
<parameter-name1>: <old-type-name1>;
<parameter-name2>: <old-type-name2>;
[<parameter-name3>: <old-type-name3>];
...
};
The new type name becomes the message name. Just as when a structure
is extended, the new message may have anonymous parameters and named
parameters. Neither group has to exist. Note that it is always
possible for extensions to add more parameters to old messages
without affecting type semantics because unrecognized parameters are
ignored by compliant agents.
o Extending a type with more semantics details:
<new-type-name>: extends <old-type-name>;
o Extending a structure- or message-base type:
<new-type-name>: extends <old-type-name> with {
<old-type-nameA> <old-type-nameB> [<old-type-nameC>] ...;
<member-name1>: <old-type-name1>;
<member-name2>: <old-type-name2>;
[<member-name3>: <old-type-name3>];
...
};
New anonymous members are appended to the anonymous members of the
old type, and new named members are merged with named members of the
old type.
o Extending a message-base type with payload semantics:
<new-type-name>: extends <old-type-name> with {
...
} and payload;
Any any OCP message can have payload, but only some message types
have known payload semantics. Like any parameter, payload may be
required or optional.
o Extending type semantics without renaming the type:
<old-type-name>: extends <namespace>::<old-type-name>;
The above template can be used by OCP Core extensions that seek to
change the semantics of OCP Core types without renaming them. This
technique is essential for extending OCP messages because the message
name is the same as the message type name. For example, an SMTP
profile for OCP might use the following declaration to extend an
Application Message Start (AMS) message with Am-Id, a parameter
defined in that profile:
AMS: extends Core::AMS with {
Am-Id: am-id;
};
All extended types may be used as replacements of the types they
extend. For example, a Negotiation Offer (NO) message uses a
parameter of type Features. Features (section 10.12) is a list of
feature (section 10.11) items. A Feature is a structure-based type.
An OCP extension (e.g., an HTTP application profile) may extend the
feature type and use a value of that extended type in a negotiation
offer. Recipients that are aware of the extension will recognize
added members in feature items and negotiate accordingly. Other
recipients will ignore them.
The OCP Core namespace tag is "Core". OCP extensions that declare
types MUST define their namespace tags (so that other extensions and
documentation can use them in their PETDM declarations).
9.1. Optional Parameters
Anonymous parameters are positional: The parameter’s position (i.e.,
the number of anonymous parameters to the left) is its "name". Thus,
when a structure or message has multiple optional anonymous
parameters, parameters to the right can be used only if all
parameters to the left are present. The following notation
[name1] [name2] [name3] ... [nameN]
is interpreted as
[name1 [ name2 [ name3 ... [nameN] ... ]]]
When an anonymous parameter is added to a structure or message that
has optional anonymous parameters, the new parameter has to be
optional and can only be used if all old optional parameters are in
use. Named parameters do not have these limitations, as they are not
positional, but associative; they are identified by their explicit
and unique names.
10. Message Parameter Types
This section defines parameter value types that are used for message
definitions (section 11). Before using a parameter value, an OCP
agent MUST check whether the value has the expected type (i.e.,
whether it complies with all rules from the type definition). A
single rule violation means that the parameter is invalid. See
Section 5 for rules on processing invalid input.
OCP extensions MAY define their own types. If they do, OCP
extensions MUST define types with exactly one base format and MUST
specify the type of every new protocol element they introduce.
10.1. uri
uri: extends atom;
Uri (universal resource identifier) is an atom formatted according to
URI rules in [RFC2396].
Often, a uri parameter is used as a unique (within a given scope)
identifier. Uni semantics is incomplete without the scope
specification. Many uri parameters are URLs. Unless it is noted
otherwise, URL identifiers do not imply the existence of a
serviceable resource at the location they specify. For example, an
HTTP request for an HTTP-based URI identifier may result in a 404
(Not Found) response.
10.2. uni
uni: extends atom;
Uni (unique numeric identifier) is an atom formatted as dec-number
and with a value in the [0, 2147483647] range, inclusive.
A uni parameter is used as a unique (within a given scope)
identifier. Uni semantics is incomplete without the scope
specification. Some OCP messages create identifiers (i.e., bring
them into scope). Some OCP messages destroy them (i.e, remove them
from scope). An OCP agent MUST NOT create the same uni value more
than once within the same scope. When creating a new identifier of
the same type and within the same scope as some old identifier, an
OCP agent MUST use a higher numerical value for the new identifier.
The first rule makes uni identifiers suitable for cross-referencing
logs and other artifacts. The second rule makes efficient checks of
the first rule possible.
For example, a previously used transaction identifier "xid" must not
be used for a new Transaction Start (TS) message within the same OCP
transaction, even if a prior Transaction End (TE) message was sent
for the same transaction.
An OCP agent MUST terminate the state associated with uni uniqueness
scope if all unique values have been used up.
10.3. size
size: extends atom;
Size is an atom formatted as dec-number and with a value in the [0,
2147483647] range, inclusive.
Size value is the number of octets in the associated data chunk.
OCP Core cannot handle application messages that exceed 2147483647
octets in size, that require larger sizes as a part of OCP marshaling
process, or that use sizes with granularity other than 8 bits. This
limitation can be addressed by OCP extensions, as hinted in section
15.1.
10.4. offset
offset: extends atom;
Offset is an atom formatted as dec-number and with a value in the [0,
2147483647] range, inclusive.
Offset is an octet position expressed in the number of octets
relative to the first octet of the associated dataflow. The offset
of the first data octet has a value of zero.
10.5. percent
percent: extends atom;
Percent is an atom formatted as dec-number and with a value in the
[0, 100] range, inclusive.
Percent semantics is incomplete unless its value is associated with a
boolean statement or assertion. The value of 0 indicates absolute
impossibility. The value of 100 indicates an absolute certainty. In
either case, the associated statement can be relied upon as if it
were expressed in boolean rather than probabilistic terms. Values in
the [1,99] inclusive range indicate corresponding levels of certainty
that the associated statement is true.
10.6. boolean
boolean: extends atom;
Boolean type is an atom with two valid values: true and false. A
boolean parameter expresses the truthfulness of the associated
statement.
10.7. xid
xid: extends uni;
Xid, an OCP transaction identifier, uniquely identifies an OCP
transaction within an OCP connection.
10.8. sg-id
sg-id: extends uni;
Sg-id, a service group identifier, uniquely identifies a group of
services on an OCP connection.
10.9. modp
modp: extends percent;
Modp extends the percent type to express the sender’s confidence that
application data will be modified. The boolean statement associated
with the percentage value is "data will be modified". Modification
is defined as adaptation that changes the numerical value of at least
one data octet.
10.10. result
result: extends structure with {
atom [atom];
};
The OCP processing result is expressed as a structure with two
documented members: a required Uni status code and an optional
reason. The reason member contains informative textual information
not intended for automated processing. For example:
{ 200 OK }
{ 200 "6:got it" }
{ 200 "27:27 octets in UTF-8 encoding" }
This specification defines the following status codes:
Result Status Codes
+--------+--------------+-------------------------------------------+
| code | text | semantics |
+--------+--------------+-------------------------------------------+
| 200 | OK | Overall success. This specification does |
| | | not contain any general actions for a 200 |
| | | status code recipient. |
| 206 | partial | Partial success. This status code is |
| | | documented for Application Message End |
| | | (AME) messages only. The code indicates |
| | | that the agent terminated the |
| | | corresponding dataflow prematurely (i.e., |
| | | more data would be needed to reconstruct |
| | | a complete application message). |
| | | Premature termination of one dataflow |
| | | does not introduce any special |
| | | requirements for the other dataflow |
| | | termination. See dataflow termination |
| | | optimizations (section 8) for use cases. |
| 400 | failure | An error, exception, or trouble. A |
| | | recipient of a 400 (failure) result of an |
| | | AME, TE, or CE message MUST destroy any |