| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| payload_reserved* | payload_len* |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| payload_offset* |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| payload_data* |
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
NORM_DATA Message Format
*NOTE: The "payload_reserved", "payload_len" and "payload_offset"
fields are present only for objects of type NORM_OBJECT_STREAM. The
"payload_len" and "payload_offset" fields allow senders to
arbitrarily vary the size of NORM_DATA payload segments for streams.
This allows applications to flush transmitted streams as needed to
meet unique streaming requirements. For objects of types
NORM_OBJECT_FILE and NORM_OBJECT_DATA, these fields are unnecessary
since the receiver can calculate the payload length and offset
information from the "fec_payload_id" using the algorithm described
in Section 5.1.1. The "payload_reserved" field is kept for
anticipated future NORM stream control functions. When systematic
FEC codes (e.g., "fec_id" = 129) are used, the "payload_len" and
"payload_offset" fields contain actual length and offset values for
the encapsulated application data segment for those NORM_DATA
messages containing source data symbols. In NORM_DATA messages that
contain parity information, these fields are not actual length or
offset values, but instead are values computed from FEC encoding the
"payload_len" and "payload_offset" fields of the _source_ data
symbols of the corresponding applicable coding block.
The "version", "type", "hdr_len", "sequence", and "source_id" fields
form the NORM Common Message Header as described in Section 4.1. The
value of the NORM_DATA "type" field is 2. The NORM_DATA _base_
"hdr_len" value is 4 (32-bit words) plus the size of the
"fec_payload_id" field. The "fec_payload_id" field size depends upon
the FEC encoding used for the referenced NormObject. The "fec_id"
field is used to indicate the FEC coding type. For example, when
small block, systematic codes are used, a "fec_id" value of 129 is
indicated and the size of the "fec_payload_id" is two 32-bit words.
In this case the NORM_DATA base "hdr_len" value is 6. The cumulative
size of any header extensions applied is added into the "hdr_len"
field.
The "instance_id" field contains a value generated by the sender to
uniquely identify its current instance of participation in the
NormSession. This allows receivers to detect when senders have
perhaps left and rejoined a session in progress. When a sender
(identified by its "source_id") is detected to have a new
"instance_id", the NORM receivers SHOULD drop their previous state on
the sender and begin reception anew.
The "grtt" field contains a non-linear quantized representation of
the sender’s current estimate of group round-trip time (GRTT) (this
is also referred to as R_max in [19]). This value is used to control
timing of the NACK repair process and other aspects of protocol
operation as described in this document. The algorithm for encoding
and decoding this field is described in the RMT NORM Building Block
document [4].
The "backoff" field value is used by receivers to determine the
maximum backoff timer value used in the timer-based NORM NACK
feedback suppression. This 4-bit field supports values from 0-15
which is multiplied by the sender GRTT to determine the maximum
backoff timeout. The "backoff" field informs the receiver set of the
sender’s backoff factor parameter "Ksender". Recommended values and
their use are described in the NORM receiver NACK procedure
description in Section 5.3. The "gsize" field contains a
representation of the sender’s current estimate of group size. This
4-bit field can roughly represent values from ten to 500 million
where the most significant bit value of 0 or 1 represents a mantissa
of 1 or 5, respectively and the three least significant bits
incremented by one represent a base 10 exponent (order of magnitude).
For examples, a field value of "0x0" represents 1.0e+01 (10), a value
of "0x8" represents 5.0e+01 (50), a value of "0x1" represents 1.0e+02
(100), and a value of "0xf" represents 5.0e+08. For NORM feedback
suppression purposes, the group size does not need to be represented
with a high degree of precision. The group size may even be
estimated somewhat conservatively (i.e., overestimated) to maintain
low levels of feedback traffic. A default group size estimate of
10,000 ("gsize" = 0x4) is recommended for general purpose reliable
multicast applications using the NORM protocol.
The "flags" field contains a number of different binary flags
providing information and hints regarding how the receiver should
handle the identified object. Defined flags in this field include:
+--------------------+-------+-----------------------------------------+
| Flag | Value | Purpose |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_REPAIR | 0x01 | Indicates message is a repair |
| | | transmission |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_EXPLICIT | 0x02 | Indicates a repair segment intended to |
| | | meet a specific receiver erasure, as |
| | | compared to parity segments provided by |
| | | the sender for general purpose (with |
| | | respect to an FEC coding block) erasure |
| | | filling. |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_INFO | 0x04 | Indicates availability of NORM_INFO for |
| | | object. |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_UNRELIABLE| 0x08 | Indicates that repair transmissions for |
| | | the specified object will be unavailable|
| | | (One-shot, best effort transmission). |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_FILE | 0x10 | Indicates object is "file-based" data |
| | | (hint to use disk storage for |
| | | reception). |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_STREAM | 0x20 | Indicates object is of type |
| | | NORM_OBJECT_STREAM. |
+--------------------+-------+-----------------------------------------+
|NORM_FLAG_MSG_START | 0x40 | Marks the first segment of application |
| | | messages embedded in |
| | | NORM_OBJECT_STREAMs. |
+--------------------+-------+-----------------------------------------+
NORM_FLAG_REPAIR is set when the associated message is a repair
transmission. This information can be used by receivers to help
observe a join policy where it is desired that newly joining
receivers only begin participating in the NACK process upon receipt
of new (non-repair) data content. NORM_FLAG_EXPLICIT is used to mark
repair messages sent when the data sender has exhausted its ability
to provide "fresh" (previously untransmitted) parity segments as
repair. This flag could possibly be used by intermediate systems
implementing functionality to control sub-casting of repair content
to different legs of a reliable multicast topology with disparate
repair needs. NORM_FLAG_INFO is set only when optional NORM_INFO
content is actually available for the associated object. Thus,
receivers will NACK for retransmission of NORM_INFO only when it is
available for a given object. NORM_FLAG_UNRELIABLE is set when the
sender wishes to transmit an object with only "best effort" delivery
and will not supply repair transmissions for the object. NORM
receivers SHOULD NOT execute repair requests for objects marked with
the NORM_FLAG_UNRELIABLE flag. Note that receivers may inadvertently
request repair of such objects when all segments (or info content)
for those objects are not received (i.e., a gap in the
"object_transport_id" sequence is noted). In this case, the sender
should invoke the NORM_CMD(SQUELCH) process as described in Section
4.2.3. NORM_FLAG_FILE can be set as a "hint" from the sender that
the associated object should be stored in non-volatile storage.
NORM_FLAG_STREAM is set when the identified object is of type
NORM_OBJECT_STREAM. When NORM_FLAG_STREAM is set, the
NORM_FLAG_MSG_START can be optionally used to mark the first data
segments of application-layer messages transported within the NORM
stream. This allows NORM receiver applications to "synchronize" with
NORM senders and to be able to properly interpret application layer
data when joining a NORM session already in progress. In practice,
the NORM implementation MAY set this flag for the segment transmitted
following an explicit "flush" of the stream by the application.
The "fec_id" field corresponds to the FEC Encoding Identifier
described in the FEC Building Block document [5]. The "fec_id" value
implies the format of the "fec_payload_id" field and, coupled with
FEC Object Transmission Information, the procedures to decode FEC
encoded content. Small block, systematic codes ("fec_id" = 129) are
expected to be used for most NORM purposes and the NORM_OBJECT_STREAM
requires systematic FEC codes for most efficient performance.
The "object_transport_id" field is a monotonically and incrementally
increasing value assigned by the sender to NormObjects being
transmitted. Transmissions and repair requests related to that
object use the same "object_transport_id" value. For sessions of
very long or indefinite duration, the "object_transport_id" field may
be repeated, but it is presumed that the 16-bit field size provides
an adequate enough sequence space to avoid object confusion amongst
receivers and sources (i.e., receivers SHOULD re-synchronize with a
server when receiving object sequence identifiers sufficiently out-
of-range with the current state kept for a given source). During the
course of its transmission within a NORM session, an object is
uniquely identified by the concatenation of the sender "source_id"
and the given "object_transport_id". Note that NORM_INFO messages
associated with the identified object carry the same
"object_transport_id" value.
The "fec_payload_id" identifies the attached NORM_DATA "payload"
content. The size and format of the "fec_payload_id" field depends
upon the FEC type indicated by the "fec_id" field. These formats are
given in the FEC Building Block document [5] and any subsequent
extensions of that document. As an example, the format of the
"fec_payload_id" format small block, systematic codes ("fec_id" =
129) given here:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| source_block_number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| source_block_len | encoding_symbol_id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Small Block, Systematic Code ("fec_id" = 129) "fec_payload_id" Format
The FEC payload identifier "source_block_number", "source_block_len",
and "encoding_symbol_id" fields correspond to the "Source Block
Number", "Source Block Length, and "Encoding Symbol ID" fields of the
FEC Payload ID format given by the IETF FEC Building Block document
[5]. The "source_block_number" identifies the coding block’s
relative position with a NormObject. Note that, for NormObjects of
type NORM_OBJECT_STREAM, the "source_block_number" may wrap for very
long lived sessions. The "source_block_len" indicates the number of
user data segments in the identified coding block. Given the
"source_block_len" information of how many symbols of application
data are contained in the block, the receiver can determine whether
the attached segment is data or parity content and treat it
appropriately. The "encoding_symbol_id" identifies which specific
symbol (segment) within the coding block the attached payload
conveys. Depending upon the value of the "encoding_symbol_id" and
the associated "source_block_len" parameters for the block, the
symbol (segment) referenced may be a user data or an FEC parity
segment. For systematic codes, encoding symbols numbered less than
the source_block_len contain original application data while segments
greater than or equal to source_block_len contain parity symbols
calculated for the block. The concatenation of
object_transport_id::fec_payload_id can be viewed as a unique
transport protocol data unit identifier for the attached segment with
respect to the NORM sender’s instance within a session.
Additional FEC Object Transmission Information (as described in the
FEC Building Block document [5]) is required to properly receive and
decode NORM transport objects. This information MAY be provided as
out-of-band session information. However, in some cases, it may be
useful for the sender to include this information "in band" to
facilitate receiver operation with minimal preconfiguration. For
this purpose, the NORM FEC Object Transmission Information Header
Extension (EXT_FTI) is defined. This header extension MAY be applied
to NORM_DATA and NORM_INFO messages to provide this necessary
information. The exact format of the extension depends upon the FEC
code in use, but in general it SHOULD contain any required details on
the FEC code in use (e.g., FEC Instance ID, etc.) and the byte size
of the associated NormObject (For the NORM_OBJECT_STREAM type, this
size corresponds to the stream buffer size maintained by the NORM
sender). As an example, the format of the EXT_FTI for small block
systematic codes ("fec_id" = 129) is given here:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| het = 64 | hel = 4 | object_length (msb) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| object_length (lsb) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| fec_instance_id | segment_size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| fec_max_block_len | fec_num_parity |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
FEC Object Transmission Information Header Extension (EXT_FTI) for
Small Block Systematic Codes ("fec_id" = 129)
The header extension type "het" field value for this header extension
is 64. The header extension length "hel" depends upon the format of
the FTI for FEC code type identified by the "fec_id" field. In this
example (for "fec_id" = 129), the "hel" field value is 4.
The 48-bit "object_length" field indicates the total size of the
object (in bytes) for the static object types of NORM_OBJECT_FILE and
NORM_OBJECT_DATA. This information is used by receivers to determine
storage requirements and/or allocate storage for the received object.
Receivers with insufficient storage capability may wish to forego
reliable reception (i.e., not NACK for) of the indicated object. In
the case of objects of type NORM_OBJECT_STREAM, the "object_length"
field is used by the sender to indicate the size of its stream buffer
to the receiver group. In turn, the receivers SHOULD use this
information to allocate a stream buffer for reception of
corresponding size.
The "fec_instance_id" corresponds to the "FEC Instance ID" described
in the FEC Building Block document [5]. In this case, the
"fec_instance_id" SHALL be a value corresponding to the particular
type of Small Block Systematic Code being used (e.g., Reed-Solomon
GF(2^8), Reed-Solomon GF(2^16), etc). The standardized assignment of
FEC Instance ID values is described in [5]. The "segment_size" field
indicates the sender’s current setting for maximum message payload
content (in bytes). This allows receivers to allocate appropriate
buffering resources and to determine other information in order to
properly process received data messaging.
The "fec_max_block_len" indicates the current maximum number of user
data segments per FEC coding block to be used by the sender during
the session. This allows receivers to allocate appropriate buffer
space for buffering blocks transmitted by the sender.
The "fec_num_parity" corresponds to the "maximum number of encoding
symbols that can be generated for any source block" as described in
for FEC Object Transmission Information for Small Block Systematic
Codes in the FEC Building Block document [5]. For example, Reed-
Solomon codes may be arbitrarily shortened to create different code
variations for a given block length. In the case of Reed-Solomon
(GF(2^8) and GF(2^16)) codes, this value indicates the maximum number
of parity segments available from the sender for the coding blocks.
This field MAY be interpreted differently for other systematic codes
as they are defined.
The payload portion of NORM_DATA messages includes source data or FEC
encoded application content.
The "payload_reserved", "payload_len" and "payload_offset" fields are
present ONLY for transport objects of type NORM_OBJECT_STREAM. These
fields indicate the size and relative position (within the stream) of
the application content represented by the message payload. For
senders employing systematic FEC encoding, these fields contain
_actual_ length and offset values (in bytes) for the payload of
messages which contain original data source symbols. For NORM_DATA
messages containing calculated parity content, these fields will
actually contain values computed by FEC encoding of the "payload_len"
and "payload_offset" values of the NORM_DATA data segments of the
corresponding FEC coding block. Thus, the "payload_len" and
"payload_offset" values of missing data content can be determined
upon decoding a FEC coding block. Note that these fields do NOT
contribute to the value of the NORM_DATA "hdr_len" field. These
fields are NOT present when the "flags" portion of the NORM_DATA
message indicate the transport object if of type NORM_OBJECT_FILE or
NORM_OBJECT_DATA. In this case, the length and offset information
can be calculated from the "fec_payload_id" using the methodology
described in Section 5.1.1. Note that for long-lived streams, the
"payload_offset" field can wrap.
The "payload_data" field contains the original application source or
parity content for the symbol identified by the "fec_payload_id".
The length of this field SHALL be limited to a maximum of the
sender’s NormSegmentSize bytes as given in the FTI for the object.
Note the length of this field for messages containing parity content
will always be of length NormSegmentSize. When encoding data
segments of varying sizes, the FEC encoder SHALL assume ZERO value
padding for data segments with length less than the NormSegmentSize.
It is RECOMMENDED that a sender’s NormSegmentSize generally be
constant for the duration of a given sender’s term of participation
in the session, but may possibly vary on a per-object basis. The
NormSegmentSize is expected to be configurable by the sender
application prior to session participation as needed for network
topology maximum transmission unit (MTU) considerations. For IPv6,
MTU discovery may be possibly leveraged at session startup to perform
this configuration. The "payload_data" content may be delivered
directly to the application for source symbols (when systematic FEC
encoding is used) or upon decoding of the FEC block. For
NORM_OBJECT_FILE and NORM_OBJECT_STREAM objects, the data segment
length and offset can be calculated using the algorithm described in
Section 5.1.1. For NORM_OBJECT_STREAM objects, the length and offset
is obtained from the segment’s corresponding "payload_len" and
"payload_offset" fields.
4.2.2. NORM_INFO Message
The NORM_INFO message is used to convey OPTIONAL, application-
defined, "out-of-band" context information for transmitted
NormObjects. An example NORM_INFO use for bulk file transfer is to
place MIME type information for the associated file, data, or stream
object into the NORM_INFO payload. Receivers may use the NORM_INFO
content to make a decision as whether to participate in reliable
reception of the associated object. Each NormObject can have an
independent unit of NORM_INFO associated with it. NORM_DATA messages
contain a flag to indicate the availability of NORM_INFO for a given
NormObject. NORM receivers may NACK for retransmission of NORM_INFO
when they have not received it for a given NormObject. The size of
the NORM_INFO content is limited to that of a single NormSegmentSize
for the given sender. This atomic nature allows the NORM_INFO to be
rapidly and efficiently repaired within the NORM reliable
transmission process.
When NORM_INFO content is available for a NormObject, the
NORM_FLAG_INFO flag SHALL be set in NORM_DATA messages for the
corresponding "object_transport_id" and the NORM_INFO message shall
be transmitted as the first message for the NormObject.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|version| type=1| hdr_len | sequence |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| source_id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| instance_id | grtt |backoff| gsize |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| flags | fec_id | object_transport_id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| header_extensions (if applicable) |
| ... |