uses this RTT_rcvrNew value as the receiver’s current RTT (RTT_rcvr).
For non-CLR and non-PLR receivers, the sender can use the "cc_rtt"
value provided in the NORM-CC Feedback header extension as the
receiver’s previous RTT measurement (RTT_rcvrPrev) to smooth
according to:
RTT_rcvr = 0.5 * RTT_rcvrPrev + 0.5 * RTT_rcvrNew
For CLR receivers where feedback is received more regularly, the
sender SHOULD maintain a more smoothed RTT estimate upon new feedback
from the CLR where:
RTT_clr = 0.9 * RTT_clr + 0.1 * RTT_clrNew
"RTT_clrNew" is the new RTT calculated from the timestamp in the
feedback message received from the CLR. The RTT_clr is initialized
to RTT_clrNew on the first feedback message received. Note that the
same procedure is observed by the sender for PLR receivers and that
if a PLR is "promoted" to CLR status, the smoothed estimate can be
continued.
There are some additional periods besides steady-state operation that
need to be considered in NORM-CC operation. These periods are:
1) during session startup,
2) when no feedback is received from the CLR, and
3) when the sender has a break in data transmission.
During session startup, the congestion control operation SHALL
observe a "slow start" procedure to quickly approach its fair
bandwidth share. An initial sender startup rate is assumed where:
Rinitial = MIN(NormSegmentSize / GRTT, NormSegmentSize) bytes/second.
The rate is increased only when feedback is received from the
receiver set. The "slow start" phase proceeds until any receiver
provides feedback indicating that loss has occurred. Rate increase
during slow start is applied as:
Rnew = Rrecv_min
where "Rrecv_min" is the minimum reported receiver rate in the
"cc_rate" field of congestion control feedback messages received from
the group. Note that during "slow start", receivers use two times
their measured rate from the sender in the "cc_rate" field of their
feedback. Rate increase adjustment is limited to once per GRTT
during slow start.
If the CLR or any receiver intends to leave the group, it will set
the NORM_FLAG_CC_LEAVE in its congestion control feedback message as
an indication that the sender should not select it as the CLR. When
the CLR changes to a lower rate receiver, the sender should
immediately adjust to the new lower rate. The sender is limited to
increasing its rate at one additional packet per RTT towards any new,
higher CLR rate.
The sender should also track the "age" of the feedback it has
received from the CLR by comparing its current "cc_sequence" value
(Seq_sender) to the last "cc_sequence" value received from the CLR
(Seq_clr). As the "age" of the CLR feedback increases with no new
feedback, the sender SHALL begin reducing its rate once per RTT_clr
as a congestion avoidance measure.
The following algorithm is used to determine the decrease in sender
rate (Rsender bytes/sec) as the CLR feedback, unexpectedly,
excessively ages:
Age = Seq_sender - Seq_clr;
if (Age > 4) Rsender = Rsender * 0.5;
This rate reduction is limited to the lower bound on NORM
transmission rate. After NORM_ROBUST_FACTOR consecutive NORM_CMD(CC)
rounds without any feedback from the CLR, the sender SHOULD assume
the CLR has left the group and pick the receiver with the next lowest
rate as the new CLR. Note this assumes that the sender does not have
explicit knowledge that the CLR intentionally left the group. If no
receiver feedback is received, the sender MAY wish to withhold
further transmissions of NORM_DATA segments and maintain NORM_CMD(CC)
transmissions only until feedback is detected. After such a CLR
timeout, the sender will be transmitting with a minimal rate and
should return to slow start as described here for a break in data
transmission.
When the sender has a break in its data transmission, it can continue
to probe the group with NORM_CMD(CC) messages to maintain RTT
collection from the group. This will enable the sender to quickly
determine an appropriate CLR upon data transmission restart.
However, the sender should exponentially reduce its target rate to be
used for transmission restart as time since the break elapses. The
target rate SHOULD be recalculated once per RTT_clr as:
Rsender = Rsender * 0.5;
If the minimum NORM rate is reached, the sender should set the
NORM_FLAG_START flag in its NORM_CMD(CC) messages upon restart and
the group should observer "slow start" congestion control procedures
until any receiver experiences a new loss event.
5.5.3. NORM Positive Acknowledgment Procedure
NORM provides options for the source application to request positive
acknowledgment (ACK) of NORM_CMD(FLUSH) and NORM_CMD(ACK_REQ)
messages from members of the group. There are some specific
acknowledgment requests defined for the NORM protocol and a range of
acknowledgment request types that are left to be defined by the
application. One predefined acknowledgment type is the
NORM_ACK_FLUSH type. This acknowledgment is used to determine if
receivers have achieved completion of reliable reception up through a
specific logical transmission point with respect to the sender’s
sequence of transmission. The NORM_ACK_FLUSH acknowledgment may be
used to assist in application flow control when the sender has
information on a portion of the receiver set. Another predefined
acknowledgment type is NORM_ACK(CC), which is used to explicitly
provide congestion control feedback in response to NORM_CMD(CC)
messages transmitted by the sender for NORM-CC operation. Note the
NORM_ACK(CC) response does NOT follow the positive acknowledgment
procedure described here. The NORM_CMD(ACK_REQ) and NORM_ACK
messages contain an "ack_type" field to identify the type of
acknowledgment requested and provided. A range of "ack_type" values
is provided for application-defined use. While the application is
responsible for initiating the acknowledgment request and interprets
application-defined "ack_type" values, the acknowledgment procedure
SHOULD be conducted within the protocol implementation to take
advantage of timing and transmission scheduling information available
to the NORM transport.
The NORM positive acknowledgment procedure uses polling by the sender
to query the receiver group for response. Note this polling
procedure is not intended to scale to very large receiver groups, but
could be used in large group setting to query a critical subset of
the group. Either the NORM_CMD(ACK_REQ), or when applicable, the
NORM_CMD(FLUSH) message is used for polling and contains a list of
NormNodeIds for receivers that should respond to the command. The
list of receivers providing acknowledgment is determined by the
source application with "a priori" knowledge of participating nodes
or via some other application-level mechanism.
The ACK process is initiated by the sender that generates
NORM_CMD(FLUSH) or NORM_CMD(ACK_REQ) messages in periodic "rounds".
For NORM_ACK_FLUSH requests, the NORM_CMD(FLUSH) contain a
"object_transport_id" and "fec_payload_id" denoting the watermark
transmission point for which acknowledgment is requested. This
watermark transmission point is "echoed" in the corresponding fields
of the NORM_ACK(FLUSH) message sent by the receiver in response.
NORM_CMD(ACK_REQ) messages contain an "ack_id" field which is
similarly "echoed" in response so that the sender may match the
response to the appropriate request.
In response to the NORM_CMD(ACK_REQ), the listed receivers randomly
spread NORM_ACK messages uniformly in time over a window of (1*GRTT).
These NORM_ACK messages are typically unicast to the sender. (Note
that NORM_ACK(CC) messages SHALL be multicast or unicast in the same
manner as NORM_NACK messages).
The ACK process is self-limiting and avoids ACK implosion in that:
1) Only a single NORM_CMD(ACK_REQ) message is generated once per
(2*GRTT), and,
2) The size of the "acking_node_list" of NormNodeIds from which
acknowledgment is requested is limited to a maximum of the sender
NormSegmentSize setting per round of the positive acknowledgment
process.
Because the size of the included list is limited to the sender’s
NormSegmentSize setting, multiple NORM_CMD(ACK_REQ) rounds may be
required to achieve responses from all receivers specified. The
content of the attached NormNodeId list will be dynamically updated
as this process progresses and NORM_ACK responses are received from
the specified receiver set. As the sender receives valid responses
(i.e., matching watermark point or "ack_id") from receivers, it SHALL
eliminate those receivers from the subsequent NORM_CMD(ACK_REQ)
message "acking_node_list" and add in any pending receiver
NormNodeIds while keeping within the NormSegmentSize limitation of
the list size. Each receiver is queried a maximum number of times
(NORM_ROBUST_FACTOR, by default). Receivers not responding within
this number of repeated requests are removed from the payload list to
make room for other potential receivers pending acknowledgment. The
transmission of the NORM_CMD(ACK_REQ) is repeated until no further
responses are required or until the repeat threshold is exceeded for
all pending receivers. The transmission of NORM_CMD(ACK_REQ) or
NORM_CMD(FLUSH) messages to conduct the positive acknowledgment
process is multiplexed with ongoing sender data transmissions.
However, the NORM_CMD(FLUSH) positive acknowledgment process may be
interrupted in response to negative acknowledgment repair requests
(NACKs) received from receivers during the acknowledgment period.
The NORM_CMD(FLUSH) positive acknowledgment process is restarted for
receivers pending acknowledgment once any the repairs have been
transmitted.
In the case of NORM_CMD(FLUSH) commands with an attached
"acking_node_list", receivers will not ACK until they have received
complete transmission of all data up to and including the given
watermark transmission point. All receivers SHALL interpret the
watermark point provided in the request NACK for repairs if needed as
for NORM_CMD(FLUSH) commands with no attached "acking_node_list".
5.5.4. Group Size Estimate
NORM sender messages contain a "gsize" field that is a representation
of the group size and is used in scaling random backoff timer ranges.
The use of the group size estimate within the NORM protocol does not
require a precise estimation and works reasonably well if the
estimate is within an order of magnitude of the actual group size.
By default, the NORM sender group size estimate may be
administratively configured. Also, given the expected scalability of
the NORM protocol for general use, a default value of 10,000 is
recommended for use as the group size estimate.
It is possible that group size may be algorithmically approximated
from the volume of congestion control feedback messages which follow
the exponentially weighted random backoff. However, the
specification of such an algorithm is currently beyond the scope of
this document.
6. Security Considerations
The same security considerations that apply to the NORM, and FEC
Building Blocks also apply to the NORM protocol. In addition to
vulnerabilities that any IP and IP multicast protocol implementation
may be generally subject to, the NACK-based feedback of NORM may be
exploited by replay attacks which force the NORM sender to
unnecessarily transmit repair information. This MAY be addressed by
network layer IP security implementations that guard against this
potential security exploitation. It is RECOMMENDED that such IP
security mechanisms be used when available. Another possible
approach is for NORM senders to use the "sequence" field from the
NORM Common Message Header to detect replay attacks. This can be
accomplished if the NORM packets are cryptographically protected and
the sender is willing to maintain state on receivers which are
NACKing. A cache of receiver state may provide some protection
against replay attacks. Note that the "sequence" field of NORM
messages should be incremented with independent values for different
destinations (e.g., group-addressed versus unicast-addressed messages
versus "receiver" messages). Thus, the congestion control loss
estimation function of the "sequence" field can be preserved for
sender messages when receiver messages are unicast to the sender.
The NORM protocol is compatible with the use of the IP security
(IPsec) architecture described in [22]. It is important to note that
while NORM does leverage FEC-based repair for scalability, this does
not alone guarantee integrity of received data. Application-level
integrity-checking of data content is highly RECOMMENDED.
7. IANA Considerations
No information in this specification is currently subject to IANA
registration. However, several Header Extensions are defined within
this document. If/when additional Header Extensions are developed,
the first RFC MUST establish an IANA registry for them, with a
"Specification Required" policy [6] and all Header Extensions,
including those in the present document, MUST be registered
thereafter. Additionally, building blocks components used by NORM
may introduce additional IANA considerations. In particular, the FEC
Building Block used by NORM does require IANA registration of the FEC
codecs used. The registration instructions for FEC codecs are
provided in [5].
8. Suggested Use
The present NORM protocol is seen as useful tool for the reliable
data transfer over generic IP multicast services. It is not the
intention of the authors to suggest it is suitable for supporting
all envisioned multicast reliability requirements. NORM provides a
simple and flexible framework for multicast applications with a
degree of concern for network traffic implosion and protocol overhead
efficiency. NORM-like protocols have been successfully demonstrated
within the MBone for bulk data dissemination applications, including
weather satellite compressed imagery updates servicing a large group
of receivers and a generic web content reliable "push" application.
In addition, this framework approach has some design features making
it attractive for bulk transfer in asymmetric and wireless
internetwork applications. NORM is capable of successfully operating
independent of network structure and in environments with high packet
loss, delay, and misordering. Hybrid proactive/reactive FEC-based
repairing improve protocol performance in some multicast scenarios.
A sender-only repair approach often makes additional engineering
sense in asymmetric networks. NORM’s unicast feedback capability may
be suitable for use in asymmetric networks or in networks where only
unidirectional multicast routing/delivery service exists. Asymmetric
architectures supporting multicast delivery are likely to make up an
important portion of the future Internet structure (e.g.,
DBS/cable/PSTN hybrids) and efficient, reliable bulk data transfer
will be an important capability for servicing large groups of
subscribed receivers.
9. Acknowledgments (and these are not Negative)
The authors would like to thank Rick Jones, Vincent Roca, Rod Walsh,
Toni Paila, Michael Luby, and Joerg Widmer for their valuable input
and comments on this document. The authors would also like to thank
the RMT working group chairs, Roger Kermode and Lorenzo Vicisano, for
their support in development of this specification, and Sally Floyd
for her early input into this document.
10. References
10.1. Normative References
[1] Kermode, R. and L. Vicisano, "Author Guidelines for Reliable
Multicast Transport (RMT) Building Blocks and Protocol
Instantiation documents", RFC 3269, April 2002.
[2] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[3] Deering, S., "Host Extensions for IP Multicasting", STD 5, RFC
1112, August 1989.
[4] Adamson, B., Bormann, C., Handley, M., and J. Macker,
"Negative-Acknowledgment (NACK)-Oriented Reliable Multicast
(NORM) Building Blocks", RFC 3941, November 2004.
[5] Luby, M., Vicisano, L., Gemmell, J., Rizzo, L., Handley, M., and
J. Crowcroft, "Forward Error Correction (FEC) Building Block",
RFC 3452, December 2002.
[6] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
Considerations Section in RFCs", BCP 26, RFC 2434, October 1998.
10.2. Informative References
[7] Handley, M. and V. Jacobson, "SDP: Session Description
Protocol", RFC 2327, April 1998.
[8] Handley, M., Perkins, C., and E. Whelan, "Session Announcement
Protocol", RFC 2974, October 2000.
[9] S. Pingali, D. Towsley, J. Kurose, "A Comparison of Sender-
Initiated and Receiver-Initiated Reliable Multicast Protocols",
In Proc. INFOCOM, San Francisco CA, October 1993.
[10] Luby, M., Vicisano, L., Gemmell, J., Rizzo, L., Handley, M., and
J. Crowcroft, "The Use of Forward Error Correction (FEC) in
Reliable Multicast", RFC 3453, December 2002.
[11] Macker, J. and B. Adamson, "The Multicast Dissemination Protocol
(MDP) Toolkit", Proc. IEEE MILCOM 99, October 1999.
[12] Nonnenmacher, J. and E. Biersack, "Optimal Multicast Feedback",
Proc. IEEE INFOCOMM, p. 964, March/April 1998.
[13] J. Macker, B. Adamson, "Quantitative Prediction of Nack Oriented
Reliable Multicast (NORM) Feedback", Proc. IEEE MILCOM 2002,
October 2002.
[14] H.W. Holbrook, "A Channel Model for Multicast", Ph.D.
Dissertation, Stanford University, Department of Computer
Science, Stanford, California, August 2001.
[15] D. Gossink, J. Macker, "Reliable Multicast and Integrated Parity
Retransmission with Channel Estimation", IEEE GLOBECOMM 98’,
September 1998.
[16] Whetten, B., Vicisano, L., Kermode, R., Handley, M., Floyd, S.,
and M. Luby, "Reliable Multicast Transport Building Blocks for
One-to-Many Bulk-Data Transfer", RFC 3048, January 2001.
[17] Mankin, A., Romanow, A., Bradner, S., and V. Paxson, "IETF
Criteria for Evaluating Reliable Multicast Transport and
Application Protocols", RFC 2357, June 1998.
[18] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson,
"RTP: A Transport Protocol for Real-Time Applications", STD 64,
RFC 3550, July 2003.
[19] J. Widmer and M. Handley, "Extending Equation-Based Congestion
Control to Multicast Applications", Proc ACM SIGCOMM 2001, San
Diego, August 2001.
[20] L. Rizzo, "pgmcc: A TCP-Friendly Single-Rate Multicast
Congestion Control Scheme", Proc ACM SIGCOMM 2000, Stockholm,
August 2000.
[21] J. Padhye, V. Firoiu, D. Towsley, and J. Kurose, "Modeling TCP
Throughput: A Simple Model and its Empirical Validation", Proc
ACM SIGCOMM 1998.
[22] Kent, S. and R. Atkinson, "Security Architecture for the
Internet Protocol", RFC 2401, November 1998.
11. Authors’ Addresses
Brian Adamson
Naval Research Laboratory
Washington, DC, USA, 20375
EMail: adamson@itd.nrl.navy.mil
Carsten Bormann
Universitaet Bremen TZI
Postfach 330440
D-28334 Bremen, Germany
EMail: cabo@tzi.org
Mark Handley
Department of Computer Science
University College London
Gower Street
London
WC1E 6BT
UK
EMail: M.Handley@cs.ucl.ac.uk
Joe Macker
Naval Research Laboratory
Washington, DC, USA, 20375
EMail: macker@itd.nrl.navy.mil
Full Copyright Statement
Copyright (C) The Internet Society (2004).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the IETF’s procedures with respect to rights in IETF Documents can
be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.