Request for Comments: 4654 DoCoMo Euro-Labs
Category: Experimental M. Handley
UCL
August 2006
TCP-Friendly Multicast Congestion Control (TFMCC):
Protocol Specification
Status of This Memo
This memo defines an Experimental Protocol for the Internet
community. It does not specify an Internet standard of any kind.
Discussion and suggestions for improvement are requested.
Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
This document specifies TCP-Friendly Multicast Congestion Control
(TFMCC). TFMCC is a congestion control mechanism for multicast
transmissions in a best-effort Internet environment. It is a
single-rate congestion control scheme, where the sending rate is
adapted to the receiver experiencing the worst network conditions.
TFMCC is reasonably fair when competing for bandwidth with TCP flows
and has a relatively low variation of throughput over time, making it
suitable for applications where a relatively smooth sending rate is
of importance, such as streaming media.
Table of Contents
1. Introduction ....................................................3
1.1. Related Documents ..........................................4
1.2. Environmental Requirements and Considerations ..............4
2. Protocol Overview ...............................................5
2.1. TCP Throughput Equation ....................................6
2.2. Packet Contents ............................................7
2.2.1. Sender Packets ......................................8
2.2.2. Feedback Packets ....................................9
3. Data Sender Protocol ...........................................10
3.1. Sender Initialization .....................................10
3.2. Determining the Maximum RTT ...............................10
3.3. Adjusting the Sending Rate ................................11
3.4. Controlling Receiver Feedback .............................12
3.5. Assisting Receiver-Side RTT Measurements ..................14
3.6. Slowstart .................................................15
3.7. Scheduling of Packet Transmissions ........................15
4. Data Receiver Protocol .........................................16
4.1. Receiver Initialization ...................................17
4.2. Receiver Leave ............................................17
4.3. Measurement of the Network Conditions .....................17
4.3.1. Updating the Loss Event Rate .......................17
4.3.2. Basic Round-Trip Time Measurement ..................17
4.3.3. One-Way Delay Adjustments ..........................18
4.3.4. Receive Rate Measurements ..........................19
4.4. Setting the Desired Rate ..................................19
4.5. Feedback and Feedback Suppression .........................20
5. Calculation of the Loss Event Rate .............................22
5.1. Detection of Lost or Marked Packets .......................22
5.2. Translation from Loss History to Loss Events ..............23
5.3. Inter-Loss Event Interval .................................24
5.4. Average Loss Interval .....................................24
5.5. History Discounting .......................................25
5.6. Initializing the Loss History after the First Loss Event ..27
6. Security Considerations ........................................28
7. Acknowledgments ................................................29
8. References .....................................................29
8.1. Normative References ......................................29
8.2. Informative References ....................................29
1. Introduction
This document specifies TCP-Friendly Multicast Congestion Control
(TFMCC) [3]. TFMCC is a source-based, single-rate congestion control
scheme that builds upon the unicast TCP-Friendly Rate Control
mechanism (TFRC) [4]. TFMCC is stable and responsive under a wide
range of network conditions and scales to receiver sets on the order
of several thousand receivers. To support scalability, as much
congestion control functionality as possible is located at the
receivers. Each receiver continuously determines a desired receive
rate that is TCP-friendly for the path from the sender to this
receiver. Selected receivers then report the rate to the sender in
feedback packets.
TFMCC is a building block as defined in RFC 3048 [1]. Instead of
specifying a complete protocol, this document simply specifies a
congestion control mechanism that could be used in a transport
protocol such as RTP [11], in an application incorporating end-to-end
congestion control at the application level. This document does not
discuss packet formats, reliability, or implementation-related
issues.
TFMCC is designed to be reasonably fair when competing for bandwidth
with TCP flows. A multicast flow is "reasonably fair" if its sending
rate is generally within a factor of two of the sending rate of a TCP
flow from the sender to the slowest receiver of the multicast group
under the same network conditions.
In general, TFMCC has a low variation of throughput, which makes it
suitable for applications where a relatively smooth sending rate is
of importance, such as streaming media. The penalty of having smooth
throughput while competing fairly for bandwidth is a reduced
responsiveness to changes in available bandwidth. Thus TFMCC should
be used when the application has a requirement for smooth throughput,
in particular, avoiding halving of the sending rate in response to a
single packet drop. For applications that simply need to multicast
as much data as possible in as short a time as possible, PGMCC [10]
may be more suitable.
This memo contains part of the definitions necessary to fully specify
a Reliable Multicast Transport protocol in accordance with RFC 2357.
As per RFC 2357, the use of any reliable multicast protocol in the
Internet requires an adequate congestion control scheme. This
document specifies an experimental congestion control scheme. While
waiting for initial deployment and experience to show this scheme to
be effective and scalable, the IETF publishes this scheme in the
"Experimental" category.
It is the intent of the Reliable Multicast Transport (RMT) Working
Group to re-submit the specification as an IETF Proposed Standard as
soon as the scheme is deemed adequate.
1.1. Related Documents
As described in RFC 3048 [1], TFMCC is a building block that is
intended to be used, in conjunction with other building blocks, to
help specify a protocol instantiation. It follows the general
guidelines provided in RFC 3269 [2]. In particular, TFMCC is a
suitable congestion control building block for NACK-Oriented Reliable
Multicast (NORM) [5].
1.2. Environmental Requirements and Considerations
TFMCC is intended to be a congestion control scheme that can be used
in a complete protocol instantiation that delivers objects and
streams (both reliable content delivery and streaming of multimedia
information).
TFMCC is most applicable for sessions that deliver a substantial
amount of data (i.e., in length from hundreds of kilobytes to many
gigabytes) and whose duration is on the order of tens of seconds or
more.
TFMCC is intended for multicast delivery. There are currently two
models of multicast delivery: the Any-Source Multicast (ASM) model as
defined in [6] and the Source-Specific Multicast (SSM) model as
defined in [7]. TFMCC works with both multicast models, but in a
slightly different way. When ASM is used, feedback from the
receivers is multicast to the sender, as well as to all other
receivers. Feedback can be either multicast on the same group
address used for sending data or on a separate multicast feedback
group address. For SSM, the receivers must unicast the feedback
directly to the sender. Hence, feedback from a receiver will not be
received by other receivers.
TFMCC inherently works with all types of networks that allow bi-
directional communication, including LANs, WANs, Intranets, the
Internet, asymmetric networks, wireless networks, and satellite
networks. However, in some network environments varying the sending
rate to the receivers may not be advantageous (e.g., for a satellite
or wireless network, there may be no mechanism for receivers to
effectively reduce their reception rate since there may be a fixed
transmission rate allocated to the session).
The difference in responsiveness of TFMCC and TCP may result in
significant throughput differences in case of a very low bitrate.
TFMCC requires an estimate of the loss event rate to calculate a fair
sending rate. This estimate may be inaccurate in case TFMCC receives
only very few packets per RTT. TFMCC should not be used together
with TCP if the capacity of the bottleneck link is less than 30KBit/s
(e.g., a very slow modem connection). TFMCC may also achieve a rate
that is very different from the average TCP rate in case buffer space
at the bottleneck is severely underprovisioned. In particular, TFMCC
is less susceptible to small buffer sizes since TFMCC spaces out
packets in time, whereas TCP sends them back to back. Thus TCP is
much more likely to see a packet loss if buffer space is scarce.
TFMCC is designed for applications that use a fixed packet size and
vary their sending rate in packets per second in response to
congestion. Some applications (e.g., those using audio) require a
fixed interval of time between packets and vary their packet size
instead of their packet rate in response to congestion. The
congestion control mechanism in this document cannot be used by those
applications.
2. Protocol Overview
TFMCC extends the basic mechanisms of TFRC into the multicast domain.
In order to compete fairly with TCP, TFMCC receivers individually
measure the prevalent network conditions and calculate a rate that is
TCP-friendly on the path from the sender to themselves. The rate is
determined using an equation for TCP throughput, which roughly
describes TCP’s sending rate as a function of the loss event rate,
round-trip time (RTT), and packet size. We define a loss event as
one or more lost or marked packets from the packets received during
one RTT, where a marked packet refers to a congestion indication from
Explicit Congestion Notification (ECN) [9]. The sending rate of the
multicast transmission is adapted to the receiver experiencing the
worst network conditions.
Basically, TFMCC’s congestion control mechanism works as follows:
o Each receiver measures the loss event rate and its RTT to the
sender.
o Each receiver then uses this information, together with an equation
for TCP throughput, to derive a TCP-friendly sending rate.
o Through a distributed feedback suppression mechanism, only a subset
of the receivers are allowed to give feedback to prevent a feedback
implosion at the sender. The feedback mechanism ensures that
receivers reporting a low desired transmission rate have a high
probability of sending feedback.
o Receivers whose feedback is not suppressed report the calculated
transmission rate back to the sender in so-called receiver reports.
The receiver reports serve two purposes: they inform the sender
about the appropriate transmit rate, and they allow the receivers
to measure their RTT.
o The sender selects the receiver that reports the lowest rate as
current limiting receiver (CLR). Whenever feedback with an even
lower rate reaches the sender, the corresponding receiver becomes
CLR and the sending rate is reduced to match that receiver’s
calculated rate. The sending rate increases when the CLR reports a
calculated rate higher than the current sending rate.
The dynamics of TFMCC are sensitive to how the measurements are
performed and applied and to what feedback suppression mechanism is
chosen. We recommend specific mechanisms below to perform and apply
these measurements. Other mechanisms are possible, but it is
important to understand how the interactions between mechanisms
affect the dynamics of TFMCC.
2.1. TCP Throughput Equation
Any realistic equation giving TCP throughput as a function of loss
event rate and RTT should be suitable for use in TFMCC. However, we
note that the TCP throughput equation used must reflect TCP’s
retransmit timeout behavior, as this dominates TCP throughput at
higher loss rates. We also note that the assumptions implicit in the
throughput equation about the loss event rate parameter have to be a
reasonable match to how the loss rate or loss event rate is actually
measured. While this match is not perfect for the throughput
equation and loss rate measurement mechanisms given below, in
practice the assumptions turn out to be close enough.
The throughput equation we currently recommend for TFMCC is a
slightly simplified version of the throughput equation for Reno TCP
from [8]:
8 s
X = --------------------------------------------------------- (1)
R * (sqrt(2*p/3) + (12*sqrt(3*p/8) * p * (1+32*p^2)))
where
X is the transmit rate in bits/second.
s is the packet size in bytes.
R is the round-trip time in seconds.
p is the loss event rate, between 0.0 and 1.0, of the number of
loss events as a fraction of the number of packets transmitted.
In the future, different TCP equations may be substituted for this
equation. The requirement is that the throughput equation be a
reasonable approximation of the sending rate of TCP for conformant
TCP congestion control.
The parameters s (packet size), p (loss event rate), and R (RTT) need
to be measured or calculated by a TFMCC implementation. The
measurement of R is specified in Section 4.3.2, and the measurement
of p is specified in Section 5. The parameter s (packet size) is
normally known to an application. This may not be so in two cases:
o The packet size naturally varies depending on the data. In this
case, although the packet size varies, that variation is not
coupled to the transmit rate. It should normally be safe to use an
estimate of the mean packet size for s.
o The application needs to change the packet size rather than the
number of packets per second to perform congestion control. This
would normally be the case with packet audio applications where a
fixed interval of time needs to be represented by each packet.
Such applications need to have a different way of measuring
parameters.
Currently, TFMCC cannot be used for the second class of applications.
2.2. Packet Contents
Before specifying the sender and receiver functionality, we describe
the congestion control information contained in packets sent by the
sender and feedback packets from the receivers. Information from the
sender can either be sent in separate congestion control messages or
piggybacked onto data packets. If separate congestion control
messages are sent at time intervals larger than the time interval
between data packets (e.g., once per feedback round), it is necessary
to be able to include timestamp information destined for more than
one receiver to allow a sufficient number of receivers to measure
their RTT.
As TFMCC will be used along with a transport protocol, we do not
specify packet formats, since these depend on the details of the
transport protocol used. The recommended representation of the
header fields is given below. Alternatively, if the computational
overhead of a floating point representation is prohibitive, fixed
point arithmetic can be used at the expense of larger packet headers.
Sender and receivers of a specific TFMCC instance need to agree on a
common encoding for the header fields.
2.2.1. Sender Packets
Each packet sent by the data sender contains the following
information:
o A sequence number i. This number is incremented by one for each
data packet transmitted. The field must be sufficiently large that
it does not wrap, causing two different packets with the same
sequence number to be in the receiver’s recent packet history at
the same time. In most cases, the sequence number will be supplied
by the transport protocol used along with TFMCC.
o A suppression rate X_supp in bits/s. Only receivers with a
calculated rate lower than the suppression rate are eligible to
give feedback, unless their RTT is higher than the maximum RTT
described below, in which case they are also eligible to give
feedback. The suppression rate should be represented as a 12-bit
floating point value with 5 bits for the unsigned exponent and 7
bits for the unsigned mantissa (to represent rates from 100 bit/s
to 400 Gbit/s with an error of less than 1%).
o A timestamp ts_i indicating when the packet is sent. The
resolution of the timestamp should typically be milliseconds, and
the timestamp should be an unsigned integer value no less than 16
bits wide.
o A receiver ID r and a copy of the timestamp tr_r’ = tr_r of that
receiver’s last report, which allows the receiver to measure its
RTT. If there is a delay ts_d between receiving the report from
receiver r and sending the data packet, then tr_r’ = tr_r + ts_d is
included in the packet instead. The receiver ID is described in
the next section. The resolution of the timestamp echo should be
milliseconds, and the timestamp should be an unsigned integer value
no less than 16 bits wide. If separate congestion control messages
are used instead of piggybacked ones, the packet needs to contain a
list of receiver IDs with corresponding timestamps to allow a
sufficient number of receivers to simultaneously measure their RTT.
For the default values used for the feedback process, this
corresponds to a list size on the order of 10 to 20 entries.
o A flag is_CLR indicating whether the receiver with ID r is the CLR.
o A feedback round counter fb_nr. This counter is incremented by the
sender at the beginning of a new feedback round to notify the
receivers that all feedback for older rounds should be suppressed.
The feedback round counter should be at least 4 bits wide.
o A maximum RTT value R_max, representing the maximum of the RTTs of
all receivers. The RTT should be measured in milliseconds. An
8-bit floating point value with 4 bits for the unsigned exponent
and 4 bits for the unsigned mantissa (to represent RTTs from 1
millisecond to 64 seconds with an error of ca. 6%) should be used
for the representation.
2.2.2. Feedback Packets
Each feedback packet sent by a data receiver contains the following
information:
o A unique receiver ID r. In most cases, the receiver ID will be
supplied by the transport protocol, but it may simply be the IP
address of the receiver.
o A flag have_RTT indicating whether the receiver has made at least
one RTT measurement since it joined the session.
o A flag have_loss indicating whether the receiver experienced at
least one loss event since it joined the session.
o A flag receiver_leave indicating that the receiver will leave the
session (and should therefore not be CLR).
o A timestamp tr_r indicating when the feedback packet is sent. The
representation of the timestamp should be the same as that of the
timestamp echo in the data packets.
o An echo ts_i’ of the timestamp of the last data packet received.
If the last packet received at the receiver has sequence number i,
then ts_i’ = ts_i is included in the feedback. If there is a delay
tr_d between receiving that last data packet and sending feedback,
then ts_i’ = ts_i + tr_d is included in the feedback instead. The
representation of the timestamp echo should be the same as that of
the timestamp in the data packets.
o A feedback round echo fb_nr, reflecting the highest feedback round
counter value received so far. The representation of the feedback
round echo should be the same as the one used for the feedback
round counter in the data packets.
o The desired sending rate X_r. This is the rate calculated by the
receiver to be TCP-friendly on the path from the sender to this
receiver. The representation of the desired sending rate should be
the same as that of the suppression rate in the data packets.
3. Data Sender Protocol
The data sender multicasts a stream of data packets to the data
receivers at a controlled rate. Whenever feedback is received, the
sender checks if it is necessary to switch CLRs and to readjust the
sending rate.
The main tasks that have to be provided by a TFMCC sender are:
o adjusting the sending rate,
o controlling receiver feedback, and
o assisting receiver-side RTT measurements.
3.1. Sender Initialization
At initialization of the sender, the maximum RTT is set to a value
that should be larger than the highest RTT to any of the receivers.
It should not be smaller than 500 milliseconds for operation in the
public Internet. The sending rate X is initialized to 1 packet per
maximum RTT.
3.2. Determining the Maximum RTT
For each feedback packet that arrives at the sender, the sender
computes the instantaneous RTT to the receiver as
R_r = ts_now - ts_i’
where ts_now is the time the feedback packet arrived. Receivers will
have adjusted ts_i’ for the time interval between receiving the last
data packet and sending the corresponding report so that this
interval will not be included in R_r. If the actual RTT is smaller
than the resolution of the timestamps and ts_now equals ts_i’, then
R_r is set to the smallest positive RTT value larger than 0 (i.e., 1
millisecond in our case). If the instantaneous RTT is larger than
the current maximum RTT, the maximum RTT is increased to that value:
R_max = R_r
Otherwise, if no feedback with a higher instantaneous RTT than the
maximum RTT is received during a feedback round (see Section 3.4),
the maximum RTT is reduced to
R_max = MAX(R_max * 0.9, R_peak)
where R_peak is the peak receiver RTT measured during the feedback
round.
The maximum RTT is mainly used for feedback suppression among
receivers with heterogeneous RTTs. Feedback suppression is closely
coupled to the sending of data packets, and for this reason, the
maximum RTT must not decrease below the maximum time interval between
consecutive data packets:
R_max = max(R_max, 8s/X + ts_gran)
where ts_gran is the granularity of the sender’s system clock (see
Section 3.7).
3.3. Adjusting the Sending Rate
When a feedback packet from receiver r arrives at the sender, the
sender has to check whether it is necessary to adjust the
transmission rate and to switch to a new CLR.
How the rate is adjusted depends on the desired rate X_r of the
receiver report. We distinguish four cases:
1. If no CLR is present, receiver r becomes the current limiting
receiver. The sending rate X is directly set to X_r, so long as
this would result in a rate increase of less than 8s/R_max bits/s
(i.e., 1 packet per R_max). Otherwise X is gradually increased