RTT), we use the fifth column of Table 2, which shows that a sending
rate of 0.066 ppr corresponds to a packet drop rate of roughly 30%.
[For a TCP connection using fine-grained timestamps, as shown in the
fourth column of Table 2, this sending rate corresponds to a packet
drop rate of roughly 45%.] Thus, for a VoIP connection with an RTT
of 0.01 sec and a minimum sending rate of 64 kbps, the VoIP
connection would be required to terminate or suspend when the
persistent packet drop rate exceeded 30%.
5.3. Open Issues
This document does not attempt to specify a complete protocol. For
example, this document does not specify the definition of a
persistent packet drop rate. The assumption would be that a
"persistent packet drop rate" would refer to the packet drop rate
over a significant number of round-trip times, e.g., at least five
seconds. Another possibility would be that the time interval for
measuring the persistent drop rate is a function of the lifetime of
the connection, with longer-lived connections using longer time
intervals for measuring the persistent drop rate.
The time period for detecting persistent congestion also affects the
potential synchronization of VoIP sessions all terminating or
suspending at the same time in response to shared congestion. If
flows use some randomization in setting the time interval for
detecting persistent congestion, or use a time interval that is a
function of the connection lifetime, this could help to prevent all
VoIP flows from terminating at the same time.
Another design issue for a complete protocol concerns whether a flow
terminates when the packet drop rate is too high, or only suspends
temporarily. For a flow that suspends temporarily, there is an issue
of how long it should wait before resuming transmission. At the very
least, the sender should wait long enough so that the flow’s overall
sending rate doesn’t exceed the allowed sending rate for that packet
drop rate.
The recommendation of this document is that VoIP flows with minimum
sending rates should have corresponding configured packet drop rates,
such that the flow terminates or suspends when the persistent packet
drop rate of the flow exceeds the configured rate. If the persistent
packet drop rate increases over time, flows with higher minimum
sending rates would have to suspend sending before flows with lower
minimum sending rates. If VoIP flows terminate when the persistent
packet drop rate is too high, this could lead to scenarios where VoIP
flows with lower minimum sending rates essentially receive all of the
link bandwidth, while the VoIP flows with higher minimum sending
rates are required to terminate. However, if VoIP flows suspend
sending for a time when the persistent packet drop rate is too high,
instead of terminating entirely, then the bandwidth could end up
being shared reasonably fairly between VoIP flows with different
minimum sending rates.
5.4. A Simple Heuristic
One simple heuristic for estimating congestion would be to use the
RTCP reported loss rate as an indicator. For example, if the RTCP-
reported lost rate is greater than 30%, or N back-to-back RTCP
reports are missing, the application could assume that the network is
too congested, and terminate or suspend sending.
6. Constraints on VoIP Systems
Ultimately, attempting to run VoIP on congested links, even with
adaptive rate codecs and minimum packet rates, is likely to run into
hard constraints due to the nature of real time traffic in heavily
congested scenarios. VoIP systems exhibit a limited ability to scale
their packet rate. If the number of packets decreases, the amount of
audio per packet is greater and error concealment at the receiver
becomes harder. Any error longer than phoneme length, which is
typically 40 to 100 ms depending on the phoneme and speaker, is
unrecoverable. Ideally, applications want sub 30ms packets and this
is what most voice codecs provide. In addition, voice media streams
exhibit greater loss sensitivity at lower data rates. Lower-data
rate codecs maintain more end-to-end state and as a result are
generally more sensitive to loss.
We note that very-low-bit-rate codecs have proved useful, although
with some performance degradation, in very low bandwidth, high noise
environments (e.g., 2.4 kbps HF radio). For example, 2.4 kbps codecs
"produce speech which although intelligible is far from natural
sounding" [W98]. Figure 5 of [W98] shows how the speech quality with
several forms of codecs varies with the bit rate of the codec.
7. Conclusions and Recommendations
In the near term, VoIP services are likely to be deployed, at least
in part, over broadband best-effort connections. Current real time
media encoding and transmission practice ignores congestion
considerations, resulting in the potential for trouble should VoIP
become a broadly deployed service in the near to intermediate term.
Poor user quality, unfairness to other VoIP and TCP users, and the
possibility of sporadic episodes of congestion collapse are some of
the potential problems in this scenario.
These problems can be mitigated in applications that use fixed-rate
codecs by requiring the best-effort VoIP application to specify its
minimum bit throughput rate. This minimum bit rate can be used to
estimate a packet drop rate at which the application would terminate.
This document specifically recommends the following:
(1) In IETF standards for protocols regarding best-effort flows with
a minimum sending rate, a packet drop rate must be specified, such
that the best-effort flow terminates, or suspends sending
temporarily, when the steady-state packet drop rate significantly
exceeds the specified drop rate.
(2) The specified drop rate for the minimum sending rate should be
consistent with the use of Tables 1 and 2 as illustrated in this
document.
We note that this is a recommendation to the IETF community, as a
specific follow-up to RFC 2914 on Congestion Control Principles.
This is not a specific or complete protocol specification.
Codecs that are able to vary their bit rate depending on estimates of
congestion can be even more effective in providing good quality
service while maintaining network efficiency under high load
conditions. Adaptive variable-bit-rate codecs are therefore
preferable as a means of supporting VOIP sessions on shared use
Internet environments.
Real-time traffic such as VoIP could derive significant benefits from
the use of ECN, where routers may indicate congestion to end-nodes by
marking packets instead of dropping them. However, ECN is only
standardized to be used with transport protocols that react
appropriately to marked packets as indications of congestion. VoIP
traffic that follows the recommendations in this document could
satisfy the congestion-control requirements for using ECN, while VoIP
traffic with no mechanism for terminating or suspending when the
packet dropping and marking rate was too high would not. However, we
repeat that this document is not a complete protocol specification.
In particular, additional mechanisms would be required before it was
safe for applications running over UDP to use ECN. For example,
before using ECN, the sending application would have to ensure that
the receiving application was capable of receiving ECN-related
information from the lower-layer UDP stack, and of interpreting this
ECN information as a congestion indication.
8. Acknowledgements
We thank Brian Adamson, Ran Atkinson, Fred Baker, Jon Crowcroft,
Christophe Diot, Alan Duric, Jeremy George, Mark Handley, Orion
Hodson, Geoff Huston, Eddie Kohler, Simon Leinen, David Meyer, Jean-
Francois Mule, Colin Perkins, Jon Peterson, Mike Pierce, Cyrus
Shaoul, and Henning Schulzrinne for feedback on this document. (Of
course, these people do not necessarily agree with all of the
document.) Ran Atkinson and Geoff Huston contributed to the text of
the document.
The analysis in Section 6.0 resulted from a session at the whiteboard
with Mark Handley. We also thank Alberto Medina for the FreeBSD
experiments showing TCP’s sending rate as a function of the packet
drop rate.
9. References
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2988] Paxson, V. and M. Allman, "Computing TCP’s
Retransmission Timer", RFC 2988, November 2000.
[RFC3267] Sjoberg, J., Westerlund, M., Lakaniemi, A. and Q. Xie,
"Real-Time Transport Protocol (RTP) Payload Format and
File Storage Format for the Adaptive Multi-Rate (AMR)
and Adaptive Multi-Rate Wideband (AMR-WB) Audio
Codecs", RFC 3267, June 2002.
9.2. Informative References
[A02] Ran Atkinson, An ISP Reality Check, Presentation to
ieprep, 55th IETF Meeting, November 2002. URL
"http://www.ietf.cnri.reston.va.us/proceedings/
02nov/219.htm#slides".
[A03] Brian Adamson, private communication, June 2003.
[BBFS01] Deepak Bansal, Hari Balakrishnan, Sally Floyd, and
Scott Shenker, Dynamic Behavior of Slowly-Responsive
Congestion Control Algorithms, SIGCOMM 2001.
[COPS] Durham, D., Ed., Boyle, J., Cohen, R., Herzog, S.,
Rajan, R. and A. Sastry, "The COPS (Common Open Policy
Service) Protocol", RFC 2748, January 2000.
[DCCP03] Eddie Kohler, Mark Handley, Sally Floyd, and Jitendra
Padhye, Datagram Congestion Control Protocol (DCCP),
internet-draft Work in Progress, March 2003. URL
"http://www.icir.org/kohler/dcp/".
[DIFFSERV] Differentiated Services (diffserv), Concluded Working
Group, URL
"http://www.ietf.cnri.reston.va.us/html.charters/
OLD/diffserv-charter.html".
[E2E] The end2end-interest mailing list, URL
"http://www.postel.org/mailman/listinfo/end2end-
interest".
[FHPW00] S. Floyd, M. Handley, J. Padhye, J. Widmer, "Equation-
Based Congestion Control for Unicast Applications", ACM
SIGCOMM 2000.
[FM03] S. Floyd and R. Mahajan, Router Primitives for
Protection Against High-Bandwidth Flows and Aggregates,
internet draft (not yet submitted).
[FWD] Free World Dialup, URL "www.pulver.com/fwd/".
[IEPREP02] Internet Emergency Preparedness (ieprep), Minutes, 55th
IETF Meeting, November 2002. URL
"http://www.ietf.cnri.reston.va.us/proceedings/
02nov/219.htm#cmr".
[ILBRC] S.V. Andersen, et. al., Internet Low Bit Rate Codec,
Work in Progress, March 2003.
[G.114] Recommendation G.114 - One-way Transmission Time, ITU,
May 2003. URL "http://www.itu.int/itudoc/itu-
t/aap/sg12aap/recaap/g.114/".
[IVOX] The Interactive VOice eXchange, URL
"http://manimac.itd.nrl.navy.mil/IVOX/".
[Jacobson88] V. Jacobson, Congestion Avoidance and Control, ACM
SIGCOMM ’88, August 1988.
[AUT] The maximum feasible drop rate for VoIP traffic depends
on the codec. These numbers are a range for a variety
of codecs; voice quality begins to deteriorate for many
codecs around a 10% drop rate. Note from authors.
[JS00] Wenyu Jiang and Henning Schulzrinne, Modeling of Packet
Loss and Delay and Their Effect on Real-Time Multimedia
Service Quality, NOSSDAV, 2000. URL
"http://citeseer.nj.nec.com/jiang00modeling.html".
[JS02] Wenyu Jiang and Henning Schulzrinne, Comparison and
Optimization of Packet Loss Repair Methods on VoIP
Perceived Quality under Bursty Loss, NOSSDAV, 2002.
URL "http://www1.cs.columbia.edu/~wenyu/".
[JS03] Wenyu Jiang, Kazummi Koguchi, and Henning Schulzrinne,
QoS Evaluation of VoIP End-points, ICC 2003. URL
"http://www1.cs.columbia.edu/~wenyu/".
[KFK79] G.S. Kang, L.J. Fransen, and E.L. Kline, "Multirate
Processor (MRP) for Digital Voice Communications", NRL
Report 8295, Naval Research Laboratory, Washington DC,
March 1979.
[KF82] G.S. Kang and L.J. Fransen, "Second Report of the
Multirate Processor (MRP) for Digital Voice
Communications", NRL Report 8614, Naval Research
Laboratory, Washington DC, September 1982.
[Measurement] Web page on "Measurement Studies of End-to-End
Congestion Control in the Internet", URL
"http://www.icir.org/floyd/ccmeasure.html". The
section on "Network Measurements at Specific Sites"
includes measurement data about the distribution of
packet sizes on various links in the Internet.
[MTK03] A. P. Markopoulou, F. A. Tobagi, and M. J. Karam,
"Assessing the Quality of Voice Communications Over
Internet Backbones", IEEE/ACM Transactions on
Networking, V. 11 N. 5, October 2003.
[NSIS] Next Steps in Signaling (nsis), IETF Working Group, URL
"http://www.ietf.cnri.reston.va.us/html.charters/nsis-
charter.html".
[PCC] Joerg Widmer, Martin Mauve, and Jan Peter Damm.
Probabilistic Congestion Control for Non-Adaptable
Flows. Technical Report 3/2001, Department of
Mathematics and Computer Science, University of
Mannheim. URL "http://www.informatik.uni-
mannheim.de/informatik/pi4/projects/
CongCtrl/pcc/index.html".
[PFTK98] J. Padhye, V. Firoiu, D. Towsley, J. Kurose, Modeling
TCP Throughput: A Simple Model and its Empirical
Validation, Tech Report TF 98-008, U. Mass, February
1998.
[RFC896] Nagle, J., "Congestion Control in IP/TCP", RFC 896,
January 1984.
[RFC1890] Schulzrinne, H., "RTP Profile for Audio and Video
Conferences with Minimal Control", RFC 1890, January
1996.
[RFC2474] Nichols, K., Blake, S., Baker, F. and D. Black,
"Definition of the Differentiated Services Field (DS
Field) in the IPv4 and IPv6 Headers", RFC 2474,
December 1998.
[RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion
Control", RFC 2581, April 1999.
[RFC2597] Heinanen, J., Baker, F., Weiss, W. and J. Wroclawski,
"Assured Forwarding PHB Group, RFC 2597, June 1999.
[RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, RFC
2914, September 2000.
[RFC2990] Huston, G., "Next Steps for the IP QoS Architecture",
RFC 2990, November 2000.
[RFC3042] Allman, M., Balakrishnan, H. and S., Floyd, "Enhancing
TCP’s Loss Recovery Using Limited Transmit", RFC 3042,
January 2001.
[RFC3168] Ramakrishnan, K., Floyd, S. and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP", RFC
3168, September 2001.
[RFC3246] Davie, B., Charny, A., Bennet, J.C.R., Benson, K., Le
Boudec, J.Y., Courtney, W., Davari, S., Firoiu, V. and
D. Stiliadis, "An Expedited Forwarding PHB (Per-Hop
Behavior)", RFC 3246, March 2002.
[RFC3448] Handley, M., Floyd, S., Pahdye, J. and J. Widmer, "TCP
Friendly Rate Control (TFRC): Protocol Specification",
RFC 3448, January 2003.
[RSVP] Resource Reservation Setup Protocol (rsvp), Concluded
Working Group, URL
"http://www.ietf.cnri.reston.va.us/html.charters/
OLD/rsvp-charter.html".
[RTTWeb] Web Page on Round-Trip Times in the Internet, URL
"http://www.icir.org/floyd/rtt-questions.html"
[S03] H. Schulzrinne, private communication, 2003.
[RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio
and Video Conferences with Minimal Control", RFC 3551,
July 2003.
[Vonage] Vonage, URL "www.vonage.com".
[W98] J. Woodward, Speech Coding, Communications Research
Group, University of Southampton, 1998. URL
"http://www-mobile.ecs.soton.ac.uk/speech_codecs/",
10. Appendix - Sending Rates with Packet Drops
The standard way to estimate TCP’s average sending rate S in packets
per round-trip as a function of the packet drop rate would be to use
the TCP response function estimated in [PFTK98]:
S = 1/(sqrt(2p/3) + K min(1,3 sqrt(3p/8)) p (1 + 32 p^2)) (1)
for acks sent for every data packet, and the RTO set to K*RTT.
The results from Equation (1) are given in the second column in
Tables 1 and 2 below. However, Equation (1) overestimates TCP’s
sending rate in the regime with heavy packet drop rates (e.g., of 30%
or more). The analysis behind Equation (1) assumes that once a
single packet is successfully transmitted, TCP’s retransmit timer is
no longer backed-off. This might be appropriate for an environment
with ECN, or for a TCP connection using fine-grained timestamps, but
this is not necessarily the case for a non-ECN-capable TCP connection
without timestamps. As specified in [RFC2988], if TCP’s retransmit
timer is backed-off, this back-off should only be removed when TCP
successfully transmits a new packet (as opposed to a retransmitted
packet), in the absence of timestamps.
When the packet drop rate is 50% or higher, for example, many of the
successful packet transmissions can be of retransmitted packets, and
the retransmit timer can remain backed-off for significant periods of
time, in the absence of timestamps. In this case, TCP’s throughput
is determined largely by the maximum backoff of the retransmit timer.
For example, in the NS simulator the maximum backoff of the
retransmit timer is 64 times the un-backed-off value. RFC 2988
specifies that "a maximum value MAY be placed on RTO provided it is
at least 60 seconds." [Although TCP implementations vary, many TCP
implementations have a maximum of 45 seconds for the backed-off RTO
after dropped SYN packets.]
Another limitation of Equation (1) is that it models Reno TCP, and
therefore underestimates the sending rate of a modern TCP connection
that used SACK and Limited Transmit.
The table below shows estimates of the average sending rate S in
packets per RTT, for TCP connections with the RTO set to 2 RTT for
Equation (1).
These estimates are compared with simulations in the third, fourth,
and fifth columns, with ECN, packet drops for TCP with fine-grained
timestamps, and packet drops for TCP without timestamps respectively.
(The simulation scripts are available from
http://www.icir.org/floyd/VoIP/sims.) Each simulation computes the
average sending rate over the second half of a 10,000-second
simulation, and for each packet drop rate, the average is given over
50 simulations. For the simulations with very high packet drop
rates, it is sometimes the case that the SYN packet is repeatedly
dropped, and the TCP sender never successfully transmits a packet.
In this case, the TCP sender also never gets a measurement of the
round-trip time.
The sixth column of Table 1 shows the average sending rate S in
packets per RTT for an experiment using a 4.8-RELEASE FreeBSD
machine. For the low packet drop rates of 0.1 and 0.2, the sending
rate in the simulations is higher than the sending rate in the
experiments; this is probably because the TCP implementation in the
simulations uses Limited Transmit [RFC3042]. With Limited Transmit,
the TCP sender can sometimes avoid a retransmit timeout when a packet
is dropped and the congestion window is small. With high packet drop
rates of 0.65 and 0.7, the sending rate in the simulations is
somewhat lower than the sending rate in the experiments. For these
high packet drop rates, the TCP connections in the experiments would
often abort prematurely, after a sufficient number of successive
packet drops.
We note that if the ECN marking rate exceeds a locally-configured
threshold, then a router is advised to switch from marking to
dropping. As a result, we do not expect to see high steady-state
marking rates in the Internet, even if ECN is in fact deployed.
Drop
Rate p Eq(1) Sims:ECN Sims:TimeStamp Sims:Drops Experiments
------ ----- -------- -------------- ---------- -----------
0.1 2.42 2.92 2.38 2.32 0.72
0.2 .89 1.82 1.26 0.82 0.29
0.25 .55 1.52 .94 .44 0.22
0.35 .23 .99 .51 .11 0.10
0.4 .16 .75 .36 .054 0.068
0.45 .11 .55 .24 .029 0.050
0.5 .10 .37 .16 .018 0.036
0.55 .060 .25 .10 .011 0.024
0.6 .045 .15 .057 .0068 0.006