advancement of the cumulative acknowledgement field, that allow the
sender to distinguish, in some cases, between three duplicate
acknowledgements following a retransmitted packet that was dropped,
and three duplicate acknowledgements from the unnecessary
retransmission of three packets [Gur03, GF04]. The TCP sender MAY
use such a heuristic to decide to invoke a Fast Retransmit in some
cases, even when the three duplicate acknowledgements do not cover
more than "recover".
For example, when three duplicate acknowledgements are caused by the
unnecessary retransmission of three packets, this is likely to be
accompanied by the cumulative acknowledgement field advancing by at
least four segments. Similarly, a heuristic based on timestamps uses
the fact that when there is a hole in the sequence space, the
timestamp echoed in the duplicate acknowledgement is the timestamp of
the most recent data packet that advanced the cumulative
acknowledgement field [RFC1323]. If timestamps are used, and the
sender stores the timestamp of the last acknowledged segment, then
the timestamp echoed by duplicate acknowledgements can be used to
distinguish between a retransmitted packet that was dropped and three
duplicate acknowledgements from the unnecessary retransmission of
three packets. The heuristics are illustrated in the NS simulator in
the validation test "./test-all-newreno".
6.1. ACK Heuristic
If the ACK-based heuristic is used, then following the advancement of
the cumulative acknowledgement field, the sender stores the value of
the previous cumulative acknowledgement as prev_highest_ack, and
stores the latest cumulative ACK as highest_ack. In addition, the
following step is performed if Step 1 in Section 3 fails, before
proceeding to Step 1B.
1*) If the Cumulative Acknowledgement field didn’t cover more than
"recover", check to see if the congestion window is greater than
SMSS bytes and the difference between highest_ack and
prev_highest_ack is at most 4*SMSS bytes. If true, duplicate
ACKs indicate a lost segment (proceed to Step 1A in Section 3).
Otherwise, duplicate ACKs likely result from unnecessary
retransmissions (proceed to Step 1B in Section 3).
The congestion window check serves to protect against fast retransmit
immediately after a retransmit timeout, similar to the
"exitFastRetrans_" variable in NS. Examples of applying the ACK
heuristic are in validation tests "./test-all-newreno
newreno_rto_loss_ack" and "./test-all-newreno newreno_rto_dup_ack" in
directory "tcl/test" of the NS simulator.
If several ACKs are lost, the sender can see a jump in the cumulative
ACK of more than three segments, and the heuristic can fail. A
validation test for this scenario is "./test-all-newreno
newreno_rto_loss_ackf". RFC 2581 recommends that a receiver should
send duplicate ACKs for every out-of-order data packet, such as a
data packet received during Fast Recovery. The ACK heuristic is more
likely to fail if the receiver does not follow this advice, because
then a smaller number of ACK losses are needed to produce a
sufficient jump in the cumulative ACK.
6.2. Timestamp Heuristic
If this heuristic is used, the sender stores the timestamp of the
last acknowledged segment. In addition, the second paragraph of step
1 in Section 3 is replaced as follows:
1**) If the Cumulative Acknowledgement field didn’t cover more than
"recover", check to see if the echoed timestamp in the last
non-duplicate acknowledgment equals the stored timestamp. If
true, duplicate ACKs indicate a lost segment (proceed to Step 1A
in Section 3). Otherwise, duplicate ACKs likely result from
unnecessary retransmissions (proceed to Step 1B in Section 3).
Examples of applying the timestamp heuristic are in validation tests
"./test-all-newreno newreno_rto_loss_tsh" and "./test-all-newreno
newreno_rto_dup_tsh". The timestamp heuristic works correctly, both
when the receiver echoes timestamps as specified by [RFC1323], and by
its revision attempts. However, if the receiver arbitrarily echoes
timestamps, the heuristic can fail. The heuristic can also fail if a
timeout was spurious and returning ACKs are not from retransmitted
segments. This can be prevented by detection algorithms such as
[RFC3522].
7. Implementation Issues for the Data Receiver
[RFC2581] specifies that "Out-of-order data segments SHOULD be
acknowledged immediately, in order to accelerate loss recovery."
Neal Cardwell has noted that some data receivers do not send an
immediate acknowledgement when they send a partial acknowledgment,
but instead wait first for their delayed acknowledgement timer to
expire [C98]. As [C98] notes, this severely limits the potential
benefit of NewReno by delaying the receipt of the partial
acknowledgement at the data sender. Echoing RFC 2581, our
recommendation is that the data receiver send an immediate
acknowledgement for an out-of-order segment, even when that out-of-
order segment fills a hole in the buffer.
8. Implementation Issues for the Data Sender
In Section 3, Step 5 above, it is noted that implementations should
take measures to avoid a possible burst of data when leaving Fast
Recovery, in case the amount of new data that the sender is eligible
to send due to the new value of the congestion window is large. This
can arise during NewReno when ACKs are lost or treated as pure window
updates, thereby causing the sender to underestimate the number of
new segments that can be sent during the recovery procedure.
Specifically, bursts can occur when the FlightSize is much less than
the new congestion window when exiting from Fast Recovery. One
simple mechanism to avoid a burst of data when leaving Fast Recovery
is to limit the number of data packets that can be sent in response
to a single acknowledgment. (This is known as "maxburst_" in the ns
simulator.) Other possible mechanisms for avoiding bursts include
rate-based pacing, or setting the slow-start threshold to the
resultant congestion window and then resetting the congestion window
to FlightSize. A recommendation on the general mechanism to avoid
excessively bursty sending patterns is outside the scope of this
document.
An implementation may want to use a separate flag to record whether
or not it is presently in the Fast Recovery procedure. The use of
the value of the duplicate acknowledgment counter for this purpose is
not reliable because it can be reset upon window updates and out-of-
order acknowledgments.
When not in Fast Recovery, the value of the state variable "recover"
should be pulled along with the value of the state variable for
acknowledgments (typically, "snd_una") so that, when large amounts of
data have been sent and acked, the sequence space does not wrap and
falsely indicate that Fast Recovery should not be entered (Section 3,
step 1, last paragraph).
It is important for the sender to respond correctly to duplicate ACKs
received when the sender is no longer in Fast Recovery (e.g., because
of a Retransmit Timeout). The Limited Transmit procedure [RFC3042]
describes possible responses to the first and second duplicate
acknowledgements. When three or more duplicate acknowledgements are
received, the Cumulative Acknowledgement field doesn’t cover more
than "recover", and a new Fast Recovery is not invoked, it is
important that the sender not execute the Fast Recovery steps (3) and
(4) in Section 3. Otherwise, the sender could end up in a chain of
spurious timeouts. We mention this only because several NewReno
implementations had this bug, including the implementation in the NS
simulator. (This bug in the NS simulator was fixed in July 2003,
with the variable "exitFastRetrans_".)
9. Simulations
Simulations with NewReno are illustrated with the validation test
"tcl/test/test-all-newreno" in the NS simulator. The command
"../../ns test-suite-newreno.tcl reno" shows a simulation with Reno
TCP, illustrating the data sender’s lack of response to a partial
acknowledgement. In contrast, the command "../../ns test-suite-
newreno.tcl newreno_B" shows a simulation with the same scenario
using the NewReno algorithms described in this paper.
10. Comparisons between Reno and NewReno TCP
As we stated in the introduction, we believe that the NewReno
modification described in this document improves the performance of
the Fast Retransmit and Fast Recovery algorithms of Reno TCP in a
wide variety of scenarios. This has been discussed in some depth in
[FF96], which illustrates Reno TCP’s poor performance when multiple
packets are dropped from a window of data and also illustrates
NewReno TCP’s good performance in that scenario.
We do, however, know of one scenario where Reno TCP gives better
performance than NewReno TCP, that we describe here for the sake of
completeness. Consider a scenario with no packet loss, but with
sufficient reordering so that the TCP sender receives three duplicate
acknowledgements. This will trigger the Fast Retransmit and Fast
Recovery algorithms. With Reno TCP or with Sack TCP, this will
result in the unnecessary retransmission of a single packet, combined
with a halving of the congestion window (shown on pages 4 and 6 of
[F03]). With NewReno TCP, however, this reordering will also result
in the unnecessary retransmission of an entire window of data (shown
on page 5 of [F03]).
While Reno TCP performs better than NewReno TCP in the presence of
reordering, NewReno’s superior performance in the presence of
multiple packet drops generally outweighs its less optimal
performance in the presence of reordering. (Sack TCP is the
preferred solution, with good performance in both scenarios.) This
document recommends the Fast Retransmit and Fast Recovery algorithms
of NewReno TCP instead of those of Reno TCP for those TCP connections
that do not support SACK. We would also note that NewReno’s Fast
Retransmit and Fast Recovery mechanisms are widely deployed in TCP
implementations in the Internet today, as documented in [PF01]. For
example, tests of TCP implementations in several thousand web servers
in 2001 showed that for those TCP connections where the web browser
was not SACK-capable, more web servers used the Fast Retransmit and
Fast Recovery algorithms of NewReno than those of Reno or Tahoe TCP
[PF01].
11. Changes Relative to RFC 2582
The purpose of this document is to advance the NewReno’s Fast
Retransmit and Fast Recovery algorithms in RFC 2582 to Standards
Track.
The main change in this document relative to RFC 2582 is to specify
the Careful variant of NewReno’s Fast Retransmit and Fast Recovery
algorithms. The base algorithm described in RFC 2582 did not attempt
to avoid unnecessary multiple Fast Retransmits that can occur after a
timeout (described in more detail in the section above). However,
RFC 2582 also defined "Careful" and "Less Careful" variants that
avoid these unnecessary Fast Retransmits, and recommended the Careful
variant. This document specifies the previously-named "Careful"
variant as the basic version of NewReno. As described below, this
algorithm uses a variable "recover", whose initial value is the send
sequence number.
The algorithm specified in Section 3 checks whether the
acknowledgement field of a partial acknowledgement covers *more* than
"recover", as defined in Section 3. Another possible variant would
be to simply require that the acknowledgement field covers *more than
or equal to* "recover" before initiating another Fast Retransmit. We
called this the Less Careful variant in RFC 2582.
There are two separate scenarios in which the TCP sender could
receive three duplicate acknowledgements acknowledging "recover" but
no more than "recover". One scenario would be that the data sender
transmitted four packets with sequence numbers higher than "recover",
that the first packet was dropped in the network, and the following
three packets triggered three duplicate acknowledgements
acknowledging "recover". The second scenario would be that the
sender unnecessarily retransmitted three packets below "recover", and
that these three packets triggered three duplicate acknowledgements
acknowledging "recover". In the absence of SACK, the TCP sender is
unable to distinguish between these two scenarios.
For the Careful variant of Fast Retransmit, the data sender would
have to wait for a retransmit timeout in the first scenario, but
would not have an unnecessary Fast Retransmit in the second scenario.
For the Less Careful variant to Fast Retransmit, the data sender
would Fast Retransmit as desired in the first scenario, and would
unnecessarily Fast Retransmit in the second scenario. This document
only specifies the Careful variant in Section 3. Unnecessary Fast
Retransmits with the Less Careful variant in scenarios with
reordering are illustrated in page 8 of [F03].
The document also specifies two heuristics that the TCP sender MAY
use to decide to invoke Fast Retransmit even when the three duplicate
acknowledgements do not cover more than "recover". These heuristics,
an ACK-based heuristic and a timestamp heuristic, are described in
Sections 6.1 and 6.2 respectively.
12. Conclusions
This document specifies the NewReno Fast Retransmit and Fast Recovery
algorithms for TCP. This NewReno modification to TCP can even be
important for TCP implementations that support the SACK option,
because the SACK option can only be used for TCP connections when
both TCP end-nodes support the SACK option. NewReno performs better
than Reno (RFC 2581) in a number of scenarios discussed herein.
A number of options to the basic algorithm presented in Section 3 are
also described. These include the handling of the retransmission
timer (Section 4), the response to partial acknowledgments (Section
5), and the value of the congestion window when leaving Fast Recovery
(section 3, step 5). Our belief is that the differences between
these variants of NewReno are small compared to the differences
between Reno and NewReno. That is, the important thing is to
implement NewReno instead of Reno, for a TCP connection without SACK;
it is less important exactly which of the variants of NewReno is
implemented.
13. Security Considerations
RFC 2581 discusses general security considerations concerning TCP
congestion control. This document describes a specific algorithm
that conforms with the congestion control requirements of RFC 2581,
and so those considerations apply to this algorithm, too. There are
no known additional security concerns for this specific algorithm.
14. Acknowledgements
Many thanks to Anil Agarwal, Mark Allman, Armando Caro, Jeffrey Hsu,
Vern Paxson, Kacheong Poon, Keyur Shah, and Bernie Volz for detailed
feedback on this document or on its precursor, RFC 2582.
15. References
15.1. Normative References
[RFC2018] Mathis, M., Mahdavi, J., Floyd, S. and A. Romanow, "TCP
Selective Acknowledgement Options", RFC 2018, October 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion
Control", RFC 2581, April 1999.
[RFC2582] Floyd, S. and T. Henderson, "The NewReno Modification to
TCP’s Fast Recovery Algorithm", RFC 2582, April 1999.
[RFC2988] Paxson, V. and M. Allman, "Computing TCP’s Retransmission
Timer", RFC 2988, November 2000.
[RFC3042] Allman, M., Balakrishnan, H. and S. Floyd, "Enhancing TCP’s
Loss Recovery Using Limited Transmit", RFC 3042, January
2001.
15.2. Informative References
[C98] Cardwell, N., "delayed ACKs for retransmitted packets:
ouch!". November 1998, Email to the tcpimpl mailing list,
Message-ID "Pine.LNX.4.02A.9811021421340.26785-
100000@sake.cs.washington.edu", archived at "http://tcp-
impl.lerc.nasa.gov/tcp-impl".
[F98] Floyd, S., Revisions to RFC 2001, "Presentation to the
TCPIMPL Working Group", August 1998. URLs
"ftp://ftp.ee.lbl.gov/talks/sf-tcpimpl-aug98.ps" and
"ftp://ftp.ee.lbl.gov/talks/sf-tcpimpl-aug98.pdf".
[F03] Floyd, S., "Moving NewReno from Experimental to Proposed
Standard? Presentation to the TSVWG Working Group", March
2003. URLs "http://www.icir.org/floyd/talks/newreno-
Mar03.ps" and "http://www.icir.org/floyd/talks/newreno-
Mar03.pdf".
[FF96] Fall, K. and S. Floyd, "Simulation-based Comparisons of
Tahoe, Reno and SACK TCP", Computer Communication Review,
July 1996. URL "ftp://ftp.ee.lbl.gov/papers/sacks.ps.Z".
[F94] Floyd, S., "TCP and Successive Fast Retransmits", Technical
report, October 1994. URL
"ftp://ftp.ee.lbl.gov/papers/fastretrans.ps".
[GF04] Gurtov, A. and S. Floyd, "Resolving Acknowledgment
Ambiguity in non-SACK TCP", Next Generation Teletraffic and
Wired/Wireless Advanced Networking (NEW2AN’04), February
2004. URL "http://www.cs.helsinki.fi/u/gurtov/papers/
heuristics.html".
[Gur03] Gurtov, A., "[Tsvwg] resolving the problem of unnecessary
fast retransmits in go-back-N", email to the tsvwg mailing
list, message ID <3F25B467.9020609@cs.helsinki.fi>, July
28, 2003. URL "http://www1.ietf.org/mail-archive/working-
groups/tsvwg/current/msg04334.html".
[Hen98] Henderson, T., Re: NewReno and the 2001 Revision. September
1998. Email to the tcpimpl mailing list, Message ID
"Pine.BSI.3.95.980923224136.26134A-
100000@raptor.CS.Berkeley.EDU", archived at "http://tcp-
impl.lerc.nasa.gov/tcp-impl".
[Hoe95] Hoe, J., "Startup Dynamics of TCP’s Congestion Control and
Avoidance Schemes", Master’s Thesis, MIT, 1995.
[Hoe96] Hoe, J., "Improving the Start-up Behavior of a Congestion
Control Scheme for TCP", ACM SIGCOMM, August 1996. URL
"http://www.acm.org/sigcomm/sigcomm96/program.html".
[LM97] Lin, D. and R. Morris, "Dynamics of Random Early
Detection", SIGCOMM 97, September 1997. URL
"http://www.acm.org/sigcomm/sigcomm97/program.html".
[NS] The Network Simulator (NS). URL
"http://www.isi.edu/nsnam/ns/".
[PF01] Padhye, J. and S. Floyd, "Identifying the TCP Behavior of
Web Servers", June 2001, SIGCOMM 2001.
[RFC1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for
High Performance", RFC 1323, May 1992.
[RFC3517] Blanton, E., Allman, M., Fall, K. and L. Wang, "A
Conservative Selective Acknowledgment (SACK)-based Loss
Recovery Algorithm for TCP", RFC 3517, April 2003.
[RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm for
TCP", RFC 3522, April 2003.
Authors’ Addresses
Sally Floyd
International Computer Science Institute
Phone: +1 (510) 666-2989
EMail: floyd@acm.org
URL: http://www.icir.org/floyd/
Tom Henderson
The Boeing Company
EMail: thomas.r.henderson@boeing.com
Andrei Gurtov
TeliaSonera
EMail: andrei.gurtov@teliasonera.com
Full Copyright Statement
Copyright (C) The Internet Society (2004). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed
to pertain to the implementation or use of the technology
described in this document or the extent to which any license
under such rights might or might not be available; nor does it
represent that it has made any independent effort to identify any
such rights. Information on the procedures with respect to
rights in RFC documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use
of such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository
at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention
any copyrights, patents or patent applications, or other
proprietary rights that may cover technology that may be required
to implement this standard. Please address the information to the
IETF at ietf-ipr@ietf.org.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.