RTTM [RFC1323], TCP SACK [RFC2018], etc.).
(iv) The performance of header compression described by RFC1144 is
significantly degraded when compressed packets are lost. An
improvement, which can still incur significant penalty on
long network paths is described in [RFC2507]. This suggests
it should only be used on links (or paths) that experience a
low level of packet loss [RFC3150].
(v) The normal implementation of Header Compression inhibits
compression when IP is used to support tunneling (e.g., L2TP,
GRE [RFC2794], IP-in-IP). The tunnel encapsulation
complicates locating the appropriate packet headers. Although
GRE allows Header Compression on the inner (tunneled) IP
header [RFC2784], this is not recommended, since loss of a
packet (e.g., due to router congestion along the tunnel path)
will result in discard of all packets for one RTT [RFC1144].
RECOMMENDATION: TCP Header Compression is a transparent modification
performed at both ends of the upstream bottleneck link. It offers no
benefit for flows employing IPSec [RFC2402, RFC2406], or when
additional protocol headers are present (e.g., IP or TCP options,
and/or tunnel encapsulation headers). The scheme is widely
implemented and deployed and used over Internet links. It is
recommended to improve TCP performance for paths that have a low-to-
medium bandwidth asymmetry (e.g., k<10).
In the form described in [RFC1144], TCP performance is degraded when
used over links (or paths) that may exhibit appreciable rates of
packet loss [RFC3150]. It may also not provide significant
improvement for upstream links with bidirectional traffic. It is
therefore not desirable for paths that have a high bandwidth
asymmetry (e.g., k>10).
5.1.2 Alternate Robust Header Compression Algorithms
TCP header compression [RFC1144] and IP header compression [RFC2507]
do not perform well when subject to packet loss. Further, they do
not compress packets with TCP option fields (e.g., SACK [RFC2018] and
Timestamp (RTTM) [RFC1323]). However, recent work on more robust
schemes suggest that a new generation of compression algorithms may
be developed which are much more robust. The IETF ROHC working group
has specified compression techniques for UDP-based traffic [RFC3095]
and is examining a number of schemes that may provide improve TCP
header compression. These could be beneficial for asymmetric network
paths.
RECOMMENDATION: Robust header compression is a transparent
modification that may be performed at both ends of an upstream
bottleneck link. This class of techniques may also be suited to
Internet paths that suffer low levels of re-ordering. The techniques
benefit paths with a low-to-medium bandwidth asymmetry (e.g., k>10)
and may be robust to packet loss.
Selection of suitable compression algorithms remains an area of
ongoing research. It is possible that schemes may be derived which
support IPSec authentication, but not IPSec payload encryption. Such
schemes do not alone provide significant improvement in asymmetric
networks with a high asymmetry and/or bidirectional traffic.
5.2 TYPE 1: Reverse Link Bandwidth Management
Techniques beyond Type 0 header compression are required to address
the performance problems caused by appreciable asymmetry (k>>1). One
set of techniques is implemented only at one point on the reverse
direction path, within the router/host connected to the upstream
bottleneck link. These use flow class or per-flow queues at the
upstream link interface to manage the queue of packets waiting for
transmission on the bottleneck upstream link.
This type of technique bounds the upstream link buffer queue size,
and employs an algorithm to remove (discard) excess ACKs from each
queue. This relies on the cumulative nature of ACKs (section 4.1).
Two approaches are described which employ this type of mitigation.
5.2.1 ACK Filtering
ACK Filtering (AF) [DMT96, BPK99] (also known as ACK Suppression
[SF98, Sam99, FSS01]) is a TCP-aware link-layer technique that
reduces the number of ACKs sent on the upstream link. This technique
has been deployed in specific production networks (e.g., asymmetric
satellite networks [ASB96]). The challenge is to ensure that the
sender does not stall waiting for ACKs, which may happen if ACKs are
indiscriminately removed.
When an ACK from the receiver is about to be enqueued at a upstream
bottleneck link interface, the router or the end host link layer (if
the host is directly connected to the upstream bottleneck link)
checks the transmit queue(s) for older ACKs belonging to the same TCP
connection. If ACKs are found, some (or all of them) are removed
from the queue, reducing the number of ACKs.
Some ACKs also have other functions in TCP [RFC1144], and should not
be deleted to ensure normal operation. AF should therefore not
delete an ACK that has any data or TCP flags set (SYN, RST, URG, and
FIN). In addition, it should avoid deleting a series of 3 duplicate
ACKs that indicate the need for Fast Retransmission [RFC2581] or ACKs
with the Selective ACK option (SACK)[RFC2018] from the queue to avoid
causing problems to TCP's data-driven loss recovery mechanisms.
Appropriate treatment is also needed to preserve correct operation of
ECN feedback (carried in the TCP header) [RFC3168].
A range of policies to filter ACKs may be used. These may be either
deterministic or random (similar to a random-drop gateway, but should
take into consideration the semantics of the items in the queue).
Algorithms have also been suggested to ensure a minimum ACK rate to
guarantee the TCP sender window is updated [Sam99, FSS01], and to
limit the number of data packets (TCP segments) acknowledged by a
Stretch ACK. Per-flow state needs to be maintained only for
connections with at least one packet in the queue (similar to FRED
[LM97]). This state is soft [Cla88], and if necessary, can easily be
reconstructed from the contents of the queue.
The undesirable effect of delayed DupACKs (section 3.4) can be
reduced by deleting duplicate ACKs above a threshold value [MJW00,
CLP98] allowing Fast Retransmission, but avoiding early TCP timeouts,
which may otherwise result from excessive queuing of DupACKs.
Future schemes may include more advanced rules allowing removal of
selected SACKs [RFC2018]. Such a scheme could prevent the upstream
link queue from becoming filled by back-to-back ACKs with SACK
blocks. Since a SACK packet is much larger than an ACK, it would
otherwise add significantly to the path delay in the reverse
direction. Selection of suitable algorithms remains an ongoing area
of research.
RECOMMENDATION: ACK Filtering requires a modification to the upstream
link interface. The scheme has been deployed in some networks where
the extra processing overhead (per ACK) may be compensated for by
avoiding the need to modify TCP. ACK Filtering can generate Stretch
ACKs resulting in large bursts of TCP data packets. Therefore on its
own, it is not recommended for use in the general Internet.
ACK Filtering when used in combination with a scheme to mitigate the
effect of Stretch ACKs (i.e., control TCP sender burst size) is
recommended for paths with appreciable asymmetry (k>1) and/or with
bidirectional traffic. Suitable algorithms to support IPSec
authentication, SACK, and ECN remain areas of ongoing research.
5.2.2 ACK Decimation
ACK Decimation is based on standard router mechanisms. By using an
appropriate configuration of (small) per-flow queues and a chosen
dropping policy (e.g., Weighted Fair Queuing, WFQ) at the upstream
bottleneck link, a similar effect to AF (section 5.2.1) may be
obtained, but with less control of the actual packets which are
dropped.
In this scheme, the router/host at the bottleneck upstream link
maintains per-flow queues and services them fairly (or with
priorities) by queuing and scheduling of ACKs and data packets in the
reverse direction. A small queue threshold is maintained to drop
excessive ACKs from the tail of each queue, in order to reduce ACK
Congestion. The inability to identify special ACK packets (c.f., AF)
introduces some major drawbacks to this approach, such as the
possibility of losing DupACKs, FIN/ACK, RST packets, or packets
carrying ECN information [RFC3168]. Loss of these packets does not
significantly impact network congestion, but does adversely impact
the performance of the TCP session observing the loss.
A WFQ scheduler may assign a higher priority to interactive traffic
(providing it has a mechanism to identify such traffic) and provide a
fair share of the remaining capacity to the bulk traffic. In the
presence of bidirectional traffic, and with a suitable scheduling
policy, this may ensure fairer sharing for ACK and data packets. An
increased forward transmission rate is achieved over asymmetric links
by an increased ACK Decimation rate, leading to generation of Stretch
ACKs. As in AF, TCP sender burst size increases when Stretch ACKs
are received unless other techniques are used in combination with
this technique.
This technique has been deployed in specific networks (e.g., a
network with high bandwidth asymmetry supporting high-speed data
services to in-transit mobile hosts [Seg00]). Although not optimal,
it offered a potential mitigation applicable when the TCP header is
difficult to identify or not visible to the link layer (e.g., due to
IPSec encryption).
RECOMMENDATION: ACK Decimation uses standard router mechanisms at the
upstream link interface to constrain the rate at which ACKs are fed
to the upstream link. The technique is beneficial with paths having
appreciable asymmetry (k>1). It is however suboptimal, in that it
may lead to inefficient TCP error recovery (and hence in some cases
degraded TCP performance), and provides only crude control of link
behavior. It is therefore recommended that where possible, ACK
Filtering should be used in preference to ACK Decimation.
When ACK Decimation is used on paths with an appreciable asymmetry
(k>1) (or with bidirectional traffic) it increases the burst size of
the TCP sender, use of a scheme to mitigate the effect of Stretch
ACKs or control burstiness is therefore strongly recommended.
5.3 TYPE 2: Handling Infrequent ACKs
TYPE 2 mitigations perform TYPE 1 upstream link bandwidth management,
but also employ a second active element which mitigates the effect of
the reduced ACK rate and burstiness of ACK transmission. This is
desirable when end hosts use standard TCP sender implementations
(e.g., those not implementing the techniques in sections 4.6, 4.7).
Consider a path where a TYPE 1 scheme forwards a Stretch ACK covering
d TCP packets (i.e., where the acknowledgement number is d*MSS larger
than the last ACK received by the TCP sender). When the TCP sender
receives this ACK, it can send a burst of d (or d+1) TCP data
packets. The sender is also constrained by the current cwnd.
Received ACKs also serve to increase cwnd (by at most one MSS).
A TYPE 2 scheme mitigates the impact of the reduced ACK frequency
resulting when a TYPE 1 scheme is used. This is achieved by
interspersing additional ACKs before each received Stretch ACK. The
additional ACKs, together with the original ACK, provide the TCP
sender with sufficient ACKs to allow the TCP cwnd to open in the same
way as if each of the original ACKs sent by the TCP receiver had been
forwarded by the reverse path. In addition, by attempting to restore
the spacing between ACKs, such a scheme can also restore the TCP
self-clocking behavior, and reduce the TCP sender burst size. Such
schemes need to ensure conservative behavior (i.e., should not
introduce more ACKs than were originally sent) and reduce the
probability of ACK Compression [ZSC91].
The action is performed at two points on the return path: the
upstream link interface (where excess ACKs are removed), and a point
further along the reverse path (after the bottleneck upstream
link(s)), where replacement ACKs are inserted. This attempts to
reconstruct the ACK stream sent by the TCP receiver when used in
combination with AF (section 5.2.1), or ACK Decimation (section
5.2.2).
TYPE 2 mitigations may be performed locally at the receive interface
directly following the upstream bottleneck link, or may alternatively
be applied at any point further along the reverse path (this is not
necessarily on the forward path, since asymmetric routing may employ
different forward and reverse internet paths). Since the techniques
may generate multiple ACKs upon reception of each individual Stretch
ACK, it is strongly recommended that the expander implements a scheme
to prevent exploitation as a "packet amplifier" in a Denial-of-
Service (DoS) attack (e.g., to verify the originator of the ACK).
Identification of the sender could be accomplished by appropriately
configured packet filters and/or by tunnel authentication procedures
(e.g., [RFC2402, RFC2406]). A limit on the number of reconstructed
ACKs that may be generated from a single packet may also be
desirable.
5.3.1 ACK Reconstruction
ACK Reconstruction (AR) [BPK99] is used in conjunction with AF
(section 5.2.1). AR deploys a soft-state [Cla88] agent called an ACK
Reconstructor on the reverse path following the upstream bottleneck
link. The soft-state can be regenerated if lost, based on received
ACKs. When a Stretch ACK is received, AR introduces additional ACKs
by filling gaps in the ACK sequence. Some potential Denial-of-
Service vulnerabilities may arise (section 6) and need to be
addressed by appropriate security techniques.
The Reconstructor determines the number of additional ACKs, by
estimating the number of filtered ACKs. This uses implicit
information present in the received ACK stream by observing the ACK
sequence number of each received ACK. An example implementation
could set an ACK threshold, ackthresh, to twice the MSS (this assumes
the chosen MSS is known by the link). The factor of two corresponds
to standard TCP delayed-ACK policy (d=2). Thus, if successive ACKs
arrive separated by delta, the Reconstructor regenerates a maximum of
((delta/ackthresh) - 2) ACKs.
To reduce the TCP sender burst size and allow the cwnd to increase at
a rate governed by the downstream link, the reconstructed ACKs must
be sent at a consistent rate (i.e., temporal spacing between
reconstructed ACKs). One method is for the Reconstructor to measure
the arrival rate of ACKs using an exponentially weighted moving
average estimator. This rate depends on the output rate from the
upstream link and on the presence of other traffic sharing the link.
The output of the estimator indicates the average temporal spacing
for the ACKs (and the average rate at which ACKs would reach the TCP
sender if there were no further losses or delays). This may be used
by the Reconstructor to set the temporal spacing of reconstructed
ACKs. The scheme may also be used in combination with TCP sender
adaptation (e.g., a combination of the techniques in sections 4.6 and
4.7).
The trade-off in AR is between obtaining less TCP sender burstiness,
and a better rate of cwnd increase, with a reduction in RTT
variation, versus a modest increase in the path RTT. The technique
cannot perform reconstruction on connections using IPSec (AH
[RFC2402] or ESP [RFC2406]), since it is unable to generate
appropriate security information. It also cannot regenerate other
packet header information (e.g., the exact pattern of bits carried in
the IP packet ECN field [RFC3168] or the TCP RTTM option [RFC1323]).
An ACK Reconstructor operates correctly (i.e., generates no spurious
ACKs and preserves the end-to-end semantics of TCP), providing:
(i) the TCP receiver uses ACK Delay (d=2) [RFC2581]
(ii) the Reconstructor receives only in-order ACKs
(iii) all ACKs are routed via the Reconstructor
(iv) the Reconstructor correctly determines the TCP MSS used by
the session
(v) the packets do not carry additional header information (e.g.,
TCP RTTM option [RFC1323], IPSec using AH [RFC2402]or ESP
[RFC2406]).
RECOMMENDATION: ACK Reconstruction is an experimental transparent
modification performed on the reverse path following the upstream
bottleneck link. It is designed to be used in conjunction with a
TYPE 1 mitigation. It reduces the burst size of TCP transmission in
the forward direction, which may otherwise increase when TYPE 1
schemes are used alone. It requires modification of equipment after
the upstream link (including maintaining per-flow soft state). The
scheme introduces implicit assumptions about the network path and has
potential Denial-of-Service vulnerabilities (i.e., acting as a packet
amplifier); these need to be better understood and addressed by
appropriate security techniques.
Selection of appropriate algorithms to pace the ACK traffic remains
an open research issue. There is also currently little experience of
the implications of using such techniques in the Internet, and
therefore it is recommended that this technique should not be used
within the Internet in its current form.
5.3.2 ACK Compaction and Companding
ACK Compaction and ACK Companding [SAM99, FSS01] are techniques that
operate at a point on the reverse path following the constrained ACK
bottleneck. Like AR (section 5.3.1), ACK Compaction and ACK
Companding are both used in conjunction with an AF technique (section
5.2.1) and regenerate filtered ACKs, restoring the ACK stream.
However, they differ from AR in that they use a modified AF (known as
a compactor or compressor), in which explicit information is added to
all Stretch ACKs generated by the AF. This is used to explicitly
synchronize the reconstruction operation (referred to here as
expansion).
The modified AF combines two modifications: First, when the
compressor deletes an ACK from the upstream bottleneck link queue, it
appends explicit information (a prefix) to the remaining ACK (this
ACK is marked to ensure it is not subsequently deleted). The
additional information contains details the conditions under which
ACKs were previously filtered. A variety of information may be
encoded in the prefix. This includes the number of ACKs deleted by
the AF and the average number of bytes acknowledged. This may
subsequently be used by an expander at the remote end of the tunnel.
Further timing information may also be added to control the pacing of
the regenerated ACKs [FSS01]. The temporal spacing of the filtered
ACKs may also be encoded.
To encode the prefix requires the subsequent expander to recognize a
modified ACK header. This would normally limit the expander to
link-local operation (at the receive interface of the upstream
bottleneck link). If remote expansion is needed further along the
reverse path, a tunnel may be used to pass the modified ACKs to the
remote expander. The tunnel introduces extra overhead, however
networks with asymmetric capacity and symmetric routing frequently
already employ such tunnels (e.g., in a UDLR network [RFC3077], the
expander may be co-located with the feed router).
ACK expansion uses a stateless algorithm to expand the ACK (i.e.,
each received packet is processed independently of previously
received packets). It uses the prefix information together with the
acknowledgment field in the received ACK, to produce an equivalent
number of ACKs to those previously deleted by the compactor. These
ACKs are forwarded to the original destination (i.e., the TCP
sender), preserving normal TCP ACK clocking. In this way, ACK
Compaction, unlike AR, is not reliant on specific ACK policies, nor
must it see all ACKs associated with the reverse path (e.g., it may
be compatible with schemes such as DAASS [RFC2760]).
Some potential Denial-of-Service vulnerabilities may arise (section
6) and need to be addressed by appropriate security techniques. The
technique cannot perform reconstruction on connections using IPSec,
since they are unable to regenerate appropriate security information.
It is possible to explicitly encode IPSec security information from
suppressed packets, allowing operation with IPSec AH, however this
remains an open research issue, and implies an additional overhead
per ACK.
RECOMMENDATION: ACK Compaction and Companding are experimental
transparent modifications performed on the reverse path following the
upstream bottleneck link. They are designed to be used in
conjunction with a modified TYPE 1 mitigation and reduce the burst
size of TCP transmission in the forward direction, which may
otherwise increase when TYPE 1 schemes are used alone.
The technique is desirable, but requires modification of equipment
after the upstream bottleneck link (including processing of a
modified ACK header). Selection of appropriate algorithms to pace
the ACK traffic also remains an open research issue. Some potential
Denial-of-Service vulnerabilities may arise with any device that may
act as a packet amplifier. These need to be addressed by appropriate
security techniques. There is little experience of using the scheme
over Internet paths. This scheme is a subject of ongoing research
and is not recommended for use within the Internet in its current
form.
5.3.3 Mitigating TCP packet bursts generated by Infrequent ACKs
The bursts of data packets generated when a Type 1 scheme is used on
the reverse direction path may be mitigated by introducing a router
supporting Generic Traffic Shaping (GTS) on the forward path [Seg00].
GTS is a standard router mechanism implemented in many deployed
routers. This technique does not eliminate the bursts of data
generated by the TCP sender, but attempts to smooth out the bursts by
employing scheduling and queuing techniques, producing traffic which
resembles that when TCP Pacing is used (section 4.6). These
techniques require maintaining per-flow soft-state in the router, and
increase per-packet processing overhead. Some additional buffer
capacity is needed to queue packets being shaped.
To perform GTS, the router needs to select appropriate traffic
shaping parameters, which require knowledge of the network policy,
connection behavior and/or downstream bottleneck characteristics. GTS
may also be used to enforce other network policies and promote
fairness between competing TCP connections (and also UDP and
multicast flows). It also reduces the probability of ACK Compression
[ZSC91].
The smoothing of packet bursts reduces the impact of the TCP
transmission bursts on routers and hosts following the point at which
GTS is performed. It is therefore desirable to perform GTS near to
the sending host, or at least at a point before the first forward
path bottleneck router.
RECOMMENDATIONS: Generic Traffic Shaping (GTS) is a transparent
technique employed at a router on the forward path. The algorithms
to implement GTS are available in widely deployed routers and may be
used on an Internet link, but do imply significant additional per-
packet processing cost.
Configuration of a GTS is a policy decision of a network service
provider. When appropriately configured the technique will reduce
size of TCP data packet bursts, mitigating the effects of Type 1
techniques. GTS is recommended for use in the Internet in
conjunction with type 1 techniques such as ACK Filtering (section
5.2.1) and ACK Decimation (section 5.2.2).
5.4 TYPE 3: Upstream Link Scheduling
Many of the above schemes imply using per flow queues (or per
connection queues in the case of TCP) at the upstream bottleneck
link. Per-flow queuing (e.g., FQ, CBQ) offers benefit when used on
any slow link (where the time to transmit a packet forms an
appreciable part of the path RTT) [RFC3150]. Type 3 schemes offer
additional benefit when used with one of the above techniques.
5.4.1 Per-Flow queuing at the Upstream Bottleneck Link
When bidirectional traffic exists in a bandwidth asymmetric network
competing ACK and packet data flows along the return path may degrade
the performance of both upstream and downstream flows [KVR98].
Therefore, it is highly desirable to use a queuing strategy combined
with a scheduling mechanism at the upstream link. This has also been
called priority-based multiplexing [RFC3135].
On a slow upstream link, appreciable jitter may be introduced by
sending large data packets ahead of ACKs [RFC3150]. A simple scheme
may be implemented using per-flow queuing with a fair scheduler
(e.g., round robin service to all flows, or priority scheduling). A
modified scheduler [KVR98] could place a limit on the number of ACKs
a host is allowed to transmit upstream before transmitting a data
packet (assuming at least one data packet is waiting in the upstream
link queue). This guarantees at least a certain minimum share of the
capacity to flows in the reverse direction, while enabling flows in
the forward direction to improve TCP throughput.
Bulk (payload) compression, a small MTU, link level transparent
fragmentation [RFC1991, RFC2686] or link level suspend/resume
capability (where higher priority frames may pre-empt transmission of
lower priority frames) may be used to mitigate the impact (jitter) of
bidirectional traffic on low speed links [RFC3150]. More advanced
schemes (e.g., WFQ) may also be used to improve the performance of
transfers with multiple ACK streams such as http [Seg00].
RECOMMENDATION: Per-flow queuing is a transparent modification
performed at the upstream bottleneck link. Per-flow (or per-class)
scheduling does not impact the congestion behavior of the Internet,
and may be used on any Internet link. The scheme has particular
benefits for slow links. It is widely implemented and widely
deployed on links operating at less than 2 Mbps. This is recommended
as a mitigation on its own or in combination with one of the other
described techniques.
5.4.2 ACKs-first Scheduling
ACKs-first Scheduling is an experimental technique to improve
performance of bidirectional transfers. In this case data packets
and ACKs compete for resources at the upstream bottleneck link
[RFC3150]. A single First-In First-Out, FIFO, queue for both data
packets and ACKs could impact the performance of forward transfers.
For example, if the upstream bottleneck link is a 28.8 kbps dialup
line, the transmission of a 1 Kbyte sized data packet would take
about 280 ms. So even if just two such data packets get queued ahead
of ACKs (not an uncommon occurrence since data packets are sent out
in pairs during slow start), they would shut out ACKs for well over
half a second. If more than two data packets are queued up ahead of
an ACK, the ACKs would be delayed by even more [RFC3150].
A possible approach to alleviating this is to schedule data and ACKs
differently from FIFO. One algorithm, in particular, is ACKs-first
scheduling, which accords a higher priority to ACKs over data
packets. The motivation for such scheduling is that it minimizes the
idle time for the forward connection by minimizing the time that ACKs
spend queued behind data packets at the upstream link. At the same
time, with Type 0 techniques such as header compression [RFC1144],
the transmission time of ACKs becomes small enough that the impact on
subsequent data packets is minimal. (Subnetworks in which the per-
packet overhead of the upstream link is large, e.g., packet radio
subnetworks, are an exception, section 3.2.) This scheduling scheme
does not require the upstream bottleneck router/host to explicitly
identify or maintain state for individual TCP connections.
ACKs-first scheduling does not help avoid a delay due to a data
packet in transmission. Link fragmentation or suspend/resume may be
beneficial in this case.
RECOMMENDATION: ACKs-first scheduling is an experimental transparent
modification performed at the upstream bottleneck link. If it is
used without a mechanism (such as ACK Congestion Control (ACC),
section 4.3) to regulate the volume of ACKs, it could lead to
starvation of data packets. This is a performance penalty
experienced by end hosts using the link and does not modify Internet
congestion behavior. Experiments indicate that ACKs-first scheduling
in combination with ACC is promising. However, there is little
experience of using the technique in the wider Internet. Further
development of the technique remains an open research issue, and
therefore the scheme is not currently recommended for use within the
Internet.
6. Security Considerations
The recommendations contained in this document do not impact the
integrity of TCP, introduce new security implications to the TCP
protocol, or applications using TCP.
Some security considerations in the context of this document arise
from the implications of using IPSec by the end hosts or routers
operating along the return path. Use of IPSec prevents, or
complicates, some of the mitigations. For example:
(i) When IPSec ESP [RFC2406] is used to encrypt the IP payload, the
TCP header can neither be read nor modified by intermediate
entities. This rules out header compression, ACK Filtering, ACK
Reconstruction, and the ACK Compaction.
(ii) The TCP header information may be visible, when some forms of
network layer security are used. For example, using IPSec AH
[RFC2402], the TCP header may be read, but not modified, by
intermediaries. This may in future allow extensions to support
ACK Filtering, but rules out the generation of new
packets by intermediaries (e.g., ACK Reconstruction). The
enhanced header compression scheme discussed in [RFC2507] would
also work with IPSec AH.
There are potential Denial-of-Service (DoS) implications when using
Type 2 schemes. Unless additional security mechanisms are used, a
Reconstructor/expander could be exploited as a packet amplifier. A
third party may inject unauthorized Stretch ACKs into the reverse
path, triggering the generation of additional ACKs. These ACKs would
consume capacity on the return path and processing resources at the
systems along the path, including the destination host. This
provides a potential platform for a DoS attack. The usual
precautions must be taken to verify the correct tunnel end point, and
to ensure that applications cannot falsely inject packets that expand
to generate unwanted traffic. Imposing a rate limit and bound on the
delayed ACK factor(d) would also lessen the impact of any undetected
exploitation.
7. Summary
This document considers several TCP performance constraints that
arise from asymmetry in the properties of the forward and reverse
paths across an IP network. Such performance constraints arise,
e.g., as a result of both bandwidth (capacity) asymmetry, asymmetric
shared media in the reverse direction, and interactions with Media
Access Control (MAC) protocols. Asymmetric capacity may cause TCP
Acknowledgments (ACKs) to be lost or become inordinately delayed
(e.g., when a bottleneck link is shared between many flows, or when
there is bidirectional traffic). This effect may be exacerbated with
media-access delays (e.g., in certain multi-hop radio subnetworks,
satellite Bandwidth on Demand access). Asymmetry, and particular
high asymmetry, raises a set of TCP performance issues.
A set of techniques providing performance improvement is surveyed.
These include techniques to alleviate ACK Congestion and techniques
that enable a TCP sender to cope with infrequent ACKs without
destroying TCP self-clocking. These techniques include both end-to-
end, local link-layer, and subnetwork schemes. Many of these
techniques have been evaluated in detail via analysis, simulation,
and/or implementation on asymmetric subnetworks forming part of the
Internet. There is however as yet insufficient operational
experience for some techniques, and these therefore currently remain
items of on-going research and experimentation.
The following table summarizes the current recommendations.
Mechanisms are classified as recommended (REC), not recommended (NOT
REC) or experimental (EXP). Experimental techniques may not be well
specified. These techniques will require further operational
experience before they can be recommended for use in the public
Internet.
The recommendations for end-to-end host modifications are summarized
in table 1. This lists each technique, the section in which each
technique is discussed, and where it is applied (S denotes the host
sending TCP data packets in the forward direction, R denotes the host
which receives these data packets).
+------------------------+-------------+------------+--------+
| Technique | Use | Section | Where |
+------------------------+-------------+------------+--------+
| Modified Delayed ACKs | NOT REC | 4.1 | TCP R |
| Large MSS & NO FRAG | REC | 4.2 | TCP SR |
| Large MSS & IP FRAG | NOT REC | 4.2 | TCP SR |
| ACK Congestion Control | EXP | 4.3 | TCP SR |
| Window Pred. Mech (WPM)| NOT REC | 4.4 | TCP R |
| Window Cwnd. Est. (ACE)| NOT REC | 4.5 | TCP R |
| TCP Sender Pacing | EXP *1 | 4.6 | TCP S |
| Byte Counting | NOT REC *2 | 4.7 | TCP S |
| Backpressure | EXP *1 | 4.8 | TCP R |
+------------------------+-------------+------------+--------+
Table 1: Recommendations concerning host modifications.
*1 Implementation of the technique may require changes to the
internal design of the protocol stack in end hosts.
*2 Dependent on a scheme for preventing excessive TCP transmission
burst.
The recommendations for techniques that do not require the TCP sender
and receiver to be aware of their existence (i.e., transparent
techniques) are summarized in table 2. Each technique is listed
along with the section in which each mechanism is discussed, and
where the technique is applied (S denotes the sending interface prior
to the upstream bottleneck link, R denotes receiving interface
following the upstream bottleneck link).
+------------------------+-------------+------------+--------+
| Mechanism | Use | Section | Type |
+------------------------+-------------+------------+--------+
| Header Compr. (V-J) | REC *1 | 5.1.1 | 0 SR |
| Header Compr. (ROHC) | REC *1 *2 | 5.1.2 | 0 SR |
+------------------------+-------------+------------+--------+
| ACK Filtering (AF) | EXP *3 | 5.2.1 | 1 S |
| ACK Decimation | EXP *3 | 5.2.2 | 1 S |
+------------------------+-------------+------------+--------+
| ACK Reconstruction (AR)| NOT REC | 5.3.1 | 2 *4 |
| ACK Compaction/Compand.| EXP | 5.3.2 | 2 S *4 |
| Gen. Traff. Shap. (GTS)| REC | 5.3.3 | 2 *5 |
+------------------------+-------------+------------+--------+
| Fair Queueing (FQ) | REC | 5.4.1 | 3 S |
| ACKs-First Scheduling | NOT REC | 5.4.2 | 3 S |
+------------------------+-------------+------------+--------+
Table 2: Recommendations concerning transparent modifications.
*1 At high asymmetry these schemes may degrade TCP performance, but
are not considered harmful to the Internet.
*2 Standardisation of new TCP compression protocols is the subject of
ongoing work within the ROHC WG, refer to other IETF RFCs on the
use of these techniques.
*3 Use in the Internet is dependent on a scheme for preventing
excessive TCP transmission burst.
*4 Performed at a point along the reverse path after the upstream
bottleneck link.
*5 Performed at a point along the forward path.
8. Acknowledgments
This document has benefited from comments from the members of the
Performance Implications of Links (PILC) Working Group. In
particular, the authors would like to thank John Border, Spencer
Dawkins, Aaron Falk, Dan Grossman, Randy Katz, Jeff Mandin, Rod
Ragland, Ramon Segura, Joe Touch, and Lloyd Wood for their useful
comments. They also acknowledge the data provided by Metricom Inc.,
concerning operation of their packet data network.
9. References
References of the form RFCnnnn are Internet Request for Comments
(RFC) documents available online at http://www.rfc-editor.org/.
9.1 Normative References
[RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC
793, September 1981.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC1122, October 1989.
[RFC1144] Jacobson, V., "Compressing TCP/IP Headers for Low-Speed
Serial Links", RFC1144, February 1990.
[RFC1191] Mogul, J. and S. Deering, "Path MTU Discovery", RFC1191,
November 1990.
[RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion
Control", RFC2581, April 1999.
[RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D. and P. Traina,
"Generic Routing Encapsulation (GRE)", RFC2784, March
2000.
[RFC3135] Border, J., Kojo, M., Griner, J., Montenegro, G. and Z.
Shelby, "Performance Enhancing Proxies Intended to Mitigate
Link-Related Degradations", RFC3135, June 2001.
9.2 Informative References
[abc-ID] Allman, M., "TCP Congestion Control with Appropriate Byte
Counting", Work in Progress.
[All97b] Allman, M., "Fixing Two BSD TCP Bugs", Technical Report
CR-204151, NASA Lewis Research Center, October 1997.
[ANS01] ANSI Standard T1.413, "Network to Customer Installation
Interfaces - Asymmetric Digital Subscriber Lines (ADSL)
Metallic Interface", November 1998.
[ASB96] Arora, V., Suphasindhu, N., Baras, J.S. and D. Dillon,
"Asymmetric Internet Access over Satellite-Terrestrial
Networks", Proc. AIAA: 16th International Communications
Satellite Systems Conference and Exhibit, Part 1,
Washington, D.C., February 25-29, 1996, pp.476-482.
[AST00] Aggarwal, A., Savage, S., and T. Anderson, "Understanding
the Performance of TCP Pacing", Proc. IEEE INFOCOM, Tel-
Aviv, Israel, V.3, March 2000, pp. 1157-1165.
[Bal98] Balakrishnan, H., "Challenges to Reliable Data Transport
over Heterogeneous Wireless Networks", Ph.D. Thesis,
University of California at Berkeley, USA, August 1998.
http://nms.lcs.mit.edu/papers/hari-phd/
[BPK99] Balakrishnan, H., Padmanabhan, V. N., and R. H. Katz, "The
Effects of Asymmetry on TCP Performance", ACM Mobile
Networks and Applications (MONET), Vol.4, No.3, 1999, pp.
219-241. An expanded version of a paper published at Proc.
ACM/IEEE Mobile Communications Conference (MOBICOM), 1997.
[BPS00] Bennett, J. C., Partridge, C., and N. Schectman, "Packet
Reordering is Not Pathological Network Behaviour", IEEE/ACM
Transactions on Networking, Vol. 7, Issue. 6, 2000,
pp.789-798.
[Cla88] Clark, D.D, "The Design Philosophy of the DARPA Internet
Protocols", ACM Computer Communications Review (CCR), Vol.
18, Issue 4, 1988, pp.106-114.
[CLC99] Clausen, H., Linder, H., and B. Collini-Nocker, "Internet
over Broadcast Satellites", IEEE Communications Magazine,
Vol. 37, Issue. 6, 1999, pp.146-151.
[CLP98] Calveras, A., Linares, J., and J. Paradells, "Window
Prediction Mechanism for Improving TCP in Wireless
Asymmetric Links". Proc. IEEE Global Communications
Conference (GLOBECOM), Sydney Australia, November 1998,
pp.533-538.
[CR98] Cohen, R., and Ramanathan, S., "Tuning TCP for High
Performance in Hybrid Fiber Coaxial Broad-Band Access
Networks", IEEE/ACM Transactions on Networking, Vol.6,
No.1, 1998, pp.15-29.
[DS00] Cable Television Laboratories, Inc., Data-Over-Cable
Service Interface Specifications---Radio Frequency
Interface Specification SP-RFIv1.1-I04-00407, 2000
[DS01] Data-Over-Cable Service Interface Specifications, Radio
Frequency Interface Specification 1.0, SP-RFI-I05-991105,
Cable Television Laboratories, Inc., November 1999.
[DMT96] Durst, R., Miller, G., and E. Travis, "TCP Extensions for
Space Communications", ACM/IEEE Mobile Communications
Conference (MOBICOM), New York, USA, November 1996, pp.15-
26.
[EN97] "Digital Video Broadcasting (DVB); DVB Specification for
Data Broadcasting", European Standard (Telecommunications
series) EN 301 192, 1997.
[EN00] "Digital Video Broadcasting (DVB); Interaction Channel for
Satellite Distribution Systems", Draft European Standard
(Telecommunications series) ETSI, Draft EN 301 790, v.1.2.1
[FJ93] Floyd, S., and V. Jacobson, "Random Early Detection
gateways for Congestion Avoidance", IEEE/ACM Transactions
on Networking, Vol.1, No.4, 1993, pp.397-413.
[FSS01] Fairhurst, G., Samaraweera, N.K.G, Sooriyabandara, M.,
Harun, H., Hodson, K., and R. Donardio, "Performance Issues
in Asymmetric Service Provision using Broadband Satellite",
IEE Proceedings on Communication, Vol.148, No.2, 2001,
pp.95-99.
[ITU01] ITU-T Recommendation E.681, "Traffic Engineering Methods
For IP Access Networks Based on Hybrid Fiber/Coax System",
September 2001.
[ITU02] ITU-T Recommendation G.992.1, "Asymmetrical Digital
Subscriber Line (ADSL) Transceivers", July 1999.
[Jac88] Jacobson, V., "Congestion Avoidance and Control", Proc. ACM
SIGCOMM, Stanford, CA, ACM Computer Communications Review
(CCR), Vol.18, No.4, 1988, pp.314-329.
[Ken87] Kent C.A., and J. C. Mogul, "Fragmentation Considered
Harmful", Proc. ACM SIGCOMM, USA, ACM Computer
Communications Review (CCR), Vol.17, No.5, 1988, pp.390-
401.
[KSG98] Krout, T., Solsman, M., and J. Goldstein, "The Effects of
Asymmetric Satellite Networks on Protocols", Proc. IEEE
Military Communications Conference (MILCOM), Bradford, MA,
USA, Vol.3, 1998, pp.1072-1076.
[KVR98] Kalampoukas, L., Varma, A., and Ramakrishnan, K.K.,
"Improving TCP Throughput over Two-Way Asymmetric Links:
Analysis and Solutions", Proc. ACM SIGMETRICS, Medison,
USA, 1998, pp.78-89.
[LM97] Lin, D., and R. Morris, "Dynamics of Random Early
Detection", Proc. ACM SIGCOMM, Cannes, France, ACM Computer
Communications Review (CCR), Vol.27, No.4, 1997, pp.78-89.
[LMS97] Lakshman, T.V., Madhow, U., and B. Suter, "Window-based
Error Recovery and Flow Control with a Slow Acknowledgement
Channel: A Study of TCP/IP Performance", Proc. IEEE
INFOCOM, Vol.3, Kobe, Japan, 1997, pp.1199-1209.
[MJW00] Ming-Chit, I.T., Jinsong, D., and W. Wang,"Improving TCP
Performance Over Asymmetric Networks", ACM SIGCOMM, ACM
Computer Communications Review (CCR), Vol.30, No.3, 2000.
[Pad98] Padmanabhan, V.N., "Addressing the Challenges of Web Data
Transport", Ph.D. Thesis, University of California at
Berkeley, USA, September 1998 (also Tech Report UCB/CSD-
98-1016). http://www.cs.berkeley.edu/~padmanab/phd-
thesis.html
[RFC1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for
High Performance", RFC1323, May 1992.
[RFC2018] Mathis, B., Mahdavi, J., Floyd, S. and A. Romanow, "TCP
Selective Acknowledgment Options", RFC2018, October 1996.
[RFC2402] Kent, S. and R. Atkinson, "IP Authentication Header", RFC
2402, November 1998.
[RFC2406] Kent, S. and R. Atkinson, "IP Encapsulating Security
Payload (ESP)",