with the CE codepoint set, as required by the transport protocol.
This section discusses issues of backwards compatibility with IP ECN
implementations in routers conformant with RFC2481, in which only
one ECT codepoint was defined. We do not believe that the
incremental deployment of ECN implementations that understand the
ECT(1) codepoint will cause significant operational problems. This
is particularly likely to be the case when the deployment of the
ECT(1) codepoint begins with routers, before the ECT(1) codepoint
starts to be used by end-nodes.
11.2.1. The Incremental Deployment of ECT(1) in Routers.
ECN has been an Experimental standard since January 1999, and there
are already implementations of ECN in routers that do not understand
the ECT(1) codepoint. When the use of the ECT(1) codepoint is
standardized for TCP or for other transport protocols, this could
mean that a data sender is using the ECT(1) codepoint, but that this
codepoint is not understood by a congested router on the path.
If allowed by the transport protocol, a data sender would be free not
to make use of ECT(1) at all, and to send all ECN-capable packets
with the codepoint ECT(0). However, if an ECN-capable sender is
using ECT(1), and the congested router on the path did not understand
the ECT(1) codepoint, then the router would end up marking some of
the ECT(0) packets, and dropping some of the ECT(1) packets, as
indications of congestion. Since TCP is required to react to both
marked and dropped packets, this behavior of dropping packets that
could have been marked poses no significant threat to the network,
and is consistent with the overall approach to ECN that allows
routers to determine when and whether to mark packets as they see fit
(see Section 5).
12. Summary of changes required in IP and TCP
This document specified two bits in the IP header to be used for ECN.
The not-ECT codepoint indicates that the transport protocol will
ignore the CE codepoint. This is the default value for the ECN
codepoint. The ECT codepoints indicate that the transport protocol
is willing and able to participate in ECN.
The router sets the CE codepoint to indicate congestion to the end
nodes. The CE codepoint in a packet header MUST NOT be reset by a
router.
TCP requires three changes for ECN, a setup phase and two new flags
in the TCP header. The ECN-Echo flag is used by the data receiver to
inform the data sender of a received CE packet. The Congestion
Window Reduced (CWR) flag is used by the data sender to inform the
data receiver that the congestion window has been reduced.
When ECN (Explicit Congestion Notification) is used, it is required
that congestion indications generated within an IP tunnel not be lost
at the tunnel egress. We specified a minor modification to the IP
protocol's handling of the ECN field during encapsulation and de-
capsulation to allow flows that will undergo IP tunneling to use ECN.
Two options for ECN in tunnels were specified:
1) A limited-functionality option that does not use ECN inside the IP
tunnel, by setting the ECN field in the outer header to not-ECT, and
not altering the inner header at the time of decapsulation.
2) The full-functionality option, which sets the ECN field in the
outer header to either not-ECT or to one of the ECT codepoints,
depending on the ECN field in the inner header. At decapsulation, if
the CE codepoint is set in the outer header, and the inner header is
set to one of the ECT codepoints, then the CE codepoint is copied to
the inner header.
For IPsec tunnels, this document also defines an optional IPsec
Security Association (SA) attribute that enables negotiation of ECN
usage within IPsec tunnels and an optional field in the Security
Association Database to indicate whether ECN is permitted in tunnel
mode on a SA. The required changes to IPsec tunnels for ECN usage
modify RFC2401 [RFC2401], which defines the IPsec architecture and
specifies some aspects of its implementation. The new IPsec SA
attribute is in addition to those already defined in Section 4.5 of
[RFC2407].
This document obsoletes RFC2481, "A Proposal to add Explicit
Congestion Notification (ECN) to IP", which defined ECN as an
Experimental Protocol for the Internet Community. The rest of this
section describes the relationship between this document and its
predecessor.
RFC2481 included a brief discussion of the use of ECN with
encapsulated packets, and noted that for the IPsec specifications at
the time (January 1999), flows could not safely use ECN if they were
to traverse IPsec tunnels. RFC2481 also described the changes that
could be made to IPsec tunnel specifications to made them compatible
with ECN.
This document also incorporates work that was done after RFC2481.
First was to describe the changes to IPsec tunnels in detail, and
extensively discuss the security implications of ECN (now included as
Sections 18 and 19 of this document). Second was to extend the
discussion of IPsec tunnels to include all IP tunnels. Because older
IP tunnels are not compatible with a flow's use of ECN, the
deployment of ECN in the Internet will create strong pressure for
older IP tunnels to be updated to an ECN-compatible version, using
either the limited-functionality or the full-functionality option.
This document does not address the issue of including ECN in non-IP
tunnels such as MPLS, GRE, L2TP, or PPTP. An earlier preliminary
document about adding ECN support to MPLS was not advanced.
A third new piece of work after RFC2481 was to describe the ECN
procedure with retransmitted data packets, that an ECT codepoint
should not be set on retransmitted data packets. The motivation for
this additional specification is to eliminate a possible avenue for
denial-of-service attacks on an existing TCP connection. Some prior
deployments of ECN-capable TCP might not conform to the (new)
requirement not to set an ECT codepoint on retransmitted packets; we
do not believe this will cause significant problems in practice.
This document also expands slightly on the specification of the use
of SYN packets for the negotiation of ECN. While some prior
deployments of ECN-capable TCP might not conform to the requirements
specified in this document, we do not believe that this will lead to
any performance or compatibility problems for TCP connections with a
combination of TCP implementations at the endpoints.
This document also includes the specification of the ECT(1)
codepoint, which may be used by TCP as part of the implementation of
an ECN nonce.
13. Conclusions
Given the current effort to implement AQM, we believe this is the
right time to deploy congestion avoidance mechanisms that do not
depend on packet drops alone. With the increased deployment of
applications and transports sensitive to the delay and loss of a
single packet (e.g., realtime traffic, short web transfers),
depending on packet loss as a normal congestion notification
mechanism appears to be insufficient (or at the very least, non-
optimal).
We examined the consequence of modifications of the ECN field within
the network, analyzing all the opportunities for an adversary to
change the ECN field. In many cases, the change to the ECN field is
no worse than dropping a packet. However, we noted that some changes
have the more serious consequence of subverting end-to-end congestion
control. However, we point out that even then the potential damage
is limited, and is similar to the threat posed by end-systems
intentionally failing to cooperate with end-to-end congestion
control.
14. Acknowledgements
Many people have made contributions to this work and this document,
including many that we have not managed to directly acknowledge in
this document. In addition, we would like to thank Kenjiro Cho for
the proposal for the TCP mechanism for negotiating ECN-Capability,
Kevin Fall for the proposal of the CWR bit, Steve Blake for material
on IPv4 Header Checksum Recalculation, Jamal Hadi-Salim for
discussions of ECN issues, and Steve Bellovin, Jim Bound, Brian
Carpenter, Paul Ferguson, Stephen Kent, Greg Minshall, and Vern
Paxson for discussions of security issues. We also thank the
Internet End-to-End Research Group for ongoing discussions of these
issues.
Email discussions with a number of people, including Dax Kelson,
Alexey Kuznetsov, Jamal Hadi-Salim, and Venkat Venkatsubra, have
addressed the issues raised by non-conformant equipment in the
Internet that does not respond to TCP SYN packets with the ECE and
CWR flags set. We thank Mark Handley, Jitentra Padhye, and others
for discussions on the TCP initialization procedures.
The discussion of ECN and IP tunnel considerations draws heavily on
related discussions and documents from the Differentiated Services
Working Group. We thank Tabassum Bint Haque from Dhaka, Bangladesh,
for feedback on IP tunnels. We thank Derrell Piper and Kero Tivinen
for proposing modifications to RFC2407 that improve the usability of
negotiating the ECN Tunnel SA attribute.
We thank David Wetherall, David Ely, and Neil Spring for the proposal
for the ECN nonce. We also thank Stefan Savage for discussions on
this issue. We thank Bob Briscoe and Jon Crowcroft for raising the
issue of fragmentation in IP, on alternate semantics for the fourth
ECN codepoint, and several other topics. We thank Richard Wendland
for feedback on several issues in the document.
We also thank the IESG, and in particular the Transport Area
Directors over the years, for their feedback and their work towards
the standardization of ECN.
15. References
[AH] Kent, S. and R. Atkinson, "IP Authentication Header",
RFC2402, November 1998.
[ECN] "The ECN Web Page", URL
"http://www.aciri.org/floyd/ecn.html". Reference for
informational purposes only.
[ESP] Kent, S. and R. Atkinson, "IP Encapsulating Security
Payload", RFC2406, November 1998.
[FIXES] ECN-under-Linux Unofficial Vendor Support Page, URL
"http://gtf.org/garzik/ecn/". Reference for
informational purposes only.
[FJ93] Floyd, S., and Jacobson, V., "Random Early Detection
gateways for Congestion Avoidance", IEEE/ACM
Transactions on Networking, V.1 N.4, August 1993, p.
397-413.
[Floyd94] Floyd, S., "TCP and Explicit Congestion Notification",
ACM Computer Communication Review, V. 24 N. 5, October
1994, p. 10-23.
[Floyd98] Floyd, S., "The ECN Validation Test in the NS
Simulator", URL "http://www-mash.cs.berkeley.edu/ns/",
test tcl/test/test-all- ecn. Reference for
informational purposes only.
[FF99] Floyd, S., and Fall, K., "Promoting the Use of End-to-
End Congestion Control in the Internet", IEEE/ACM
Transactions on Networking, August 1999.
[FRED] Lin, D., and Morris, R., "Dynamics of Random Early
Detection", SIGCOMM '97, September 1997.
[GRE] Hanks, S., Li, T., Farinacci, D. and P. Traina, "Generic
Routing Encapsulation (GRE)", RFC1701, October 1994.
[Jacobson88] V. Jacobson, "Congestion Avoidance and Control", Proc.
ACM SIGCOMM '88, pp. 314-329.
[Jacobson90] V. Jacobson, "Modified TCP Congestion Avoidance
Algorithm", Message to end2end-interest mailing list,
April 1990. URL
"ftp://ftp.ee.lbl.gov/email/vanj.90apr30.txt".
[K98] Krishnan, H., "Analyzing Explicit Congestion
Notification (ECN) benefits for TCP", Master's thesis,
UCLA, 1998. Citation for acknowledgement purposes only.
[L2TP] Townsley, W., Valencia, A., Rubens, A., Pall, G., Zorn,
G. and B. Palter, "Layer Two Tunneling Protocol "L2TP"",
RFC2661, August 1999.
[MJV96] S. McCanne, V. Jacobson, and M. Vetterli, "Receiver-
driven Layered Multicast", SIGCOMM '96, August 1996, pp.
117-130.
[MPLS] Awduche, D., Malcolm, J., Agogbua, J., O'Dell, M. and J.
McManus, Requirements for Traffic Engineering Over MPLS,
RFC2702, September 1999.
[PPTP] Hamzeh, K., Pall, G., Verthein, W., Taarud, J., Little,
W. and G. Zorn, "Point-to-Point Tunneling Protocol
(PPTP)", RFC2637, July 1999.
[RFC791] Postel, J., "Internet Protocol", STD 5, RFC791,
September 1981.
[RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC
793, September 1981.
[RFC1141] Mallory, T. and A. Kullberg, "Incremental Updating of
the Internet Checksum", RFC1141, January 1990.
[RFC1349] Almquist, P., "Type of Service in the Internet Protocol
Suite", RFC1349, July 1992.
[RFC1455] Eastlake, D., "Physical Link Security Type of Service",
RFC1455, May 1993.
[RFC1701] Hanks, S., Li, T., Farinacci, D. and P. Traina, "Generic
Routing Encapsulation (GRE)", RFC1701, October 1994.
[RFC1702] Hanks, S., Li, T., Farinacci, D. and P. Traina, "Generic
Routing Encapsulation over IPv4 networks", RFC1702,
October 1994.
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC2003,
October 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC2119, March 1997.
[RFC2309] Braden, B., et al., "Recommendations on Queue Management
and Congestion Avoidance in the Internet", RFC2309,
April 1998.
[RFC2401] Kent, S. and R. Atkinson, Security Architecture for the
Internet Protocol, RFC2401, November 1998.
[RFC2407] Piper, D., "The Internet IP Security Domain of
Interpretation for ISAKMP", RFC2407, November 1998.
[RFC2408] Maughan, D., Schertler, M., Schneider, M. and J. Turner,
"Internet Security Association and Key Management
Protocol (ISAKMP)", RFC2409, November 1998.
[RFC2409] Harkins D. and D. Carrel, "The Internet Key Exchange
(IKE)", RFC2409, November 1998.
[RFC2474] Nichols, K., Blake, S., Baker, F. and D. Black,
"Definition of the Differentiated Services Field (DS
Field) in the IPv4 and IPv6 Headers", RFC2474, December
1998.
[RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.
and W. Weiss, "An Architecture for Differentiated
Services", RFC2475, December 1998.
[RFC2481] Ramakrishnan K. and S. Floyd, "A Proposal to add
Explicit Congestion Notification (ECN) to IP", RFC2481,
January 1999.
[RFC2581] Alman, M., Paxson, V. and W. Stevens, "TCP Congestion
Control", RFC2581, April 1999.
[RFC2884] Hadi Salim, J. and U. Ahmed, "Performance Evaluation of
Explicit Congestion Notification (ECN) in IP Networks",
RFC2884, July 2000.
[RFC2983] Black, D., "Differentiated Services and Tunnels",
RFC2983, October 2000.
[RFC2780] Bradner S. and V. Paxson, "IANA Allocation Guidelines
For Values In the Internet Protocol and Related
Headers", BCP 37, RFC2780, March 2000.
[RJ90] K. K. Ramakrishnan and Raj Jain, "A Binary Feedback
Scheme for Congestion Avoidance in Computer Networks",
ACM Transactions on Computer Systems, Vol.8, No.2, pp.
158-181, May 1990.
[SCWA99] Stefan Savage, Neal Cardwell, David Wetherall, and Tom
Anderson, TCP Congestion Control with a Misbehaving
Receiver, ACM Computer Communications Review, October
1999.
[TBIT] Jitendra Padhye and Sally Floyd, "Identifying the TCP
Behavior of Web Servers", ICSI TR-01-002, February 2001.
URL "http://www.aciri.org/tbit/".
16. Security Considerations
Security considerations have been discussed in Sections 7, 8, 18, and
19.
17. IPv4 Header Checksum Recalculation
IPv4 header checksum recalculation is an issue with some high-end
router architectures using an output-buffered switch, since most if
not all of the header manipulation is performed on the input side of
the switch, while the ECN decision would need to be made local to the
output buffer. This is not an issue for IPv6, since there is no IPv6
header checksum. The IPv4 TOS octet is the last byte of a 16-bit
half-word.
RFC1141 [RFC1141] discusses the incremental updating of the IPv4
checksum after the TTL field is decremented. The incremental
updating of the IPv4 checksum after the CE codepoint was set would
work as follows: Let HC be the original header checksum for an ECT(0)
packet, and let HC' be the new header checksum after the CE bit has
been set. That is, the ECN field has changed from '10' to '11'.
Then for header checksums calculated with one's complement
subtraction, HC' would be recalculated as follows:
HC' = { HC - 1 HC > 1
{ 0x0000 HC = 1
For header checksums calculated on two's complement machines, HC'
would be recalculated as follows after the CE bit was set:
HC' = { HC - 1 HC > 0
{ 0xFFFE HC = 0
A similar incremental updating of the IPv4 checksum can be carried
out when the ECN field is changed from ECT(1) to CE, that is, from '
01' to '11'.
18. Possible Changes to the ECN Field in the Network
This section discusses in detail possible changes to the ECN field in
the network, such as falsely reporting congestion, disabling ECN-
Capability for an individual packet, erasing the ECN congestion
indication, or falsely indicating ECN-Capability.
18.1. Possible Changes to the IP Header
18.1.1. Erasing the Congestion Indication
First, we consider the changes that a router could make that would
result in effectively erasing the congestion indication after it had
been set by a router upstream. The convention followed is: ECN
codepoint of received packet -> ECN codepoint of packet transmitted.
Replacing the CE codepoint with the ECT(0) or ECT(1) codepoint
effectively erases the congestion indication. However, with the use
of two ECT codepoints, a router erasing the CE codepoint has no way
to know whether the original ECT codepoint was ECT(0) or ECT(1).
Thus, it is possible for the transport protocol to deploy mechanisms
to detect such erasures of the CE codepoint.
The consequence of the erasure of the CE codepoint for the upstream
router is that there is a potential for congestion to build for a
time, because the congestion indication does not reach the source.
However, the packet would be received and acknowledged.
The potential effect of erasing the congestion indication is complex,
and is discussed in depth in Section 19 below. Note that the effect
of erasing the congestion indication is different from dropping a
packet in the network. When a data packet is dropped, the drop is
detected by the TCP sender, and interpreted as an indication of
congestion. Similarly, if a sufficient number of consecutive
acknowledgement packets are dropped, causing the cumulative
acknowledgement field not to be advanced at the sender, the sender is
limited by the congestion window from sending additional packets, and
ultimately the retransmit timer expires.
In contrast, a systematic erasure of the CE bit by a downstream
router can have the effect of causing a queue buildup at an upstream
router, including the possible loss of packets due to buffer
overflow. There is a potential of unfairness in that another flow
that goes through the congested router could react to the CE bit set
while the flow that has the CE bit erased could see better
performance. The limitations on this potential unfairness are
discussed in more detail in Section 19 below.
The last of the three changes is to replace the CE codepoint with the
not-ECT codepoint, thus erasing the congestion indication and
disabling ECN-Capability at the same time.
The `erasure' of the congestion indication is only effective if the
packet does not end up being marked or dropped again by a downstream
router. If the CE codepoint is replaced by an ECT codepoint, the
packet remains ECN-Capable, and could be either marked or dropped by
a downstream router as an indication of congestion. If the CE
codepoint is replaced by the not-ECT codepoint, the packet is no
longer ECN-capable, and can therefore be dropped but not marked by a
downstream router as an indication of congestion.
18.1.2. Falsely Reporting Congestion
This change is to set the CE codepoint when an ECT codepoint was
already set, even though there was no congestion. This change does
not affect the treatment of that packet along the rest of the path.
In particular, a router does not examine the CE codepoint in deciding
whether to drop or mark an arriving packet.
However, this could result in the application unnecessarily invoking
end-to-end congestion control, and reducing its arrival rate. By
itself, this is no worse (for the application or for the network)
than if the tampering router had actually dropped the packet.
18.1.3. Disabling ECN-Capability
This change is to turn off the ECT codepoint of a packet. This means
that if the packet later encounters congestion (e.g., by arriving to
a RED queue with a moderate average queue size), it will be dropped
instead of being marked. By itself, this is no worse (for the
application) than if the tampering router had actually dropped the
packet. The saving grace in this particular case is that there is no
congested router upstream expecting a reaction from setting the CE
bit.
18.1.4. Falsely Indicating ECN-Capability
This change would incorrectly label a packet as ECN-Capable. The
packet may have been sent either by an ECN-Capable transport or a
transport that is not ECN-Capable.
If the packet later encounters moderate congestion at an ECN-Capable
router, the router could set the CE codepoint instead of dropping the
packet. If the transport protocol in fact is not ECN-Capable, then
the transport will never receive this indication of congestion, and
will not reduce its sending rate in response. The potential
consequences of falsely indicating ECN-capability are discussed
further in Section 19 below.
If the packet never later encounters congestion at an ECN-Capable
router, then the first of these two changes would have no effect,
other than possibly interfering with the use of the ECN nonce by the
transport protocol. The last change, however, would have the effect
of giving false reports of congestion to a monitoring device along
the path. If the transport protocol is ECN-Capable, then this change
could also have an effect at the transport level, by combining
falsely indicating ECN-Capability with falsely reporting congestion.
For an ECN-capable transport, this would cause the transport to
unnecessarily react to congestion. In this particular case, the
router that is incorrectly changing the ECN field could have dropped
the packet. Thus for this case of an ECN-capable transport, the
consequence of this change to the ECN field is no worse than dropping
the packet.
18.2. Information carried in the Transport Header
For TCP, an ECN-capable TCP receiver informs its TCP peer that it is
ECN-capable at the TCP level, conveying this information in the TCP
header at the time the connection is setup. This document does not
consider potential dangers introduced by changes in the transport
header within the network. We note that when IPsec is used, the
transport header is protected both in tunnel and transport modes
[ESP, AH].
Another issue concerns TCP packets with a spoofed IP source address
carrying invalid ECN information in the transport header. For
completeness, we examine here some possible ways that a node spoofing
the IP source address of another node could use the two ECN flags in
the TCP header to launch a denial-of-service attack. However, these
attacks would require an ability for the attacker to use valid TCP
sequence numbers, and any attacker with this ability and with the
ability to spoof IP source addresses could damage the TCP connection
without using the ECN flags. Therefore, ECN does not add any new
vulnerabilities in this respect.
An acknowledgement packet with a spoofed IP source address of the TCP
data receiver could include the ECE bit set. If accepted by the TCP
data sender as a valid packet, this spoofed acknowledgement packet
could result in the TCP data sender unnecessarily halving its
congestion window. However, to be accepted by the data sender, such
a spoofed acknowledgement packet would have to have the correct 32-
bit sequence number as well as a valid acknowledgement number. An
attacker that could successfully send such a spoofed acknowledgement
packet could also send a spoofed RST packet, or do other equally
damaging operations to the TCP connection.
Packets with a spoofed IP source address of the TCP data sender could
include the CWR bit set. Again, to be accepted, such a packet would
have to have a valid sequence number. In addition, such a spoofed
packet would have a limited performance impact. Spoofing a data
packet with the CWR bit set could result in the TCP data receiver
sending fewer ECE packets than it would otherwise, if the data
receiver was sending ECE packets when it received the spoofed CWR
packet.
18.3. Split Paths
In some cases, a malicious or broken router might have access to only
a subset of the packets from a flow. The question is as follows:
can this router, by altering the ECN field in this subset of the
packets, do more damage to that flow than if it had simply dropped
that set of packets?
We will classify the packets in the flow as A packets and B packets,
and assume that the adversary only has access to A packets. Assume
that the adversary is subverting end-to-end congestion control along
the path traveled by A packets only, by either falsely indicating
ECN-Capability upstream of the point where congestion occurs, or
erasing the congestion indication downstream. Consider also that
there exists a monitoring device that sees both the A and B packets,
and will "punish" both the A and B packets if the total flow is
determined not to be properly responding to indications of
congestion. Another key characteristic that we believe is likely to
be true is that the monitoring device, before `punishing' the A&B
flow, will first drop packets instead of setting the CE codepoint,
and will drop arriving packets of that flow that already have the CE
codepoint set. If the end nodes are in fact using end-to-end
congestion control, they will see all of the indications of
congestion seen by the monitoring device, and will begin to respond
to these indications of congestion. Thus, the monitoring device is
successful in providing the indications to the flow at an early
stage.
It is true that the adversary that has access only to the A packets
might, by subverting ECN-based congestion control, be able to deny
the benefits of ECN to the other packets in the A&B aggregate. While
this is unfortunate, this is not a reason to disable ECN.
A variant of falsely reporting congestion occurs when there are two
adversaries along a path, where the first adversary falsely reports
congestion, and the second adversary `erases' those reports. (Unlike
packet drops, ECN congestion reports can be `reversed' later in the
network by a malicious or broken router. However, the use of the ECN
nonce could help the transport to detect this behavior.) While this
would be transparent to the end node, it is possible that a
monitoring device between the first and second adversaries would see
the false indications of congestion. Keep in mind our recommendation
in this document, that before `punishing' a flow for not responding
appropriately to congestion, the router will first switch to dropping
rather than marking as an indication of congestion, for that flow.
When this includes dropping arriving packets from that flow that have
the CE codepoint set, this ensures that these indications of
congestion are being seen by the end nodes. Thus, there is no
additional harm that we are able to postulate as a result of multiple
conflicting adversaries.
19. Implications of Subverting End-to-End Congestion Control
This section focuses on the potential repercussions of subverting
end-to-end congestion control by either falsely indicating ECN-
Capability, or by erasing the congestion indication in ECN (the CE
codepoint). Subverting end-to-end congestion control by either of
these two methods can have consequences both for the application and
for the network. We discuss these separately below.
The first method to subvert end-to-end congestion control, that of
falsely indicating ECN-Capability, effectively subverts end-to-end
congestion control only if the packet later encounters congestion
that results in the setting of the CE codepoint. In this case, the
transport protocol (which may not be ECN-capable) does not receive
the indication of congestion from these downstream congested routers.
The second method to subvert end-to-end congestion control, `erasing'
the CE codepoint in a packet, effectively subverts end-to-end
congestion control only when the CE codepoint in the packet was set
earlier by a congested router. In this case, the transport protocol
does not receive the indication of congestion from the upstream
congested routers.
Either of these two methods of subverting end-to-end congestion
control can potentially introduce more damage to the network (and
possibly to the flow itself) than if the adversary had simply dropped
packets from that flow. However, as we discuss later in this section
and in Section 7, this potential damage is limited.
19.1. Implications for the Network and for Competing Flows
The CE codepoint of the ECN field is only used by routers as an
indication of congestion during periods of *moderate* congestion.
ECN-capable routers should drop rather than mark packets during heavy
congestion even if the router's queue is not yet full. For example,
for routers using active queue management based on RED, the router
should drop rather than mark packets that arrive while the average
queue sizes exceed the RED queue's maximum threshold.
One consequence for the network of subverting end-to-end congestion
control is that flows that do not receive the congestion indications
from the network might increase their sending rate until they drive
the network into heavier congestion. Then, the congested router
could begin to drop rather than mark arriving packets. For flows
that are not isolated by some form of per-flow scheduling or other
per-flow mechanisms, but are instead aggregated with other flows in a
single queue in an undifferentiated fashion, this packet-dropping at
the congested router would apply to all flows that share that queue.
Thus, the consequences would be to increase the level of congestion
in the network.
In some cases, the increase in the level of congestion will lead to a
substantial buffer buildup at the congested queue that will be
sufficient to drive the congested queue from the packet-marking to
the packet-dropping regime. This transition could occur either
because of buffer overflow, or because of the active queue management
policy described above that drops packets when the average queue is
above RED's maximum threshold. At this point, all flows, including
the subverted flow, will begin to see packet drops instead of packet
marks, and a malicious or broken router will no longer be able to `
erase' these indications of congestion in the network. If the end
nodes are deploying appropriate end-to-end congestion control, then
the subverted flow will reduce its arrival rate in response to
congestion. When the level of congestion is sufficiently reduced,
the congested queue can return from the packet-dropping regime to the
packet-marking regime. The steady-state pattern could be one of the
congested queue oscillating between these two regimes.
In other cases, the consequences of subverting end-to-end congestion
control will not be severe enough to drive the congested link into
sufficiently-heavy congestion that packets are dropped instead of
being marked. In this case, the implications for competing flows in
the network will be a slightly-increased rate of packet marking or
dropping, and a corresponding decrease in the bandwidth available to
those flows. This can be a stable state if the arrival rate of the
subverted flow is sufficiently small, relative to the link bandwidth,
that the average queue size at the congested router remains under
control. In particular, the subverted flow could have a limited
bandwidth demand on the link at this router, while still getting more
than its "fair" share of the link. This limited demand could be due
to a limited demand from the data source; a limitation from the TCP
advertised window; a lower-bandwidth access pipe; or other factors.
Thus the subversion of ECN-based congestion control can still lead to
unfairness, which we believe is appropriate to note here.
The threat to the network posed by the subversion of ECN-based
congestion control in the network is essentially the same as the
threat posed by an end-system that intentionally fails to cooperate
with end-to-end congestion control. The deployment of mechanisms in
routers to address this threat is an open research question, and is
discussed further in Section 10.
Let us take the example described in Section 18.1.1, where the CE
codepoint that was set in a packet is erased: {'11' -> '10' or '11'
-> '01'}. The consequence for the congested upstream router that set
the CE codepoint is that this congestion indication does not reach
the end nodes for that flow. The source (even one which is completely
cooperative and not malicious) is thus allowed to continue to
increase its sending rate (if it is a TCP flow, by increasing its
congestion window). The flow potentially achieves better throughput
than the other flows that also share the congested router, especially
if there are no policing mechanisms or per-flow queuing mechanisms at
that router. Consider the behavior of the other flows, especially if
they are cooperative: that is, the flows that do not experience
subverted end-to-end congestion control. They are likely to reduce
their load (e.g., by reducing their window size) on the congested
router, thus benefiting our subverted flow. This results in
unfairness. As we discussed above, this unfairness could either be
transient (because the congested queue is driven into the packet-
marking regime), oscillatory (because the congested queue oscillates
between the packet marking and the packet dropping regime), or more
moderate but a persistent stable state (because the congested queue
is never driven to the packet dropping regime).
The results would be similar if the subverted flow was intentionally
avoiding end-to-end congestion control. One difference is that a
flow that is intentionally avoiding end-to-end congestion control at
the end nodes can avoid end-to-end congestion control even when the
congested queue is in packet-dropping mode, by refusing to reduce its
sending rate in response to packet drops in the network. Thus the
problems for the network from the subversion of ECN-based congestion
control are less severe than the problems caused by the intentional
avoidance of end-to-end congestion control in the end nodes. It is
also the case that it is considerably more difficult to control the
behavior of the end nodes than it is to control the behavior of the
infrastructure itself. This is not to say that the problems for the
network posed by the network's subversion of ECN-based congestion
control are small; just that they are dwarfed by the problems for the
network posed by the subversion of either ECN-based or other
currently known packet-based congestion control mechanisms by the end
nodes.
19.2. Implications for the Subverted Flow
When a source indicates that it is ECN-capable, there is an
expectation that the routers in the network that are capable of
participating in ECN will use the CE codepoint for indication of
congestion. There is the potential benefit of using ECN in reducing
the amount of packet loss (in addition to the reduced queuing delays
because of active queue management policies). When the packet flows
through an IPsec tunnel where the nodes that the tunneled packets
traverse are untrusted in some way, the expectation is that IPsec
will protect the flow from subversion that results in undesirable
consequences.
In many cases, a subverted flow will benefit from the subversion of
end-to-end congestion control for that flow in the network, by
receiving more bandwidth than it would have otherwise, relative to
competing non-subverted flows. If the congested queue reaches the
packet-dropping stage, then the subversion of end-to-end congestion
control might or might not be of overall benefit to the subverted
flow, depending on that flow's relative tradeoffs between throughput,
loss, and delay.
One form of subverting end-to-end congestion control is to falsely
indicate ECN-capability by setting the ECT codepoint. This has the
consequence of downstream congested routers setting the CE codepoint
in vain. However, as described in Section 9.1.2, if an ECT codepoint
is changed in an IP tunnel, this can be detected at the egress point
of the tunnel, as long as the inner header was not changed within the
tunnel.
The second form of subverting end-to-end congestion control is to
erase the congestion indication by erasing the CE codepoint. In this
case, it is the upstream congested routers that set the CE codepoint
in vain.
If an ECT codepoint is erased within an IP tunnel, then this can be
detected at the egress point of the tunnel, as long as the inner
header was not changed within the tunnel. If the CE codepoint is set
upstream of the IP tunnel, then any erasure of the outer header's CE
codepoint within the tunnel will have no effect because the inner
header preserves the set value of the CE codepoint. However, if the
CE codepoint is set within the tunnel, and erased either within or
downstream of the tunnel, this is not necessarily detected at the
egress point of the tunnel.
With this subversion of end-to-end congestion control, an end-system
transport does not respond to the congestion indication. Along with
the increased unfairness for the non-subverted flows described in the
previous section, the congested router's queue could continue to
build, resulting in packet loss at the congested router - which is a
means for indicating congestion to the transport in any case. In the
interim, the flow might experience higher queuing delays, possibly
along with an increased bandwidth relative to other non-subverted
flows. But transports do not inherently make assumptions of
consistently experiencing carefully managed queuing in the path. We
believe that these forms of subverting end-to-end congestion control
are no worse for the subverted flow than if the adversary had simply
dropped the packets of that flow itself.
19.3. Non-ECN-Based Methods of Subverting End-to-end Congestion Control
We have shown that, in many cases, a malicious or broken router that
is able to change the bits in the ECN field can do no more damage
than if it had simply dropped the packet in question. However, this
is not true in all cases, in particular in the cases where the broken
router subverted end-to-end congestion control by either falsely
indicating ECN-Capability or by erasing the ECN congestion indication
(in the CE codepoint). While there are many ways that a router can
harm a flow by dropping packets, a router cannot subvert end-to-end
congestion control by dropping packets. As an example, a router
cannot subvert TCP congestion control by dropping data packets,
acknowledgement packets, or control packets.
Even though packet-dropping cannot be used to subvert end-to-end
congestion control, there *are* non-ECN-based methods for subverting
end-to-end congestion control that a broken or malicious router could
use. For example, a broken router could duplicate data packets, thus
effectively negating the effects of end-to-end congestion control
along some portion of the path. (For a router that duplicated
packets within an IPsec tunnel, the security administrator can cause
the duplicate packets to be discarded by configuring anti-replay
protection for the tunnel.) This duplication of packets within the
network would have similar implications for the network and for the
subverted flow as those described in Sections 18.1.1 and 18.1.4
above.
20. The Motivation for the ECT Codepoints.
20.1. The Motivation for an ECT Codepoint.
The need for an ECT codepoint is motivated by the fact that ECN will
be deployed incrementally in an Internet where some transport
protocols and routers understand ECN and some do not. With an ECT
codepoint, the router can drop packets from flows that are not ECN-
capable, but can *instead* set the CE codepoint in packets that *are*
ECN-capable. Because an ECT codepoint allows an end node to have the
CE codepoint set in a packet *instead* of having the packet dropped,
an end node might have some incentive to deploy ECN.
If there was no ECT codepoint, then the router would have to set the
CE codepoint for packets from both ECN-capable and non-ECN-capable
flows. In this case, there would be no incentive for end-nodes to
deploy ECN, and no viable path of incremental deployment from a non-
ECN world to an ECN-capable world. Consider the first stages of such
an incremental deployment, where a subset of the flows are ECN-
capable. At the onset of congestion, when the packet
dropping/marking rate would be low, routers would only set CE
codepoints, rather than dropping packets. However, only those flows
that are ECN-capable would understand and respond to CE packets. The
result is that the ECN-capable flows would back off, and the non-
ECN-capable flows would be unaware of the ECN signals and would
continue to open their congestion windows.
In this case, there are two possible outcomes: (1) the ECN-capable
flows back off, the non-ECN-capable flows get all of the bandwidth,
and congestion remains mild, or (2) the ECN-capable flows back off,
the non-ECN-capable flows don't, and congestion increases until the
router transitions from setting the CE codepoint to dropping packets.
While this second outcome evens out the fairness, the ECN-capable
flows would still receive little benefit from being ECN-capable,
because the increased congestion would drive the router to packet-
dropping behavior.
A flow that advertised itself as ECN-Capable but does not respond to
CE codepoints is functionally equivalent to a flow that turns off
congestion control, as discussed earlier in this document.
Thus, in a world when a subset of the flows are ECN-capable, but
where ECN-capable flows have no mechanism for indicating that fact to
the routers, there would be less effective and less fair congestion
control in the Internet, resulting in a strong incentive for end
nodes not to deploy ECN.
20.2. The Motivation for two ECT Codepoints.
The primary motivation for the two ECT codepoints is to provide a
one-bit ECN nonce. The ECN nonce allows the development of
mechanisms for the sender to probabilistically verify that network
elements are not erasing the CE codepoint, and that data receivers
are properly reporting to the sender the receipt of packets with the
CE codepoint set.
Another possibility for senders to detect misbehaving network
elements or receivers would be for the data sender to occasionally
send a data packet with the CE codepoint set, to see if the receiver
reports receiving the CE codepoint. Of course, if these packets
encountered congestion in the network, the router might make no
change in the packets, because the CE codepoint would already be set.