configurable load splitting ratio. This is especially useful when
no single recovery path can be found that can carry the entire
traffic of the working path in case of a fault. Split path
protection may require handshaking between the PSL and the PML(s),
and may require the PML(s) to correlate the traffic arriving on
multiple recovery paths with the working path. Although this is
an attractive option, the details of split path protection are
beyond the scope of this document.
3.4.3 Bypass Tunnels
It may be convenient, in some cases, to create a "bypass tunnel" for
a PPG between a PSL and PML, thereby allowing multiple recovery paths
to be transparent to intervening LSRs [RFC2702]. In this case, one
LSP (the tunnel) is established between the PSL and PML following an
acceptable route and a number of recovery paths can be supported
through the tunnel via label stacking. It is not necessary to apply
label stacking when using a bypass tunnel. A bypass tunnel can be
used with any of the path mapping options discussed in the previous
section.
As with recovery paths, the bypass tunnel may or may not have
resource reservations sufficient to provide recovery without service
degradation. It is possible that the bypass tunnel may have
sufficient resources to recover some number of working paths, but not
all at the same time. If the number of recovery paths carrying
traffic in the tunnel at any given time is restricted, this is
similar to the n-to-1 or n-to-m protection cases mentioned in Section
3.4.2.
3.4.4 Recovery Granularity
Another dimension of recovery considers the amount of traffic
requiring protection. This may range from a fraction of a path to a
bundle of paths.
3.4.4.1 Selective Traffic Recovery
This option allows for the protection of a fraction of traffic within
the same path. The portion of the traffic on an individual path that
requires protection is called a protected traffic portion (PTP). A
single path may carry different classes of traffic, with different
protection requirements. The protected portion of this traffic may
be identified by its class, as for example, via the EXP bits in the
MPLS shim header or via the priority bit in the ATM header.
3.4.4.2 Bundling
Bundling is a technique used to group multiple working paths together
in order to recover them simultaneously. The logical bundling of
multiple working paths requiring protection, each of which is routed
identically between a PSL and a PML, is called a protected path group
(PPG). When a fault occurs on the working path carrying the PPG, the
PPG as a whole can be protected either by being switched to a bypass
tunnel or by being switched to a recovery path.
3.4.5 Recovery Path Resource Use
In the case of pre-reserved recovery paths, there is the question of
what use these resources may be put to when the recovery path is not
in use. There are two options:
Dedicated-resource: If the recovery path resources are dedicated,
they may not be used for anything except carrying the working
traffic. For example, in the case of 1+1 protection, the working
traffic is always carried on the recovery path. Even if the recovery
path is not always carrying the working traffic, it may not be
possible or desirable to allow other traffic to use these resources.
Extra-traffic-allowed: If the recovery path only carries the working
traffic when the working path fails, then it is possible to allow
extra traffic to use the reserved resources at other times. Extra
traffic is, by definition, traffic that can be displaced (without
violating service agreements) whenever the recovery path resources
are needed for carrying the working path traffic.
Shared-resource: A shared recovery resource is dedicated for use by
multiple primary resources that (according to SRLGs) are not expected
to fail simultaneously.
3.5. Fault Detection
MPLS recovery is initiated after the detection of either a lower
layer fault or a fault at the IP layer or in the operation of MPLS-
based mechanisms. We consider four classes of impairments: Path
Failure, Path Degraded, Link Failure, and Link Degraded.
Path Failure (PF) is a fault that indicates to an MPLS-based recovery
scheme that the connectivity of the path is lost. This may be
detected by a path continuity test between the PSL and PML. Some,
and perhaps the most common, path failures may be detected using a
link probing mechanism between neighbor LSRs. An example of a
probing mechanism is a liveness message that is exchanged
periodically along the working path between peer LSRs [MPLS-PATH].
For either a link probing mechanism or path continuity test to be
effective, the test message must be guaranteed to follow the same
route as the working or recovery path, over the segment being tested.
In addition, the path continuity test must take the path merge points
into consideration. In the case of a bi-directional link implemented
as two unidirectional links, path failure could mean that either one
or both unidirectional links are damaged.
Path Degraded (PD) is a fault that indicates to MPLS-based recovery
schemes/mechanisms that the path has connectivity, but that the
quality of the connection is unacceptable. This may be detected by a
path performance monitoring mechanism, or some other mechanism for
determining the error rate on the path or some portion of the path.
This is local to the LSR and consists of excessive discarding of
packets at an interface, either due to label mismatch or due to TTL
errors, for example.
Link Failure (LF) is an indication from a lower layer that the link
over which the path is carried has failed. If the lower layer
supports detection and reporting of this fault (that is, any fault
that indicates link failure e.g., SONET LOS (Loss of Signal)), this
may be used by the MPLS recovery mechanism. In some cases, using LF
indications may provide faster fault detection than using only MPLS-
based fault detection mechanisms.
Link Degraded (LD) is an indication from a lower layer that the link
over which the path is carried is performing below an acceptable
level. If the lower layer supports detection and reporting of this
fault, it may be used by the MPLS recovery mechanism. In some cases,
using LD indications may provide faster fault detection than using
only MPLS-based fault detection mechanisms.
3.6. Fault Notification
MPLS-based recovery relies on rapid and reliable notification of
faults. Once a fault is detected, the node that detected the fault
must determine if the fault is severe enough to require path
recovery. If the node is not capable of initiating direct action
(e.g., as a point of repair, POR) the node should send out a
notification of the fault by transmitting a FIS to the POR. This can
take several forms:
(i) control plane messaging: relayed hop-by-hop along the path
upstream of the failed LSP until a POR is reached.
(ii) user plane messaging: sent downstream to the PML, which may take
corrective action (as a POR for 1+1) or communicate with a POR
upstream (for 1:n) by any of several means:
- control plane messaging
- user plane return path (either through a bi-directional LSP or
via other means)
Since the FIS is a control message, it should be transmitted with
high priority to ensure that it propagates rapidly towards the
affected POR(s). Depending on how fault notification is configured
in the LSRs of an MPLS domain, the FIS could be sent either as a
Layer 2 or Layer 3 packet [MPLS-PATH]. The use of a Layer 2-based
notification requires a Layer 2 path direct to the POR. An example
of a FIS could be the liveness message sent by a downstream LSR to
its upstream neighbor, with an optional fault notification field set
or it can be implicitly denoted by a teardown message.
Alternatively, it could be a separate fault notification packet. The
intermediate LSR should identify which of its incoming links to
propagate the FIS on.
3.7. Switch-Over Operation
3.7.1 Recovery Trigger
The activation of an MPLS protection switch following the detection
or notification of a fault requires a trigger mechanism at the PSL.
MPLS protection switching may be initiated due to automatic inputs or
external commands. The automatic activation of an MPLS protection
switch results from a response to a defect or fault conditions
detected at the PSL or to fault notifications received at the PSL.
It is possible that the fault detection and trigger mechanisms may be
combined, as is the case when a PF, PD, LF, or LD is detected at a
PSL and triggers a protection switch to the recovery path. In most
cases, however, the detection and trigger mechanisms are distinct,
involving the detection of fault at some intermediate LSR followed by
the propagation of a fault notification to the POR via the FIS, which
serves as the protection switch trigger at the POR. MPLS protection
switching in response to external commands results when the operator
initiates a protection switch by a command to a POR (or alternatively
by a configuration command to an intermediate LSR, which transmits
the FIS towards the POR).
Note that the PF fault applies to hard failures (fiber cuts,
transmitter failures, or LSR fabric failures), as does the LF fault,
with the difference that the LF is a lower layer impairment that may
be communicated to MPLS-based recovery mechanisms. The PD (or LD)
fault, on the other hand, applies to soft defects (excessive errors
due to noise on the link, for instance). The PD (or LD) results in a
fault declaration only when the percentage of lost packets exceeds a
given threshold, which is provisioned and may be set based on the
service level agreement(s) in effect between a service provider and a
customer.
3.7.2 Recovery Action
After a fault is detected or FIS is received by the POR, the recovery
action involves either a rerouting or protection switching operation.
In both scenarios, the next hop label forwarding entry for a recovery
path is bound to the working path.
3.8. Post Recovery Operation
When traffic is flowing on the recovery path, decisions can be made
as to whether to let the traffic remain on the recovery path and
consider it as a new working path or to do a switch back to the old
or to a new working path. This post recovery operation has two
styles, one where the protection counterparts, i.e., the working and
recovery path, are fixed or "pinned" to their routes, and one in
which the PSL or other network entity with real-time knowledge of
failure dynamically performs re-establishment or controlled
rearrangement of the paths comprising the protected service.
3.8.1 Fixed Protection Counterparts
For fixed protection counterparts the PSL will be pre-configured with
the appropriate behavior to take when the original fixed path is
restored to service. The choices are revertive and non-revertive
mode. The choice will typically be dependent on relative costs of
the working and protection paths, and the tolerance of the service to
the effects of switching paths yet again. These protection modes
indicate whether or not there is a preferred path for the protected
traffic.
3.8.1.1 Revertive Mode
If the working path always is the preferred path, this path will be
used whenever it is available. Thus, in the event of a fault on this
path, its unused resources will not be reclaimed by the network on
failure. Resources here may include assigned labels, links,
bandwidth etc. If the working path has a fault, traffic is switched
to the recovery path. In the revertive mode of operation, when the
preferred path is restored the traffic is automatically switched back
to it.
There are a number of implications to pinned working and recovery
paths:
- upon failure and after traffic has been moved to the recovery
path, the traffic is unprotected until such time as the path
defect in the original working path is repaired and that path
restored to service.
- upon failure and after traffic has been moved to the recovery
path, the resources associated with the original path remain
reserved.
3.8.1.2 Non-revertive Mode
In the non-revertive mode of operation, there is no preferred path or
it may be desirable to minimize further disruption of the service
brought on by a revertive switching operation. A switch-back to the
original working path is not desired or not possible since the
original path may no longer exist after the occurrence of a fault on
that path. If there is a fault on the working path, traffic is
switched to the recovery path. When or if the faulty path (the
originally working path) is restored, it may become the recovery path
(either by configuration, or, if desired, by management actions).
In the non-revertive mode of operation, the working traffic may or
may not be restored to a new optimal working path or to the original
working path anyway. This is because it might be useful, in some
cases, to either: (a) administratively perform a protection switch
back to the original working path after gaining further assurances
about the integrity of the path, or (b) it may be acceptable to
continue operation on the recovery path, or (c) it may be desirable
to move the traffic to a new optimal working path that is calculated
based on network topology and network policies. Once a new working
path has been defined, an associated recovery path may be setup.
3.8.2 Dynamic Protection Counterparts
For dynamic protection counterparts when the traffic is switched over
to a recovery path, the association between the original working path
and the recovery path may no longer exist, since the original path
itself may no longer exist after the fault. Instead, when the
network reaches a stable state following routing convergence, the
recovery path may be switched over to a different preferred path
either optimization based on the new network topology and associated
information or based on pre-configured information.
Dynamic protection counterparts assume that upon failure, the PSL or
other network entity will establish new working paths if another
switch-over will be performed.
3.8.3 Restoration and Notification
MPLS restoration deals with returning the working traffic from the
recovery path to the original or a new working path. Restoration is
performed by the PSL either upon receiving notification, via FRS,
that the working path is repaired, or upon receiving notification
that a new working path is established.
For fixed counterparts in revertive mode, an LSR that detected the
fault on the working path also detects the restoration of the working
path. If the working path had experienced a LF defect, the LSR
detects a return to normal operation via the receipt of a liveness
message from its peer. If the working path had experienced a LD
defect at an LSR interface, the LSR could detect a return to normal
operation via the resumption of error-free packet reception on that
interface. Alternatively, a lower layer that no longer detects a LF
defect may inform the MPLS-based recovery mechanisms at the LSR that
the link to its peer LSR is operational. The LSR then transmits FRS
to its upstream LSR(s) that were transmitting traffic on the working
path. At the point the PSL receives the FRS, it switches the working
traffic back to the original working path.
A similar scheme is used for dynamic counterparts where e.g., an
update of topology and/or network convergence may trigger
installation or setup of new working paths and may send notification
to the PSL to perform a switch over.
We note that if there is a way to transmit fault information back
along a recovery path towards a PSL and if the recovery path is an
equivalent working path, it is possible for the working path and its
recovery path to exchange roles once the original working path is
repaired following a fault. This is because, in that case, the
recovery path effectively becomes the working path, and the restored
working path functions as a recovery path for the original recovery
path. This is important, since it affords the benefits of non-
revertive switch operation outlined in Section 4.8.1, without leaving
the recovery path unprotected.
3.8.4 Reverting to Preferred Path (or Controlled Rearrangement)
In the revertive mode, "make before break" restoration switching can
be used, which is less disruptive than performing protection
switching upon the occurrence of network impairments. This will
minimize both packet loss and packet reordering. The controlled
rearrangement of paths can also be used to satisfy traffic
engineering requirements for load balancing across an MPLS domain.
3.9. Performance
Resource/performance requirements for recovery paths should be
specified in terms of the following attributes:
I. Resource Class Attribute:
Equivalent Recovery Class: The recovery path has the same
performance guarantees as the working path. In other words, the
recovery path meets the same SLAs as the working path.
Limited Recovery Class: The recovery path does not have the same
performance guarantees as the working path.
A. Lower Class:
The recovery path has lower resource requirements or less
stringent performance requirements than the working path.
B. Best Effort Class:
The recovery path is best effort.
II. Priority Attribute:
The recovery path has a priority attribute just like the working
path (i.e., the priority attribute of the associated traffic
trunks). It can have the same priority as the working path or
lower priority.
III. Preemption Attribute:
The recovery path can have the same preemption attribute as the
working path or a lower one.
4. MPLS Recovery Features
The following features are desirable from an operational point of
view:
I. It is desirable that MPLS recovery provides an option to
identify protection groups (PPGs) and protection portions
(PTPs).
II. Each PSL should be capable of performing MPLS recovery upon the
detection of the impairments or upon receipt of notifications of
impairments.
III. A MPLS recovery method should not preclude manual protection
switching commands. This implies that it would be possible
under administrative commands to transfer traffic from a working
path to a recovery path, or to transfer traffic from a recovery
path to a working path, once the working path becomes
operational following a fault.
IV. A PSL may be capable of performing either a switch back to the
original working path after the fault is corrected or a
switchover to a new working path, upon the discovery or
establishment of a more optimal working path.
V. The recovery model should take into consideration path merging
at intermediate LSRs. If a fault affects the merged segment,
all the paths sharing that merged segment should be able to
recover. Similarly, if a fault affects a non-merged segment,
only the path that is affected by the fault should be recovered.
5. Comparison Criteria
Possible criteria to use for comparison of MPLS-based recovery
schemes are as follows:
Recovery Time
We define recovery time as the time required for a recovery path
to be activated (and traffic flowing) after a fault. Recovery
Time is the sum of the Fault Detection Time, Hold-off Time,
Notification Time, Recovery Operation Time, and the Traffic
Restoration Time. In other words, it is the time between a
failure of a node or link in the network and the time before a
recovery path is installed and the traffic starts flowing on it.
Full Restoration Time
We define full restoration time as the time required for a
permanent restoration. This is the time required for traffic to
be routed onto links, which are capable of or have been engineered
sufficiently to handle traffic in recovery scenarios. Note that
this time may or may not be different from the "Recovery Time"
depending on whether equivalent or limited recovery paths are
used.
Setup vulnerability
The amount of time that a working path or a set of working paths
is left unprotected during such tasks as recovery path computation
and recovery path setup may be used to compare schemes. The
nature of this vulnerability should be taken into account, e.g.,
End to End schemes correlate the vulnerability with working paths,
Local Repair schemes have a topological correlation that cuts
across working paths and Network Plan approaches have a
correlation that impacts the entire network.
Backup Capacity
Recovery schemes may require differing amounts of "backup
capacity" in the event of a fault. This capacity will be
dependent on the traffic characteristics of the network. However,
it may also be dependent on the particular protection plan
selection algorithms as well as the signaling and re-routing
methods.
Additive Latency
Recovery schemes may introduce additive latency for traffic. For
example, a recovery path may take many more hops than the working
path. This may be dependent on the recovery path selection
algorithms.
Quality of Protection
Recovery schemes can be considered to encompass a spectrum of
"packet survivability" which may range from "relative" to
"absolute". Relative survivability may mean that the packet is on
an equal footing with other traffic of, as an example, the same
diff-serv code point (DSCP) in contending for the resources of the
portion of the network that survives the failure. Absolute
survivability may mean that the survivability of the protected
traffic has explicit guarantees.
Re-ordering
Recovery schemes may introduce re-ordering of packets. Also the
action of putting traffic back on preferred paths might cause
packet re-ordering.
State Overhead
As the number of recovery paths in a protection plan grows, the
state required to maintain them also grows. Schemes may require
differing numbers of paths to maintain certain levels of coverage,
etc. The state required may also depend on the particular scheme
used for recovery. The state overhead may be a function of
several parameters. For example, the number of recovery paths
and the number of the protected facilities (links, nodes, or
shared link risk groups (SRLGs)).
Loss
Recovery schemes may introduce a certain amount of packet loss
during switchover to a recovery path. Schemes that introduce loss
during recovery can measure this loss by evaluating recovery times
in proportion to the link speed.
In case of link or node failure a certain packet loss is
inevitable.
Coverage
Recovery schemes may offer various types of failover coverage.
The total coverage may be defined in terms of several metrics:
I. Fault Types: Recovery schemes may account for only link faults
or both node and link faults or also degraded service. For
example, a scheme may require more recovery paths to take node
faults into account.
II. Number of concurrent faults: dependent on the layout of recovery
paths in the protection plan, multiple fault scenarios may be
able to be restored.
III. Number of recovery paths: for a given fault, there may be one or
more recovery paths.
IV. Percentage of coverage: dependent on a scheme and its
implementation, a certain percentage of faults may be covered.
This may be subdivided into percentage of link faults and
percentage of node faults.
V. The number of protected paths may effect how fast the total set
of paths affected by a fault could be recovered. The ratio of
protection is n/N, where n is the number of protected paths and
N is the total number of paths.
6. Security Considerations
The MPLS recovery that is specified herein does not raise any
security issues that are not already present in the MPLS
architecture.
Confidentiality or encryption of information on the recovery path is
outside the scope of this document, but any method designed to do
this in other contexts may be used with the methods described in this
document.
7. Intellectual Property Considerations
The IETF has been notified of intellectual property rights claimed in
regard to some or all of the specification contained in this
document. For more information consult the online list of claimed
rights.
8. Acknowledgements
We would like to thank members of the MPLS WG mailing list for their
suggestions on the earlier versions of this document. In particular,
Bora Akyol, Dave Allan, Dave Danenberg, Sharam Davari, and Neil
Harrison whose suggestions and comments were very helpful in revising
the document.
The editors would like to give very special thanks to Curtis
Villamizar for his careful and extremely thorough reading of the
document and for taking the time to provide numerous suggestions,
which were very helpful in the last couple of revisions of the
document. Thanks are also due to Adrian Farrel for a through reading
of the last version of the document, and to Jean-Phillipe Vasseur and
Anna Charny for several useful editorial comments and suggestions,
and for input on bandwidth recovery.
9. References
9.1 Normative
[RFC3031] Rosen, E., Viswanathan, A. and R. Callon,
"Multiprotocol Label Switching Architecture", RFC3031,
January 2001.
[RFC2702] Awduche, D., Malcolm, J., Agogbua, J., O'Dell, M. and
J. McManus, "Requirements for Traffic Engineering Over
MPLS", RFC2702, September 1999.
[RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan,
V. and G. Swallow, "RSVP-TE Extensions to RSVP for LSP
Tunnels", RFC3209, December 2001.
[RFC3212] Jamoussi, B. (Ed.), Andersson, L., Callon, R., Dantu,
R., Wu, L., Doolan, P., Worster, T., Feldman, N.,
Fredette, A., Girish, M., Gray, E., Heinanen, J.,
Kilty, T. and A. Malis, "Constraint-Based LSP Setup
using LDP", RFC3212, January 2002.
9.2 Informative
[MPLS-BACKUP] Vasseur, J. P., Charny, A., LeFaucheur, F., and
Achirica, "MPLS Traffic Engineering Fast reroute:
backup tunnel path computation for bandwidth
protection", Work in Progress.
[MPLS-PATH] Haung, C., Sharma, V., Owens, K., Makam, V. "Building
Reliable MPLS Networks Using a Path Protection
Mechanism", IEEE Commun. Mag., Vol. 40, Issue 3, March
2002, pp. 156-162.
[RFC2205] Braden, R., Zhang, L., Berson, S., Herzog, S.,
"Resource ReSerVation Protocol (RSVP) -- Version 1
Functional Specification", RFC2205, September 1997.
10. Contributing Authors
This document was the collective work of several individuals over a
period of three years. The text and content of this document was
contributed by the editors and the co-authors listed below. (The
contact information for the editors appears in Section 11, and is not
repeated below.)
Ben Mack-Crane
Tellabs Operations, Inc.
1415 West Diehl Road
Naperville, IL 60563
Phone: (630) 798-6197
EMail: Ben.Mack-Crane@tellabs.com
Srinivas Makam
Eshernet, Inc.
1712 Ada Ct.
Naperville, IL 60540
Phone: (630) 308-3213
EMail: Smakam60540@yahoo.com
Ken Owens
Edward Jones Investments
201 Progress Parkway
St. Louis, MO 63146
Phone: (314) 515-3431
EMail: ken.owens@edwardjones.com
Changcheng Huang
Carleton University
Minto Center, Rm. 3082
1125 Colonial By Drive
Ottawa, Ont. K1S 5B6 Canada
Phone: (613) 520-2600 x2477
EMail: Changcheng.Huang@sce.carleton.ca
Jon Weil
Brad Cain
Storigen Systems
650 Suffolk Street
Lowell, MA 01854
Phone: (978) 323-4454
EMail: bcain@storigen.com
Loa Andersson
EMail: loa@pi.se
Bilel Jamoussi
Nortel Networks
3 Federal Street, BL3-03
Billerica, MA 01821, USA
Phone:(978) 288-4506
EMail: jamoussi@nortelnetworks.com
Angela Chiu
AT&T Labs-Research
200 Laurel Ave. Rm A5-1F13
Middletown , NJ 07748
Phone: (732) 420-9061
EMail: chiu@research.att.com
Seyhan Civanlar
Lemur Networks, Inc.
135 West 20th Street, 5th Floor
New York, NY 10011
Phone: (212) 367-7676
EMail: scivanlar@lemurnetworks.com
11. Editors' Addresses
Vishal Sharma (Editor)
Metanoia, Inc.
1600 Villa Street, Unit 352
Mountain View, CA 94041-1174
Phone: (650) 386-6723
EMail: v.sharma@ieee.org
Fiffi Hellstrand (Editor)
Nortel Networks
St Eriksgatan 115
PO Box 6701
113 85 Stockholm, Sweden
Phone: +46 8 5088 3687
EMail: fiffi@nortelnetworks.com
12. Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFCEditor function is currently provided by the
Internet Society.