consideration.
4.3.3. Use of Route Reflectors
Rather than having a complete IBGP mesh among the PEs, it is
advantageous to make use of BGP Route Reflectors [BGP-RR] to improve
scalability. All the usual techniques for using route reflectors to
improve scalability (e.g., route reflector hierarchies) are
available.
Route reflectors are the only systems that need to have routing
information for VPNs to which they are not directly attached.
However, there is no need to have any one route reflector know all
the VPN-IPv4 routes for all the VPNs supported by the backbone.
We outline below two different ways to partition the set of VPN-IPv4
routes among a set of route reflectors.
1. Each route reflector is preconfigured with a list of Route
Targets. For redundancy, more than one route reflector may be
preconfigured with the same list. A route reflector uses the
preconfigured list of Route Targets to construct its inbound
route filtering. The route reflector may use the techniques of
[BGP-ORF] to install on each of its peers (regardless of
whether the peer is another route reflector or a PE) the set of
Outbound Route Filters (ORFs) that contains the list of its
preconfigured Route Targets. Note that route reflectors should
accept ORFs from other route reflectors, which means that route
reflectors should advertise the ORF capability to other route
reflectors.
A service provider may modify the list of preconfigured Route
Targets on a route reflector. When this is done, the route
reflector modifies the ORFs it installs on all of its IBGP
peers. To reduce the frequency of configuration changes on
route reflectors, each route reflector may be preconfigured
with a block of Route Targets. This way, when a new Route
Target is needed for a new VPN, there is already one or more
route reflectors that are (pre)configured with this Route
Target.
Unless a given PE is a client of all route reflectors, when a
new VPN is added to the PE ("VPN Join"), it will need to become
a client of the route reflector(s) that maintain routes for
that VPN. Likewise, deleting an existing VPN from the PE ("VPN
Prune") may result in a situation where the PE no longer needs
to be a client of some route reflector(s). In either case, the
Join or Prune operation is non-disruptive (as long as
[BGP-RFSH] is used, and never requires a BGP connection to be
brought down, only to be brought right back up.
(By "adding a new VPN to a PE", we really mean adding a new
import Route Target to one of its VRFs, or adding a new VRF
with an import Route Target not had by any of the PE’s other
VRFs.)
2. Another method is to have each PE be a client of some subset of
the route reflectors. A route reflector is not preconfigured
with the list of Route Targets, and does not perform inbound
route filtering of routes received from its clients (PEs);
rather, it accepts all the routes received from all of its
clients (PEs). The route reflector keeps track of the set of
the Route Targets carried by all the routes it receives. When
the route reflector receives from its client a route with a
Route Target that is not in this set, this Route Target is
immediately added to the set. On the other hand, when the
route reflector no longer has any routes with a particular
Route Target that is in the set, the route reflector should
delay (by a few hours) the deletion of this Route Target from
the set.
The route reflector uses this set to form the inbound route
filters that it applies to routes received from other route
reflectors. The route reflector may also use ORFs to install
the appropriate outbound route filtering on other route
reflectors. Just like with the first approach, a route
reflector should accept ORFs from other route reflectors. To
accomplish this, a route reflector advertises ORF capability to
other route reflectors.
When the route reflector changes the set, it should immediately
change its inbound route filtering. In addition, if the route
reflector uses ORFs, then the ORFs have to be immediately
changed to reflect the changes in the set. If the route
reflector doesn’t use ORFs, and a new Route Target is added to
the set, the route reflector, after changing its inbound route
filtering, must issue BGP Refresh to other route reflectors.
The delay of "a few hours" mentioned above allows a route
reflector to hold onto routes with a given RT, even after it
loses the last of its clients that are interested in such
routes. This protects against the need to reacquire all such
routes if the clients’ "disappearance" is only temporary.
With this procedure, VPN Join and Prune operations are also
non-disruptive.
Note that this technique will not work properly if some client
PE has a VRF with an import Route Target that is not one of its
export Route Targets.
In these procedures, a PE router which attaches to a particular VPN
"auto-discovers" the other PEs that attach to the same VPN. When a
new PE router is added, or when an existing PE router attaches to a
new VPN, no reconfiguration of other PE routers is needed.
Just as there is no one PE router that needs to know all the VPN-IPv4
routes supported over the backbone, these distribution rules ensure
that there is no one Route Reflector (RR) that needs to know all the
VPN-IPv4 routes supported over the backbone. As a result, the total
number of such routes that can be supported over the backbone is not
bounded by the capacity of any single device, and therefore can
increase virtually without bound.
4.3.4. How VPN-IPv4 NLRI Is Carried in BGP
The BGP Multiprotocol Extensions [BGP-MP] are used to encode the
NLRI. If the Address Family Identifier (AFI) field is set to 1, and
the Subsequent Address Family Identifier (SAFI) field is set to 128,
the NLRI is an MPLS-labeled VPN-IPv4 address. AFI 1 is used since
the network layer protocol associated with the NLRI is still IP.
Note that this VPN architecture does not require the capability to
distribute unlabeled VPN-IPv4 addresses.
In order for two BGP speakers to exchange labeled VPN-IPv4 NLRI, they
must use BGP Capabilities Advertisement to ensure that they both are
capable of properly processing such NLRI. This is done as specified
in [BGP-MP], by using capability code 1 (multiprotocol BGP), with an
AFI of 1 and an SAFI of 128.
The labeled VPN-IPv4 NLRI itself is encoded as specified in
[MPLS-BGP], where the prefix consists of an 8-byte RD followed by an
IPv4 prefix.
4.3.5. Building VPNs Using Route Targets
By setting up the Import Targets and Export Targets properly, one can
construct different kinds of VPNs.
Suppose it is desired to create a fully meshed closed user group,
i.e., a set of sites where each can send traffic directly to the
other, but traffic cannot be sent to or received from other sites.
Then each site is associated with a VRF, a single Route Target
attribute is chosen, that Route Target is assigned to each VRF as
both the Import Target and the Export Target, and that Route Target
is not assigned to any other VRFs as either the Import Target or the
Export Target.
Alternatively, suppose one desired, for whatever reason, to create a
"hub and spoke" kind of VPN. This could be done by the use of two
Route Target values, one meaning "Hub" and one meaning "Spoke". At
the VRFs attached to the hub sites, "Hub" is the Export Target and
"Spoke" is the Import Target. At the VRFs attached to the spoke
site, "Hub" is the Import Target and "Spoke" is the Export Target.
Thus, the methods for controlling the distribution of routing
information among various sets of sites are very flexible, which in
turn provides great flexibility in constructing VPNs.
4.3.6. Route Distribution Among VRFs in a Single PE
It is possible to distribute routes from one VRF to another, even if
both VRFs are in the same PE, even though in this case one cannot say
that the route has been distributed by BGP. Nevertheless, the
decision to distribute a particular route from one VRF to another
within a single PE is the same decision that would be made if the
VRFs were on different PEs. That is, it depends on the Route Target
attribute that is assigned to the route (or would be assigned if the
route were distributed by BGP), and the import target of the second
VRF.
5. Forwarding
If the intermediate routers in the backbone do not have any
information about the routes to the VPNs, how are packets forwarded
from one VPN site to another?
When a PE receives an IP packet from a CE device, it chooses a
particular VRF in which to look up the packet’s destination address.
This choice is based on the packet’s ingress attachment circuit.
Assume that a match is found. As a result we learn the packet’s
"next hop".
If the packet’s next hop is reached directly over a VRF attachment
circuit from this PE (i.e., the packet’s egress attachment circuit is
on the same PE as its ingress attachment circuit), then the packet is
sent on the egress attachment circuit, and no MPLS labels are pushed
onto the packet’s label stack.
If the ingress and egress attachment circuits are on the same PE, but
are associated with different VRFs, and if the route that best
matches the destination address in the ingress attachment circuit’s
VRF is an aggregate of several routes in the egress attachment
circuit’s VRF, it may be necessary to look up the packet’s
destination address in the egress VRF as well.
If the packet’s next hop is NOT reached through a VRF attachment
circuit, then the packet must travel at least one hop through the
backbone. The packet thus has a "BGP Next Hop", and the BGP Next Hop
will have assigned an MPLS label for the route that best matches the
packet’s destination address. Call this label the "VPN route label".
The IP packet is turned into an MPLS packet with the VPN route label
as the sole label on the label stack.
The packet must then be tunneled to the BGP Next Hop.
If the backbone supports MPLS, this is done as follows:
- The PE routers (and any Autonomous System border routers) that
redistribute VPN-IPv4 addresses need to insert /32 address
prefixes for themselves into the IGP routing tables of the
backbone. This enables MPLS, at each node in the backbone
network, to assign a label corresponding to the route to each PE
router. To ensure interoperability among different
implementations, it is required to support LDP for setting up the
label switched paths across the backbone. However, other methods
of setting up these label switched paths are also possible.
(Some of these other methods may not require the presence of the
/32 address prefixes in the IGP.)
- If there are any traffic engineering tunnels to the BGP next hop,
and if one or more of those is available for use by the packet in
question, one of these tunnels is chosen. This tunnel will be
associated with an MPLS label, the "tunnel label". The tunnel
label gets pushed on the MPLS label stack, and the packet is
forwarded to the tunnel’s next hop.
- Otherwise,
* The packet will have an "IGP Next Hop", which is the next hop
along the IGP route to the BGP Next Hop.
* If the BGP Next Hop and the IGP Next Hop are the same, and if
penultimate hop popping is used, the packet is then sent to
the IGP Next Hop, carrying only the VPN route label.
* Otherwise, the IGP Next Hop will have assigned a label for
the route that best matches the address of the BGP Next Hop.
Call this the "tunnel label". The tunnel label gets pushed
on as the packet’s top label. The packet is then forwarded
to the IGP Next Hop.
- MPLS will then carry the packet across the backbone to the BGP
Next Hop, where the VPN label will be examined.
If the backbone does not support MPLS, the MPLS packet carrying only
the VPN route label may be tunneled to the BGP Next Hop using the
techniques of [MPLS-in-IP-GRE]. When the packet emerges from the
tunnel, it will be at the BGP Next Hop, where the VPN route label
will be examined.
At the BGP Next Hop, the treatment of the packet depends on the VPN
route label (see Section 4.3.2). In many cases, the PE will be able
to determine, from this label, the attachment circuit over which the
packet should be transmitted (to a CE device), as well as the proper
data link layer header for that interface. In other cases, the PE
may only be able to determine that the packet’s destination address
needs to be looked up in a particular VRF before being forwarded to a
CE device. There are also intermediate cases in which the VPN route
label may determine the packet’s egress attachment circuit, but a
lookup (e.g., ARP) still needs to be done in order to determine the
packet’s data link header on that attachment circuit.
Information in the MPLS header itself, and/or information associated
with the label, may also be used to provide QoS on the interface to
the CE.
In any event, if the packet was an unlabeled IP packet when it
arrived at its ingress PE, it will again be an unlabeled packet when
it leaves its egress PE.
The fact that packets with VPN route labels are tunneled through the
backbone is what makes it possible to keep all the VPN routes out of
the P routers. This is crucial to ensuring the scalability of the
scheme. The backbone does not even need to have routes to the CEs,
only to the PEs.
With respect to the tunnels, it is worth noting that this
specification:
- DOES NOT require that the tunnels be point-to-point; multipoint-
to-point can be used;
- DOES NOT require that there be any explicit setup of the tunnels,
either via signaling or via manual configuration;
- DOES NOT require that there be any tunnel-specific signaling;
- DOES NOT require that there be any tunnel-specific state in the P
or PE routers, beyond what is necessary to maintain the routing
information and (if used) the MPLS label information.
Of course, this specification is compatible with the use of point-
to-point tunnels that must be explicitly configured and/or signaled,
and in some situations there may be reasons for using such tunnels.
The considerations that are relevant to choosing a particular
tunneling technology are outside the scope of this specification.
6. Maintaining Proper Isolation of VPNs
To maintain proper isolation of one VPN from another, it is important
that no router in the backbone accept a tunneled packet from outside
the backbone, unless it is sure that both endpoints of that tunnel
are outside the backbone.
If MPLS is being used as the tunneling technology, this means that a
router in the backbone MUST NOT accept a labeled packet from any
adjacent non-backbone device unless the following two conditions
hold:
1. the label at the top of the label stack was actually
distributed by that backbone router to that non-backbone
device, and
2. the backbone router can determine that use of that label will
cause the packet to leave the backbone before any labels lower
in the stack will be inspected, and before the IP header will
be inspected.
The first condition ensure that any labeled packets received from
non-backbone routers have a legitimate and properly assigned label at
the top of the label stack. The second condition ensures that the
backbone routers will never look below that top label. Of course,
the simplest way to meet these two conditions is just to have the
backbone devices refuse to accept labeled packets from non-backbone
devices.
If MPLS is not being used as the tunneling technology, then filtering
must be done to ensure that an MPLS-in-IP or MPLS-in-GRE packet can
be accepted into the backbone only if the packet’s IP destination
address will cause it to be sent outside the backbone.
7. How PEs Learn Routes from CEs
The PE routers that attach to a particular VPN need to know, for each
attachment circuit leading to that VPN, which of the VPN’s addresses
should be reached over that attachment circuit.
The PE translates these addresses into VPN-IPv4 addresses, using a
configured RD. The PE then treats these VPN-IPv4 routes as input to
BGP. Routes from a VPN site are NOT leaked into the backbone’s IGP.
Exactly which PE/CE route distribution techniques are possible
depends on whether or not a particular CE is in a "transit VPN". A
"transit VPN" is one that contains a router that receives routes from
a "third party" (i.e., from a router that is not in the VPN, but is
not a PE router) and that redistributes those routes to a PE router.
A VPN that is not a transit VPN is a "stub VPN". The vast majority
of VPNs, including just about all corporate enterprise networks,
would be expected to be "stubs" in this sense.
The possible PE/CE distribution techniques are:
1. Static routing (i.e., configuration) may be used. (This is
likely to be useful only in stub VPNs.)
2. PE and CE routers may be Routing Information Protocol (RIP)
[RIP] peers, and the CE may use RIP to tell the PE router the
set of address prefixes that are reachable at the CE router’s
site. When RIP is configured in the CE, care must be taken to
ensure that address prefixes from other sites (i.e., address
prefixes learned by the CE router from the PE router) are never
advertised to the PE. More precisely: if a PE router, say,
PE1, receives a VPN-IPv4 route R1, and as a result distributes
an IPv4 route R2 to a CE, then R2 must not be distributed back
from that CE’s site to a PE router, say, PE2, (where PE1 and
PE2 may be the same router or different routers), unless PE2
maps R2 to a VPN-IPv4 route that is different than (i.e.,
contains a different RD than) R1.
3. The PE and CE routers may be OSPF peers. A PE router that is
an OSPF peer of a CE router appears, to the CE router, to be an
area 0 router. If a PE router is an OSPF peer of CE routers
that are in distinct VPNs, the PE must of course be running
multiple instances of OSPF.
IPv4 routes that the PE learns from the CE via OSPF are
redistributed into BGP as VPN-IPv4 routes. Extended Community
attributes are used to carry, along with the route, all the
information needed to enable the route to be distributed to
other CE routers in the VPN in the proper type of OSPF Link
State Advertisement (LSA). OSPF route tagging is used to
ensure that routes received from the MPLS/BGP backbone are not
sent back into the backbone.
Specification of the complete set of procedures for the use of
OSPF between PE and CE can be found in [VPN-OSPF] and
[OSPF-2547-DNBIT].
4. The PE and CE routers may be BGP peers, and the CE router may
use BGP (in particular, EBGP to tell the PE router the set of
address prefixes that are at the CE router’s site. (This
technique can be used in stub VPNs or transit VPNs.)
This technique has a number of advantages over the others:
a) Unlike the IGP alternatives, this does not require the PE
to run multiple routing algorithm instances in order to
talk to multiple CEs.
b) BGP is explicitly designed for just this function:
passing routing information between systems run by
different administrations.
c) If the site contains "BGP backdoors", i.e., routers with
BGP connections to routers other than PE routers, this
procedure will work correctly in all circumstances. The
other procedures may or may not work, depending on the
precise circumstances.
d) Use of BGP makes it easy for the CE to pass attributes of
the routes to the PE. A complete specification of the
set of attributes and their use is outside the scope of
this document. However, some examples of the way this
may be used are the following:
- The CE may suggest a particular Route Target for each
route, from among the Route Targets that the PE is
authorized to attach to the route. The PE would then
attach only the suggested Route Target, rather than
the full set. This gives the CE administrator some
dynamic control of the distribution of routes from
the CE.
- Additional types of Extended Community attributes may
be defined, where the intention is to have those
attributes passed transparently (i.e., without being
changed by the PE routers) from CE to CE. This would
allow CE administrators to implement additional route