together with the Data Link Connection Identifier (DLCI) field in the
packet’s Frame Relay header.
Although the PE’s conclusion that a particular packet arrived on a
particular attachment circuit may be partially determined by the
packet’s layer 2 header, it must be impossible for a customer, by
writing the header fields, to fool the SP into thinking that a packet
that was received over one attachment circuit really arrived over a
different one. In the example above, although the attachment circuit
is determined partially by inspection of the DLCI field in the Frame
Relay header, this field cannot be set freely by the customer.
Rather, it must be set to a value specified by the SP, or else the
packet cannot arrive at the PE router.
In some cases, a particular site may be divided by the customer into
several "virtual sites". The SP may designate a particular set of
VRFs to be used for routing packets from that site and may allow the
customer to set some characteristic of the packet, which is then used
for choosing a particular VRF from the set.
For example, each virtual site might be realized as a VLAN. The SP
and the customer could agree that on packets arriving from a
particular CE, certain VLAN values would be used to identify certain
VRFs. Of course, packets from that CE would be discarded by the PE
if they carry VLAN tag values that are not in the agreed-upon set.
Another way to accomplish this is to use IP source addresses. In
this case, the PE uses the IP source address in a packet received
from the CE, along with the interface over which the packet is
received, to assign the packet to a particular VRF. Again, the
customer would only be able to select from among the particular set
of VRFs that that customer is allowed to use.
If it is desired to have a particular host be in multiple virtual
sites, then that host must determine, for each packet, which virtual
site the packet is associated with. It can do this, e.g., by sending
packets from different virtual sites on different VLANs, or out
different network interfaces.
3.3. Populating the VRFs
With what set of routes are the VRFs populated?
As an example, let PE1, PE2, and PE3 be three PE routers, and let
CE1, CE2, and CE3 be three CE routers. Suppose that PE1 learns, from
CE1, the routes that are reachable at CE1’s site. If PE2 and PE3 are
attached, respectively, to CE2 and CE3, and there is some VPN V
containing CE1, CE2, and CE3, then PE1 uses BGP to distribute to PE2
and PE3 the routes that it has learned from CE1. PE2 and PE3 use
these routes to populate the VRFs that they associate, respectively,
with the sites of CE2 and CE3. Routes from sites that are not in VPN
V do not appear in these VRFs, which means that packets from CE2 or
CE3 cannot be sent to sites that are not in VPN V.
When we speak of a PE "learning" routes from a CE, we are not
presupposing any particular learning technique. The PE may learn
routes by means of a dynamic routing algorithm, but it may also
"learn" routes by having those routes configured (i.e., static
routing). (In this case, to say that the PE "learned" the routes
from the CE is perhaps to exercise a bit of poetic license.)
PEs also need to learn, from other PEs, the routes that belong to a
given VPN. The procedures to be used for populating the VRFs with
the proper sets of routes are specified in Section 4.
If there are multiple attachment circuits leading from a particular
PE router to a particular site, they might all be mapped to the same
forwarding table. But if policy dictates, they could be mapped to
different forwarding tables. For instance, the policy might be that
a particular attachment circuit from a site is used only for intranet
traffic, while another attachment circuit from that site is used only
for extranet traffic. (Perhaps, e.g., the CE attached to the
extranet attachment circuit is a firewall, while the CE attached to
the intranet attachment circuit is not.) In this case, the two
attachment circuits would be associated with different VRFs.
Note that if two attachment circuits are associated with the same
VRF, then packets that the PE receives over one of them will be able
to reach exactly the same set of destinations as packets that the PE
receives over the other. So two attachment circuits cannot be
associated with the same VRF unless each CE is in the exact same set
of VPNs as is the other.
If an attachment circuit leads to a site which is in multiple VPNs,
the attachment circuit may still associated with a single VRF, in
which case the VRF will contain routes from the full set of VPNs of
which the site is a member.
4. VPN Route Distribution via BGP
PE routers use BGP to distribute VPN routes to each other (more
accurately, to cause VPN routes to be distributed to each other).
We allow each VPN to have its own address space, which means that a
given address may denote different systems in different VPNs. If two
routes to the same IP address prefix are actually routes to different
systems, it is important to ensure that BGP not treat them as
comparable. Otherwise, BGP might choose to install only one of them,
making the other system unreachable. Further, we must ensure that
POLICY is used to determine which packets get sent on which routes;
given that several such routes are installed by BGP, only one such
must appear in any particular VRF.
We meet these goals by the use of a new address family, as specified
below.
4.1. The VPN-IPv4 Address Family
The BGP Multiprotocol Extensions [BGP-MP] allow BGP to carry routes
from multiple "address families". We introduce the notion of the
"VPN-IPv4 address family". A VPN-IPv4 address is a 12-byte quantity,
beginning with an 8-byte Route Distinguisher (RD) and ending with a
4-byte IPv4 address. If several VPNs use the same IPv4 address
prefix, the PEs translate these into unique VPN-IPv4 address
prefixes. This ensures that if the same address is used in several
different VPNs, it is possible for BGP to carry several completely
different routes to that address, one for each VPN.
Since VPN-IPv4 addresses and IPv4 addresses are different address
families, BGP never treats them as comparable addresses.
An RD is simply a number, and it does not contain any inherent
information; it does not identify the origin of the route or the set
of VPNs to which the route is to be distributed. The purpose of the
RD is solely to allow one to create distinct routes to a common IPv4
address prefix. Other means are used to determine where to
redistribute the route (see Section 4.3).
The RD can also be used to create multiple different routes to the
very same system. We have already discussed a situation in which the
route to a particular server should be different for intranet traffic
than for extranet traffic. This can be achieved by creating two
different VPN-IPv4 routes that have the same IPv4 part, but different
RDs. This allows BGP to install multiple different routes to the
same system, and allows policy to be used (see Section 4.3.5) to
decide which packets use which route.
The RDs are structured so that every Service Provider can administer
its own "numbering space" (i.e., can make its own assignments of
RDs), without conflicting with the RD assignments made by any other
Service Provider. An RD consists of three fields: a 2-byte type
field, an administrator field, and an assigned number field. The
value of the type field determines the lengths of the other two
fields, as well as the semantics of the administrator field. The
administrator field identifies an assigned number authority, and the
assigned number field contains a number that has been assigned, by
the identified authority, for a particular purpose. For example, one
could have an RD whose administrator field contains an Autonomous
System number (ASN), and whose (4-byte) number field contains a
number assigned by the SP to whom that ASN belongs (having been
assigned to that SP by the appropriate authority).
RDs are given this structure in order to ensure that an SP that
provides VPN backbone service can always create a unique RD when it
needs to do so. However, the structure is not meaningful to BGP;
when BGP compares two such address prefixes, it ignores the structure
entirely.
A PE needs to be configured such that routes that lead to a
particular CE become associated with a particular RD. The
configuration may cause all routes leading to the same CE to be
associated with the same RD, or it may cause different routes to be
associated with different RDs, even if they lead to the same CE.
4.2. Encoding of Route Distinguishers
As stated, a VPN-IPv4 address consists of an 8-byte Route
Distinguisher followed by a 4-byte IPv4 address. The RDs are encoded
as follows:
- Type Field: 2 bytes
- Value Field: 6 bytes
The interpretation of the Value field depends on the value of the
type field. At the present time, three values of the type field are
defined: 0, 1, and 2.
- Type 0: The Value field consists of two subfields:
* Administrator subfield: 2 bytes
* Assigned Number subfield: 4 bytes
The Administrator subfield must contain an Autonomous System
number. If this ASN is from the public ASN space, it must have
been assigned by the appropriate authority (use of ASN values
from the private ASN space is strongly discouraged). The
Assigned Number subfield contains a number from a numbering space
that is administered by the enterprise to which the ASN has been
assigned by an appropriate authority.
- Type 1: The Value field consists of two subfields:
* Administrator subfield: 4 bytes
* Assigned Number subfield: 2 bytes
The Administrator subfield must contain an IP address. If this
IP address is from the public IP address space, it must have been
assigned by an appropriate authority (use of addresses from the
private IP address space is strongly discouraged). The Assigned
Number subfield contains a number from a numbering space which is
administered by the enterprise to which the IP address has been
assigned.
- Type 2: The Value field consists of two subfields:
* Administrator subfield: 4 bytes
* Assigned Number subfield: 2 bytes
The Administrator subfield must contain a 4-byte Autonomous
System number [BGP-AS4]. If this ASN is from the public ASN
space, it must have been assigned by the appropriate authority
(use of ASN values from the private ASN space is strongly
discouraged). The Assigned Number subfield contains a number
from a numbering space which is administered by the enterprise to
which the ASN has been assigned by an appropriate authority.
4.3. Controlling Route Distribution
In this section, we discuss the way in which the distribution of the
VPN-IPv4 routes is controlled.
If a PE router is attached to a particular VPN (by being attached to
a particular CE in that VPN), it learns some of that VPN’s IP routes
from the attached CE router. Routes learned from a CE routing peer
over a particular attachment circuit may be installed in the VRF
associated with that attachment circuit. Exactly which routes are
installed in this manner is determined by the way in which the PE
learns routes from the CE. In particular, when the PE and CE are
routing protocol peers, this is determined by the decision process of
the routing protocol; this is discussed in Section 7.
These routes are then converted to VPN-IP4 routes, and "exported" to
BGP. If there is more than one route to a particular VPN-IP4 address
prefix, BGP chooses the "best" one, using the BGP decision process.
That route is then distributed by BGP to the set of other PEs that
need to know about it. At these other PEs, BGP will again choose the
best route for a particular VPN-IP4 address prefix. Then the chosen
VPN-IP4 routes are converted back into IP routes, and "imported" into
one or more VRFs. Whether they are actually installed in the VRFs
depends on the decision process of the routing method used between
the PE and those CEs that are associated with the VRF in question.
Finally, any route installed in a VRF may be distributed to the
associated CE routers.
4.3.1. The Route Target Attribute
Every VRF is associated with one or more Route Target (RT)
attributes.
When a VPN-IPv4 route is created (from an IPv4 route that the PE has
learned from a CE) by a PE router, it is associated with one or more
Route Target attributes. These are carried in BGP as attributes of
the route.
Any route associated with Route Target T must be distributed to every
PE router that has a VRF associated with Route Target T. When such a
route is received by a PE router, it is eligible to be installed in
those of the PE’s VRFs that are associated with Route Target T.
(Whether it actually gets installed depends upon the outcome of the
BGP decision process, and upon the outcome of the decision process of
the IGP (i.e., the intra-domain routing protocol) running on the
PE/CE interface.)
A Route Target attribute can be thought of as identifying a set of
sites. (Though it would be more precise to think of it as
identifying a set of VRFs.) Associating a particular Route Target
attribute with a route allows that route to be placed in the VRFs
that are used for routing traffic that is received from the
corresponding sites.
There is a set of Route Targets that a PE router attaches to a route
received from site S; these may be called the "Export Targets". And
there is a set of Route Targets that a PE router uses to determine
whether a route received from another PE router could be placed in
the VRF associated with site S; these may be called the "Import
Targets". The two sets are distinct, and need not be the same. Note
that a particular VPN-IPv4 route is only eligible for installation in
a particular VRF if there is some Route Target that is both one of
the route’s Route Targets and one of the VRF’s Import Targets.
The function performed by the Route Target attribute is similar to
that performed by the BGP Communities attribute. However, the format
of the latter is inadequate for present purposes, since it allows
only a 2-byte numbering space. It is desirable to structure the
format, similar to what we have described for RDs (see Section 4.2),
so that a type field defines the length of an administrator field,
and the remainder of the attribute is a number from the specified
administrator’s numbering space. This can be done using BGP Extended
Communities. The Route Targets discussed herein are encoded as BGP
Extended Community Route Targets [BGP-EXTCOMM]. They are structured
similarly to the RDs.
When a BGP speaker has received more than one route to the same VPN-
IPv4 prefix, the BGP rules for route preference are used to choose
which VPN-IPv4 route is installed by BGP.
Note that a route can only have one RD, but it can have multiple
Route Targets. In BGP, scalability is improved if one has a single
route with multiple attributes, as opposed to multiple routes. One
could eliminate the Route Target attribute by creating more routes
(i.e., using more RDs), but the scaling properties would be less
favorable.
How does a PE determine which Route Target attributes to associate
with a given route? There are a number of different possible ways.
The PE might be configured to associate all routes that lead to a
specified site with a specified Route Target. Or the PE might be
configured to associate certain routes leading to a specified site
with one Route Target, and certain with another.
If the PE and the CE are themselves BGP peers (see Section 7), then
the SP may allow the customer, within limits, to specify how its
routes are to be distributed. The SP and the customer would need to
agree in advance on the set of RTs that are allowed to be attached to
the customer’s VPN routes. The CE could then attach one or more of
those RTs to each IP route that it distributes to the PE. This gives
the customer the freedom to specify in real time, within agreed-upon
limits, its route distribution policies. If the CE is allowed to
attach RTs to its routes, the PE MUST filter out all routes that
contain RTs that the customer is not allowed to use. If the CE is
not allowed to attach RTs to its routes, but does so anyway, the PE
MUST remove the RT before converting the customer’s route to a VPN-
IPv4 route.
4.3.2. Route Distribution Among PEs by BGP
If two sites of a VPN attach to PEs that are in the same Autonomous
System, the PEs can distribute VPN-IPv4 routes to each other by means
of an IBGP connection between them. (The term "IBGP" refers to the
set of protocols and procedures used when there is a BGP connection
between two BGP speakers in the same Autonomous System. This is
distinguished from "EBGP", the set of procedures used between two BGP
speakers in different Autonomous Systems.) Alternatively, each can
have an IBGP connection to a route reflector [BGP-RR].
When a PE router distributes a VPN-IPv4 route via BGP, it uses its
own address as the "BGP next hop". This address is encoded as a
VPN-IPv4 address with an RD of 0. ([BGP-MP] requires that the next
hop address be in the same address family as the Network Layer
Reachability Information (NLRI).) It also assigns and distributes an
MPLS label. (Essentially, PE routers distribute not VPN-IPv4 routes,
but Labeled VPN-IPv4 routes. Cf. [MPLS-BGP].) When the PE processes
a received packet that has this label at the top of the stack, the PE
will pop the stack, and process the packet appropriately.
The PE may distribute the exact set of routes that appears in the
VRF, or it may perform summarization and distribute aggregates of
those routes, or it may do some of one and some of the other.
Suppose that a PE has assigned label L to route R, and has
distributed this label mapping via BGP. If R is an aggregate of a
set of routes in the VRF, the PE will know that packets from the
backbone that arrive with this label must have their destination
addresses looked up in a VRF. When the PE looks up the label in its
Label Information Base, it learns which VRF must be used. On the
other hand, if R is not an aggregate, then when the PE looks up the
label, it learns the egress attachment circuit, as well as the
encapsulation header for the packet. In this case, no lookup in the
VRF is done.
We would expect that the most common case would be the case where the
route is NOT an aggregate. The case where it is an aggregate can be
very useful though if the VRF contains a large number of host routes
(e.g., as in dial-in), or if the VRF has an associated Local Area
Network (LAN) interface (where there is a different outgoing layer 2
header for each system on the LAN, but a route is not distributed for
each such system).
Whether or not each route has a distinct label is an implementation
matter. There are a number of possible algorithms one could use to
determine whether two routes get assigned the same label:
- One may choose to have a single label for an entire VRF, so that
a single label is shared by all the routes from that VRF. Then
when the egress PE receives a packet with that label, it must
look up the packet’s IP destination address in that VRF (the
packet’s "egress VRF"), in order to determine the packet’s egress
attachment circuit and the corresponding data link encapsulation.
- One may choose to have a single label for each attachment
circuit, so that a single label is shared by all the routes with
the same "outgoing attachment circuit". This enables one to
avoid doing a lookup in the egress VRF, though some sort of
lookup may need to be done in order to determine the data link
encapsulation, e.g., an Address Resolution Protocol (ARP) lookup.
- One may choose to have a distinct label for each route. Then if
a route is potentially reachable over more than one attachment
circuit, the PE/CE routing can switch the preferred path for a
route from one attachment circuit to another, without there being
any need to distribute new a label for that route.
There may be other possible algorithms as well. The choice of
algorithm is entirely at the discretion of the egress PE, and is
otherwise transparent.
In using BGP-distributed MPLS labels in this manner, we presuppose
that an MPLS packet carrying such a label can be tunneled from the
router that installs the corresponding BGP-distributed route to the
router that is the BGP next hop of that route. This requires either
that a label switched path exist between those two routers or else
that some other tunneling technology (e.g., [MPLS-in-IP-GRE]) can be
used between them.
This tunnel may follow a "best effort" route, or it may follow a
traffic-engineered route. Between a given pair of routers, there may
be one such tunnel, or there may be several, perhaps with different
Quality of Service (QoS) characteristics. All that matters for the
VPN architecture is that some such tunnel exists. To ensure
interoperability among systems that implement this VPN architecture
using MPLS label switched paths as the tunneling technology, all such
systems MUST support Label Distribution Protocol (LDP) [MPLS-LDP].
In particular, Downstream Unsolicited mode MUST be supported on
interfaces that are neither Label Controlled ATM (LC-ATM) [MPLS-ATM]
nor Label Controlled Frame Relay (LC-FR) [MPLS-FR] interfaces, and
Downstream on Demand mode MUST be supported on LC-ATM interfaces and
LC-FR interfaces.
If the tunnel follows a best-effort route, then the PE finds the
route to the remote endpoint by looking up its IP address in the
default forwarding table.
A PE router, UNLESS it is a route reflector (see Section 4.3.3) or an
Autonomous System Border Router (ASBR) for an inter-provider VPN (see
Section 10), should not install a VPN-IPv4 route unless it has at
least one VRF with an Import Target identical to one of the route’s
Route Target attributes. Inbound filtering should be used to cause
such routes to be discarded. If a new Import Target is later added
to one of the PE’s VRFs (a "VPN Join" operation), it must then
acquire the routes it may previously have discarded. This can be
done using the refresh mechanism described in [BGP-RFSH]. The
outbound route filtering mechanism of [BGP-ORF] can also be used to
advantage to make the filtering more dynamic.
Similarly, if a particular Import Target is no longer present in any
of a PE’s VRFs (as a result of one or more "VPN Prune" operations),
the PE may discard all routes that, as a result, no longer have any
of the PE’s VRF’s Import Targets as one of their Route Target
attributes.
A router that is not attached to any VPN and that is not a Route
Reflector (i.e., a P router) never installs any VPN-IPv4 routes at
all.
Note that VPN Join and Prune operations are non-disruptive and do not
require any BGP connections to be brought down, as long as the
refresh mechanism of [BGP-RFSH] is used.
As a result of these distribution rules, no one PE ever needs to
maintain all routes for all VPNs; this is an important scalability