database and sees that it has a 10kbps Marked allocation to NEARNet
(the next region in that direction) that is being unused. The policy
is that ESNet-BB must always inform ("ask") NEARNet-BB when it is
about to use part of its allocation. NEARNET-BB authenticates the
message, checks its database and discovers that 20kbps of the
allocation to MIT is unused and the policy at that boundary is to not
inform MIT when part of the allocation is about to be used ("<50 ok"
where the total allocation is 50). The dotted lines indicate the
"implied" transaction, that is the transaction that would have
happened if the policy hadn't said "don't ask me". Now each BB can
pass an "ok" message to this request across its boundary. This allows
V to send to D, but not vice versa. It would also be possible for the
request to originate from D.
Figure 7. End-to-end example with static allocation.
Consider the same example where the ESNet-BB finds all of its Marked
allocation to NEARNet, 10 kbps, in use. With static allocations,
ESNet must transmit a "no" to this request back to the LBL-BB.
Presumably, the LBL-BB would record this information to complain to
ESNet about the overbooking at the end of the month! One solution to
this sort of "busy signal" is for ESNet to get better at anticipating
its customers needs or require long advance bookings for every flow,
but it's also possible for bandwidth brokerage decisions to become
dynamic.
Figure 8. End-to-end static allocation example with no remaining
allocation
Dynamic Allocation and additional mechanism: As we shall see, dynamic
allocation requires more complex BBs as well as more complex border
policing, including the necessity to keep more state. However, it
enables an important service with a small increase in state.
The next set of figures (starting with figure 9) show what happens in
the case of dynamic allocation. As before, V requests 10kbps to talk
to D at MIT. Since the allocation is dynamic, the border policers do
not have a preset value, instead being set to reflect the current
peak value of Marked traffic permitted to cross that boundary. The
request is sent to the LBL-BB.
Figure 9. First step in end-to-end dynamic allocation example.
In figure 10, note that ESNet has no allocation set up to NEARNet.
This system is capable of dynamic allocations in addition to static,
so it asks NEARNet if it can "add 10" to its allocation from ESNet.
As in the figure 7 example, MIT's policy is set to "don't ask" for
this case, so the dotted lines represent "implicit transactions"
where no messages were exchanged. However, NEARNet does update its
table to indicate that it is now using 20kbps of the Marked
allocation to MIT.
Figure 10. Second step in end-to-end dynamic allocation example
In figure 11, we see the third step where MIT's "virtual ok" allows
the NEARNet-BB to tell its border router to increase the Marked
allocation across the ESNet-NEARNet boundary by 10 kbps.
Figure 11. Third step in end-to-end dynamic allocation example
Figure 11 shows NEARNet-BB's "ok" for that request transmitted back
to ESNet-BB. This causes ESNet-BB to send its border router a message
to create a 10 kbps subclass for the flow "V->D". This is required in
order to ensure that the 10kpbs that has just been dynamically
allocated gets used only for that connection. Note that this does
require that the per flow state be passed from LBL-BB to ESNet-BB,
but this is the only boundary that needs that level of flow
information and this further classification will only need to be done
at that one boundary router and only on packets coming from LBL. Thus
dynamic allocation requires more complex Profile Metering than that
shown in figure 5.
Figure 12. Fourth step in end-to-end dynamic allocation example.
In figure 12, the ESNet border router gives the "ok" that a subclass
has been created, causing the ESNet-BB to send an "ok" to the LBL-BB
which lets V know the request has been approved.
Figure 13. Final step in end-to-end dynamic allocation example
For dynamic allocation, a basic version of a CBQ scheduler [5] would
have all the required functionality to set up the subclasses. RSVP
currently provides a way to move the TSpec for the flow.
For multicast flows, we assume that packets that are bound for at
least one egress can be carried through a domain at that level of
service to all egress points. If a particular multicast branch has
been subscribed to at best-effort when upstream branches are Marked,
it will have its bit settings cleared before it crosses the boundary.
The information required for this flow identification is used to
augment the existing state that is already kept on this flow because
it is a multicast flow. We note that we are already "catching" this
flow, but now we must potentially clear the bit-pattern.
5. RSVP/int-serv and this architecture
Much work has been done in recent years on the definition of related
integrated services for the internet and the specification of the
RSVP signalling protocol. The two-bit architecture proposed in this
work can easily interoperate with those specifications. In this
section we first discuss how the forwarding mechanisms described in
section 3 can be used to support integrated services. Second, we
discuss how RSVP could interoperate with the administrative structure
of the BBs to provide better scaling.
5.1 Providing Controlled-Load and Guaranteed Service
We believe that the forwarding path mechanisms described in section 3
are general enough that they can also be used to provide the
Controlled-Load service [8] and a version of the Guaranteed Quality
of Service [9], as developed by the int-serv WG. First note that
Premium service can be thought of as a constrained case of
Controlled-Load service where the burst size is limited to one packet
and where non-conforming packets are dropped. A network element that
has implemented the mechanisms to support premium service can easily
support the more general controlled-load service by making one or
more minor parameter adjustments, e.g. by lifting the constraint on
the token bucket size, or configuring the Premium service rate with
the peak traffic rate parameter in the Controlled-Load specification,
and by changing the policing action on out-of-profile packets from
dropping to sending the packets to the Best-effort queue.
It is also possible to implement Guaranteed Quality of Service using
the mechanisms of Premium service. From RFC2212 [9]: "The definition
of guaranteed service relies on the result that the fluid delay of a
flow obeying a token bucket (r, b) and being served by a line with
bandwidth R is bounded by b/R as long as R is no less than r.
Guaranteed service with a service rate R, where now R is a share of
bandwidth rather than the bandwidth of a dedicated line approximates
this behavior." The service model of Premium clearly fits this model.
RFC2212 states that "Non-conforming datagrams SHOULD be treated as
best-effort datagrams." Thus, a policing Profile Meter that drops
non-conforming datagrams would be acceptable, but it's also possible
to change the action for non-compliant packets from a drop to sending
to the best-effort queue.
5.2 RSVP and BBs
In this section we discuss how RSVP signaling can be used in
conjunction with the BBs described in section 4 to deliver a more
scalable end-to-end resource set up for Integrated Services. First we
note that the BB architecture has three major differences with the
original RSVP resource set up model:
1. There exist apriori bilateral business relations between BBs of
adjacent trust regions before one can set up end-to-end resource
allocation; real-time signaling is used only to activate/confirm the
availability of pre-negotiated Marked bandwidth, and to dynamically
readjust the allocation amount when necessary. We note that this
real-time signaling across domains is not required, but depends on
the nature of the bilateral agreement (e.g., the agreement might
state "I'll tell you whenever I'm going to use some of my allocation"
or not).
2. A few bits in the packet header, i.e. the P-bit and A-bit, are
used to mark the service class of each packet, therefore a full
packet classification (by checking all relevant fields in the header)
need be done only once at the leaf router; after that packets will be
served according to their class bit settings.
3. RSVP resource set up assumes that resources will be reserved hop-
by-hop at each router along the entire end-to-end path.
RSVP messages sent to leaf routers by hosts can be intercepted and
sent to the local domain's BB. The BB processes the message and, if
the request is approved, forwards a message to the leaf router that
sets up appropriate per-flow packet classification. A message should
also be sent to the egress border router to add to the aggregate
Marked traffic allocation for packet shaping by the Profile Meter on
outbound traffic. (Its possible that this is always set to the full
allocation.) An RSVP message must be sent across the boundary to
adjacent ISP's border router, either from the local domain's border
router or from the local domain's BB. If the ISP is also implementing
the RSVP with a BB and diff-serv framework, its border router
forwards the message to the ISP's local BB. A similar process (to
what happened in the first domain) can be carried out in the ISP
domain, then an RSVP message gets forwarded to the next ISP along the
path. Inside a domain, packets are served solely according to the
Marked bits. The local BB knows exactly how much Premium traffic is
permitted to enter at each border router and from which border router
packets exit.
6. Recommendations
This document has presented a reference architecture for
differentiated services. Several variations can be envisioned,
particularly for early and partial deployments, but we do not
enumerate all of these variations here. There has been a great market
demand for differentiated services lately. As one of the many efforts
to meet that demand this memo sketches out the framework of a
flexible architecture for offering differential services, and in
particular defines a simple set of packet forwarding path mechanisms
to support two basic types of differential services. Although there
remain a number of issues and parameters that need further
exploration and refinement, we believe it is both possible and
feasible at this time to start deployment of differentiated services
incrementally. First, given that the basic mechanisms required in the
packet forwarding path are clearly understood, both Assured and
Premium services can be implemented today with manually configured
BBs and static resource allocation. Initially we recommend
conservative choices on the amount of Marked traffic that is admitted
into the network. Second, we plan to continue the effort started with
this memo and the experimental work of the authors to define and
deploy increasingly sophisticated BBs. We hope to turn the experience
gained from in-progress trial implementations on ESNet and CAIRN into
future proposals to the IETF.
Future revisions of this memo will present the receiver-based and
multicast flow allocations in detail. After this step is finished,
we believe the basic picture of an scalable, robust, secure resource
management and allocation system will be completed. In this memo, we
described how the proposed architecture supports two services that
seem to us to provide at least a good starting point for trial
deployment of differentiated services. Our main intent is to define
an architecture with three services, Premium, Assured, and Best
effort, that can be determined by specific bit- patterns, but not to
preclude additional levels of differentiation within each service. It
seems that more experimentation and experience is required before we
could standardize more than one level per service class. Our base-
level approach says that everyone has to provide "at least" Premium
service and Assured service as documented. We feel rather strongly
about both 1) that we should not try to define, at this time,
something beyond the minimalist two service approach and 2) that the
architecture we define must be open-ended so that more levels of
differentiation might be standardized in the future. We believe this
architecture is completely compatible with approaches that would
define more levels of differentiation within a particular service, if
the benefits of doing so become well understood.
7. Acknowledgments
The authors have benefited from many discussions, both in person and
electronically and wish to particularly thank Dave Clark who has been
responsible for the genesis of many of the ideas presented here,
though he does not agree with all of the content this document. We
also thank Sally Floyd for comments on an earlier draft. A comment
from Jon Crowcroft was partially responsible for our including
section 5. Comments from Fred Baker made us try to make it clearer
that we are defining two base-level services, irrespective of the bit
patterns used to encode them.
8. Security Considerations
There are no security considerations associated with this document.
9. References
[1] D. Clark, "Adding Service Discrimination to the Internet",
Proceedings of the 23rd Annual Telecommunications Policy Research
Conference (TPRC), Solomons, MD, October 1995.
[2] V. Jacobson, "Differentiated Services Architecture", talk in the
Int-Serv WG at the Munich IETF, August, 1997.
[3] Clark, D. and J. Wroclawski, "An Approach to Service Allocation
in the Internet", Work in Progress, also talk by D. Clark in the
Int-Serv WG at the Munich IETF, August, 1997.
[4] Braden, et al., "Recommendations on Queue Management and
Congestion Avoidance in the Internet", RFC2309, April 1998.
[4] Braden, R., Zhang, L., Berson, S., Herzog, S. and S. Jamin,
"Resource Reservation Protocol (RSVP) - Version 1 Functional
Specification", RFC2205, September 1997.
[5] S. Floyd and V. Jacobson, "Link-sharing and Resource Management
Models for Packet Networks", IEEE/ACM Transactions on Networking,
pp 365-386, August 1995.
[6] D. Clark, private communication, October 26, 1997.
[7] "Advanced QoS Services for the Intelligent Internet", Cisco
Systems White Paper, 1997.
[8] Wroclawski, J., "Specification of the Controlled-Load Network
Element Service", RFC2211, September 1997.
[9] Shenker, S., Partirdge, C. and R. Guerin, "Specification of
Guaranteed Quality of Service", RFC2212, September 1997.
[10] D. Clark and W. Fang, "Explicit Allocation of Best Effort packet
Delivery Service", IEEE/ACM Transactions on Networking, August,
1998, Vol6, No 4, pp. 362-373. also at: http://
diffserv.lcs.mit.edu/Papers/exp-alloc-ddc-wf.pdf
Authors' Addresses
Kathleen Nichols
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
Phone: 408-525-4857
EMail: kmn@cisco.com
Van Jacobson
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
EMail: van@cisco.com
Lixia Zhang
UCLA
4531G Boelter Hall
Los Angeles, CA 90095
Phone: 310-825-2695
EMail: lixia@cs.ucla.edu
Appendix: A Combined Approach to Differential Service in the Internet by
David D. Clark
After the draft-nichols-diff-svc-00 was submitted, the co-authors had
a discussion with Dave Clark and John Wroclawski which resulted in
Clark's using the presentation slot for the draft at the December
1997 IETF Integrated Services Working Group meeting. A reading of the
slides shows that it was Clark's proposal on "mechanisms",
"services", and "rules" and how to proceed in the standards process
that has guided much of the process in the subsequently formed IETF
Differentiated Services Working Group. We believe Dave Clark's talk
gave us a solid approach for bringing quality of service to the
Internet in a manner that is compatible with its strengths.
The slides presented at the December 1997 IETF Integrated Services
Working Group are included with the Postscript version.
Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFCEditor function is currently provided by the
Internet Society.