used when the overall traffic level is unusually high). If a task is
instructed to use rule set 0, it will cease measuring; all packets
will be ignored until another (non-zero) rule set is made current.
Each rule in a rule set is an instruction for the Packet Matching
Engine, i.e. it is an instruction for a Virtual Machine. PME
instructions have five component fields, forming two logical groups
as follows:
+-------- test ---------+ +---- action -----+
attribute & mask = value: opcode, parameter;
The test group allows PME to test the value of an attribute. This is
done by ANDing the attribute value with the mask and comparing the
result with the value field. Note that there is no explicit
provision to test a range, although this can be done where the range
can be covered by a mask, e.g. attribute value less than 2048.
The PME maintains a Boolean indicator called the 'test indicator',
which determines whether or not a rule's test is performed. The test
indicator is initially set (true).
The action group specifies what action may be performed when the rule
is executed. Opcodes contain two flags: 'goto' and 'test', as
detailed in the table below. Execution begins with rule 1, the first
in the rule set. It proceeds as follows:
If the test indicator is true:
Perform the test, i.e. AND the attribute value with the
mask and compare it with the value.
If these are equal the test has succeeded; perform the
rule's action (below).
If the test fails execute the next rule in the rule set.
If there are no more rules in the rule set, return from the
match() function indicating NoMatch.
If the test indicator is false, or the test (above) succeeded:
Set the test indicator to this opcode's test flag value.
Determine the next rule to execute.
If the opcode has its goto flag set, its parameter value
specifies the number of the next rule.
Opcodes which don't have their goto flags set either
determine the next rule in special ways (Return),
or they terminate execution (Ignore, NoMatch, Count,
CountPkt).
Perform the action.
The PME maintains two 'history' data structures. The first, the
'return' stack, simply records the index (i.e. 1-origin rule number)
of each Gosub rule as it is executed; Return rules pop their Gosub
rule index. Note that when the Ignore, NoMatch, Count and CountPkt
actions are performed, PME execution is terminated regardless of
whether the PME is executing a subroutine ('return' stack is non-
empty) or not.
The second data structure, the 'pattern' queue, is used to save
information for later use in building a flow key. A flow key is
built by zeroing all its attribute values, then copying attribute
number, mask and value information from the pattern queue in the
order it was enqueued.
An attribute number identifies the attribute actually used in a test.
It will usually be the rule's attribute field, unless the attribute
is a 'meter variable'. Details of meter variables are given after
the table of opcode actions below.
The opcodes are:
opcode goto test
1 Ignore 0 -
2 NoMatch 0 -
3 Count 0 -
4 CountPkt 0 -
5 Return 0 0
6 Gosub 1 1
7 GosubAct 1 0
8 Assign 1 1
9 AssignAct 1 0
10 Goto 1 1
11 GotoAct 1 0
12 PushRuleTo 1 1
13 PushRuleToAct 1 0
14 PushPktTo 1 1
15 PushPktToAct 1 0
16 PopTo 1 1
17 PopToAct 1 0
The actions they perform are:
Ignore: Stop matching, return from the match() function
indicating that the packet is to be ignored.
NoMatch: Stop matching, return from the match() function
indicating failure.
Count: Stop matching. Save this rule's attribute number,
mask and value in the PME's pattern queue, then
construct a flow key for the flow to which this
packet belongs. Return from the match() function
indicating success. The meter will use the flow
key to search for the flow record for this
packet's flow.
CountPkt: As for Count, except that the masked value from
the packet header (as it would have been used in
the rule's test) is saved in the PME's pattern
queue instead of the rule's value.
Gosub: Call a rule-matching subroutine. Push the current
rule number on the PME's return stack, set the
test indicator then goto the specified rule.
GosubAct: Same as Gosub, except that the test indicator is
cleared before going to the specified rule.
Return: Return from a rule-matching subroutine. Pop the
number of the calling gosub rule from the PME's
'return' stack and add this rule's parameter value
to it to determine the 'target' rule. Clear the
test indicator then goto the target rule.
A subroutine call appears in a rule set as a Gosub
rule followed by a small group of following rules.
Since a Return action clears the test flag, the
action of one of these 'following' rules will be
executed; this allows the subroutine to return a
result (in addition to any information it may save
in the PME's pattern queue).
Assign: Set the attribute specified in this rule to the
parameter value specified for this rule. Set the
test indicator then goto the specified rule.
AssignAct: Same as Assign, except that the test indicator
is cleared before going to the specified rule.
Goto: Set the test indicator then goto the
specified rule.
GotoAct: Clear the test indicator then goto the specified
rule.
PushRuleTo: Save this rule's attribute number, mask and value
in the PME's pattern queue. Set the test
indicator then goto the specified rule.
PushRuleToAct: Same as PushRuleTo, except that the test indicator
is cleared before going to the specified rule.
PushRuleTo actions may be used to save the value
and mask used in a test, or (if the test is not
performed) to save an arbitrary value and mask.
PushPktTo: Save this rule's attribute number, mask, and the
masked value from the packet header (as it would
have been used in the rule's test), in the PME's
pattern queue. Set the test indicator then goto
the specified rule.
PushPktToAct: Same as PushPktTo, except that the test indicator
is cleared before going to the specified rule.
PushPktTo actions may be used to save a value from
the packet header using a specified mask. The
simplest way to program this is to use a zero value
for the PushPktTo rule's value field, and to
GoToAct to the PushPktTo rule (so that it's test is
not executed).
PopTo: Delete the most recent item from the pattern
queue, so as to remove the information saved by
an earlier 'push' action. Set the test indicator
then goto the specified rule.
PopToAct: Same as PopTo, except that the test indicator
is cleared before going to the specified rule.
As well as the attributes applying directly to packets (such as
SourcePeerAddress, DestTransAddress, etc.) the PME implements
several further attribtes. These are:
Null: Tests performed on the Null attribute always
succeed.
MatchingStoD: Indicates whether the PME is matching the packet
with its addresses in 'wire order' or with its
addresses reversed. MatchingStoD's value is 1 if
the addresses are in wire order (StoD), and zero
otherwise.
v1 .. v5: v1, v2, v3, v4 and v5 are 'meter variables'. They
provide a way to pass parameters into rule-
matching subroutines. Each may hold the number of
a normal attribute; its value is set by an Assign
action. When a meter variable appears as the
attribute of a rule, its value specifies the
actual attribute to be tested. For example, if v1
had been assigned SourcePeerAddress as its value,
a rule with v1 as its attribute would actually
test SourcePeerAddress.
SourceClass, DestClass, FlowClass,
SourceKind, DestKind, FlowKind:
These six attributes may be set by executing
PushRuleTo actions. They allow the PME to save
(in flow records) information which has been built
up during matching. Their values may be tested in
rules; this allows one to set them early in a rule
set, and test them later.
The opcodes detailed above (with their above 'goto' and 'test'
values) form a minimum set, but one which has proved very effective
in current meter implementations. From time to time it may be useful
to add further opcodes; IANA considerations for allocating these are
set out in section 8 below.
4.5 Maintaining the Flow Table
The flow table may be thought of as a 1-origin array of flow records.
(A particular implementation may, of course, use whatever data
structure is most suitable). When the meter starts up there are no
known flows; all the flow records are in the 'inactive' state.
Each time a packet is matched for a flow which is not in a current
flow set a flow record is created for it; the state of such a record
is
'current'. When selecting a record for the new flow the meter
searches the flow table for an 'inactive' record. If no inactive
records are available it will search for an 'idle' one instead. Note
that there is no particular significance in the ordering of records
within the flow table.
A meter's memory management routines should aim to minimise the time
spent finding flow records for new flows, so as to minimise the setup
overhead associated with each new flow.
Flow data may be collected by a 'meter reader' at any time. There is
no requirement for collections to be synchronized. The reader may
collect the data in any suitable manner, for example it could upload
a copy of the whole flow table using a file transfer protocol, or it
could read the records in the current flow set row by row using a
suitable data transfer protocol.
The meter keeps information about collections, in particular it
maintains ReaderLastTime variables which remember the time the last
collection was made by each reader. A second variable,
InactivityTime, specifies the minimum time the meter will wait before
considering that a flow is idle.
The meter must recover records used for idle flows, if only to
prevent it running out of flow records. Recovered flow records are
returned to the 'inactive' state. A variety of recovery strategies
are possible, including the following:
One possible recovery strategy is to recover idle flow records as
soon as possible after their data has been collected by all readers
which have registered to do so. To implement this the meter could
run a background process which scans the flow table looking for '
current' flows whose 'last packet' time is earlier than the meter's
LastCollectTime.
Another recovery strategy is to leave idle flows alone as long as
possible, which would be acceptable if one was only interested in
measuring total traffic volumes. It could be implemented by having
the meter search for collected idle flows only when it ran low on '
inactive' flow records.
One further factor a meter should consider before recovering a flow
is the number of meter readers which have collected the flow's data.
If there are multiple meter readers operating, each reader should
collect a flow's data before its memory is recovered.
Of course a meter reader may fail, so the meter cannot wait forever
for it. Instead the meter must keep a table of active meter readers,
with a timeout specified for each. If a meter reader fails to
collect flow data within its timeout interval, the meter should
delete that reader from the meter's active meter reader table.
4.6 Handling Increasing Traffic Levels
Under normal conditions the meter reader specifies which set of usage
records it wants to collect, and the meter provides them. If,
however, memory usage rises above the high-water mark the meter
should switch to a STANDBY RULE SET so as to decrease the rate at
which new flows are created.
When the manager, usually as part of a regular poll, becomes aware
that the meter is using its standby rule set, it could decrease the
interval between collections. This would shorten the time that flows
sit in memory waiting to be collected, allowing the meter to free
flow memory faster.
The meter could also increase its efforts to recover flow memory so
as to reduce the number of idle flows in memory. When the situation
returns to normal, the manager may request the meter to switch back
to its normal rule set.
5 Meter Readers
Usage data is accumulated by a meter (e.g. in a router) as memory
permits. It is collected at regular reporting intervals by meter
readers, as specified by a manager. The collected data is recorded
in stable storage as a FLOW DATA FILE, as a sequence of USAGE
RECORDS.
The following sections describe the contents of usage records and
flow data files. Note, however, that at this stage the details of
such records and files is not specified in the architecture.
Specifying a common format for them would be a worthwhile future
development.
5.1 Identifying Flows in Flow Records
Once a packet has been classified and is ready to be counted, an
appropriate flow data record must already exist in the flow table;
otherwise one must be created. The flow record has a flexible format
where unnecessary identification attributes may be omitted. The
determination of which attributes of the flow record to use, and of
what values to put in them, is specified by the current rule set.
Note that the combination of start time, rule set number and flow
subscript (row number in the flow table) provide a unique flow
identifier, regardless of the values of its other attributes.
The current rule set may specify additional information, e.g. a
computed attribute value such as FlowKind, which is to be placed in
the attribute section of the usage record. That is, if a particular
flow is matched by the rule set, then the corresponding flow record
should be marked not only with the qualifying identification
attributes, but also with the additional information. Using this
feature, several flows may each carry the same FlowKind value, so
that the resulting usage records can be used in post-processing or
between meter reader and meter as a criterion for collection.
5.2 Usage Records, Flow Data Files
The collected usage data will be stored in flow data files on the
meter reader, one file for each meter. As well as containing the
measured usage data, flow data files must contain information
uniquely identifiying the meter from which it was collected.
A USAGE RECORD contains the descriptions of and values for one or
more flows. Quantities are counted in terms of number of packets and
number of bytes per flow. Other quantities, e.g. short-term flow
rates, may be added later; work on such extensions is described in
the RTFM 'New Attributes' document [RTFM-NEW].
Each usage record contains the metered traffic group identifier of
the meter (a set of network addresses), a time stamp and a list of
reported flows (FLOW DATA RECORDS). A meter reader will build up a
file of usage records by regularly collecting flow data from a meter,
using this data to build usage records and concatenating them to the
tail of a file. Such a file is called a FLOW DATA FILE.
A usage record contains the following information in some form:
+-------------------------------------------------------------------+
| RECORD IDENTIFIERS: |
| Meter Id (& digital signature if required) |
| Timestamp |
| Collection Rules ID |
+-------------------------------------------------------------------+
| FLOW IDENTIFIERS: | COUNTERS |
| Address List | Packet Count |
| Subscriber ID (Optional) | Byte Count |
| Attributes (Optional) | Flow Start/Stop Time |
+-------------------------------------------------------------------+
5.3 Meter to Meter Reader: Usage Record Transmission
The usage record contents are the raison d'etre of the system. The
accuracy, reliability, and security of transmission are the primary
concerns of the meter/meter reader exchange. Since errors may occur
on networks, and Internet packets may be dropped, some mechanism for
ensuring that the usage information is transmitted intact is needed.
Flow data is moved from meter to meter reader via a series of
protocol exchanges between them. This may be carried out in various
ways, moving individual attribute values, complete flows, or the
entire flow table (i.e. all the active and idle flows). One possible
method of achieving this transfer is to use SNMP; the 'Traffic Flow
Measurement: Meter MIB' RFC[RTFM-MIB] gives details. Note that
this is simply one example; the transfer of flow data from meter to
meter reader is not specified in this document.
The reliability of the data transfer method under light, normal, and
extreme network loads should be understood before selecting among
collection methods.
In normal operation the meter will be running a rule file which
provides the required degree of flow reporting granularity, and the
meter reader(s) will collect the flow data often enough to allow the
meter's garbage collection mechanism to maintain a stable level of
memory usage.
In the worst case traffic may increase to the point where the meter
is in danger of running completely out of flow memory. The meter
implementor must decide how to handle this, for example by switching
to a default (extremely coarse granularity) rule set, by sending a
trap message to the manager, or by attempting to dump flow data to
the meter reader.
Users of the Traffic Flow Measurement system should analyse their
requirements carefully and assess for themselves whether it is more
important to attempt to collect flow data at normal granularity
(increasing the collection frequency as needed to keep up with
traffic volumes), or to accept flow data with a coarser granularity.
Similarly, it may be acceptable to lose flow data for a short time in
return for being sure that the meter keeps running properly, i.e. is
not overwhelmed by rising traffic levels.
6 Managers
A manager configures meters and controls meter readers. It does this
via the interactions described below.
6.1 Between Manager and Meter: Control Functions
- DOWNLOAD RULE SET: A meter may hold an array of rule sets. One
of these, the 'default' rule set, is built in to the meter and
cannot be changed; this is a diagnostic feature, ensuring that
when a meter starts up it will be running a known ruleset.
All other rule sets must be downloaded by the manager. A manager
may use any suitable protocol exchange to achieve this, for
example an FTP file transfer or a series of SNMP SETs, one for
each row of the rule set.
- SPECIFY METER TASK: Once the rule sets have been downloaded, the
manager must instruct the meter which rule sets will be the
'current' and 'standby' ones for each task the meter is to
perform.
- SET HIGH WATER MARK: A percentage of the flow table capacity,
used by the meter to determine when to switch to its standby rule
set (so as to increase the granularity of the flows and conserve
the meter's flow memory). Once this has happened, the manager
may also change the polling frequency or the meter's control
parameters (so as to increase the rate at which the meter can
recover memory from idle flows). The meter has a separate high
water mark value for each task it is currently running.
If the high traffic levels persist, the meter's normal rule set
may have to be rewritten to permanently reduce the reporting
granularity.
- SET FLOW TERMINATION PARAMETERS: The meter should have the good
sense in situations where lack of resources may cause data loss
to purge flow records from its tables. Such records may include:
- Flows that have already been reported to all registered meter
readers, and show no activity since the last report,
- Oldest flows, or
- Flows with the smallest number of observed packets.
- SET INACTIVITY TIMEOUT: This is a time in seconds since the last
packet was seen for a flow. Flow records may be reclaimed if
they have been idle for at least this amount of time, and have
been collected in accordance with the current collection
criteria.
It might be useful if a manager could set the FLOW TERMINATION
PARAMETERS to different values for different tasks. Current meter
implementations have only single ('whole meter') values for these
parameters, and experience to date suggests that this provides an
adequate degree of control for the tasks.
6.2 Between Manager and Meter Reader: Control Functions
Because there are a number of parameters that must be set for traffic
flow measurement to function properly, and viable settings may change
as a result of network traffic characteristics, it is desirable to
have dynamic network management as opposed to static meter
configurations. Many of these operations have to do with space
tradeoffs - if memory at the meter is exhausted, either the
collection interval must be decreased or a coarser granularity of
aggregation must be used to reduce the number of active flows.
Increasing the collection interval effectively stores data in the
meter; usage data in transit is limited by the effective bandwidth of
the virtual link between the meter and the meter reader, and since
these limited network resources are usually also used to carry user
data (the purpose of the network), the level of traffic flow
measurement traffic should be kept to an affordable fraction of the
bandwidth. ("Affordable" is a policy decision made by the Network
Operations personnel). At any rate, it must be understood that the
operations below do not represent the setting of independent
variables; on the contrary, each of the values set has a direct and
measurable effect on the behaviour of the other variables.
Network management operations follow:
- MANAGER and METER READER IDENTIFICATION: The manager should
ensure that meters are read by the correct set of meter readers,
and take steps to prevent unauthorised access to usage
information. The meter readers so identified should be prepared
to poll if necessary and accept data from the appropriate meters.
Alternate meter readers may be identified in case both the
primary manager and the primary meter reader are unavailable.
Similarly, alternate managers may be identified.
- REPORTING INTERVAL CONTROL: The usual reporting interval should
be selected to cope with normal traffic patterns. However, it
may be possible for a meter to exhaust its memory during traffic
spikes even with a correctly set reporting interval. Some
mechanism should be available for the meter to tell the manager
that it is in danger of exhausting its memory (by declaring a '
high water' condition), and for the manager to arbitrate (by
decreasing the polling interval, letting nature take its course,
or by telling the meter to ask for help sooner next time).
- GRANULARITY CONTROL: Granularity control is a catch-all for all
the parameters that can be tuned and traded to optimise the
system's ability to reliably measure and store information on all
the traffic (or as close to all the traffic as an administration
requires). Granularity:
- Controls the amount of address information identifying each
flow, and
- Determines the number of buckets into which user traffic
will be lumped together.
Since granularity is controlled by the meter's current rule set,
the manager can only change it by requesting the meter to switch
to a different rule set. The new rule set could be downloaded
when required, or it could have been downloaded as part of the
meter's initial configuration.
- FLOW LIFETIME CONTROL: Flow termination parameters include
timeout parameters for obsoleting inactive flows and removing
them from tables, and maximum flow lifetimes. This is
intertwined with reporting interval and granularity, and must be
set in accordance with the other parameters.
6.3 Exception Conditions
Exception conditions must be handled, particularly occasions when the
meter runs out of space for flow data. Since - to prevent an active
task from counting any packet twice - packets can only be counted in
a single flow, discarding records will result in the loss of
information. The mechanisms to deal with this are as follows:
- METER OUTAGES: In case of impending meter outages (controlled
restarts, etc.) the meter could send a trap to the manager. The
manager could then request one or more meter readers to pick up
the data from the meter.
Following an uncontrolled meter outage such as a power failure,
the meter could send a trap to the manager indicating that it has
restarted. The manager could then download the meter's correct
rule set and advise the meter reader(s) that the meter is running
again. Alternatively, the meter reader may discover from its
regular poll that a meter has failed and restarted. It could
then advise the manager of this, instead of relying on a trap
from the meter.
- METER READER OUTAGES: If the collection system is down or
isolated, the meter should try to inform the manager of its
failure to communicate with the collection system. Usage data is
maintained in the flows' rolling counters, and can be recovered
when the meter reader is restarted.
- MANAGER OUTAGES: If the manager fails for any reason, the meter
should continue measuring and the meter reader(s) should keep
gathering usage records.
- BUFFER PROBLEMS: The network manager may realise that there is a
'low memory' condition in the meter. This can usually be
attributed to the interaction between the following controls:
- The reporting interval is too infrequent, or
- The reporting granularity is too fine.
Either of these may be exacerbated by low throughput or bandwidth
of circuits carrying the usage data. The manager may change any
of these parameters in response to the meter (or meter reader's)
plea for help.
6.4 Standard Rule Sets
Although the rule table is a flexible tool, it can also become very
complex. It may be helpful to develop some rule sets for common
applications:
- PROTOCOL TYPE: The meter records packets by protocol type. This
will be the default rule table for Traffic Flow Meters.
- ADJACENT SYSTEMS: The meter records packets by the MAC address of
the Adjacent Systems (neighbouring originator or next-hop).
(Variants on this table are "report source" or "report sink"
only.) This strategy might be used by a regional or backbone
network which wants to know how much aggregate traffic flows to
or from its subscriber networks.
- END SYSTEMS: The meter records packets by the IP address pair
contained in the packet. (Variants on this table are "report
source" or "report sink" only.) This strategy might be used by
an End System network to get detailed host traffic matrix usage
data.
- TRANSPORT TYPE: The meter records packets by transport address;
for IP packets this provides usage information for the various IP
services.
- HYBRID SYSTEMS: Combinations of the above, e.g. for one interface
report End Systems, for another interface report Adjacent
Systems. This strategy might be used by an enterprise network to
learn detail about local usage and use an aggregate count for the
shared regional network.
7 Security Considerations
7.1 Threat Analysis
A traffic flow measurement system may be subject to the following
kinds of attacks:
- ATTEMPTS TO DISABLE A TRAFFIC METER: An attacker may attempt to
disrupt traffic measurement so as to prevent users being charged
for network usage. For example, a network probe sending packets
to a large number of destination and transport addresses could
produce a sudden rise in the number of flows in a meter's flow
table, thus forcing it to use its coarser standby rule set.
- UNAUTHORIZED USE OF SYSTEM RESOURCES: An attacker may wish to
gain advantage or cause mischief (e.g. denial of service) by
subverting any of the system elements - meters, meter readers or
managers.
- UNAUTHORIZED DISCLOSURE OF DATA: Any data that is sensitive to
disclosure can be read through active or passive attacks unless
it is suitably protected. Usage data may or may not be of this
type. Control messages, traps, etc. are not likely to be
considered sensitive to disclosure.
- UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA:
Similarly, any data whose integrity is sensitive can be altered,
replaced/injected or deleted through active or passive attacks
unless it is suitably protected. Attackers may modify message
streams to falsify usage data or interfere with the proper
operation of the traffic flow measurement system. Therefore, all
messages, both those containing usage data and those containing
control data, should be considered vulnerable to such attacks.
7.2 Countermeasures
The following countermeasures are recommended to address the possible
threats enumerated above:
- ATTEMPTS TO DISABLE A TRAFFIC METER can't be completely
countered. In practice, flow data records from network security
attacks have proved very useful in determining what happened.
The most effective approach is first to configure the meter so
that it has three or more times as much flow memory as it needs
in normal operation, and second to collect the flow data fairly
frequently so as to minimise the time needed to recover flow
memory after such an attack.
- UNAUTHORIZED USE OF SYSTEM RESOURCES is countered through the use
of authentication and access control services.
- UNAUTHORIZED DISCLOSURE OF DATA is countered through the use of a
confidentiality (encryption) service.
- UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA is
countered through the use of an integrity service.
A Traffic Measurement system must address all of these concerns.
Since a high degree of protection is required, the use of strong
cryptographic methodologies is recommended. The security
requirements for communication between pairs of traffic measurmement
system elements are summarized in the table below. It is assumed
that meters do not communicate with other meters, and that meter
readers do not communicate directly with other meter readers (if
synchronization is required, it is handled by the manager, see
Section 2.5). Each entry in the table indicates which kinds of
security services are required. Basically, the requirements are as
follows:
Security Service Requirements for RTFM elements
+------------------------------------------------------------------+
| from\to | meter | meter reader | application | manager |
|---------+--------------+--------------+-------------+------------|
| meter | N/A | authent | N/A | authent |
| | | acc ctrl | | acc ctrl |
| | | integrity | | |
| | | confid ** | | |
|---------+--------------+--------------+-------------+------------|
| meter | authent | N/A | authent | authent |
| reader | acc ctrl | | acc ctrl | acc ctrl |
| | | | integrity | |
| | | | confid ** | |
|---------+--------------+--------------+-------------+------------|
| appl | N/A | authent | | |
| | | acc ctrl | ## | ## |
|---------+--------------+--------------+-------------+------------|
| manager | authent | authent | ## | authent |
| | acc ctrl | acc ctrl | | acc ctrl |
| | integrity | integrity | | integrity |
+------------------------------------------------------------------+
N/A = Not Applicable ** = optional ## = outside RTFM scope
- When any two elements intercommunicate they should mutually
authenticate themselves to one another. This is indicated by '
authent' in the table. Once authentication is complete, an
element should check that the requested type of access is
allowed; this is indicated on the table by 'acc ctrl'.
- Whenever there is a transfer of information its integrity should
be protected.
- Whenever there is a transfer of usage data it should be possible
to ensure its confidentiality if it is deemed sensitive to
disclosure. This is indicated by 'confid' in the table.
Security protocols are not specified in this document. The system
elements' management and collection protocols are responsible for
providing sufficient data integrity, confidentiality, authentication
and access control services.
8 IANA Considerations
The RTFM Architecture, as set out in this document, has two sets of
assigned numbers. Considerations for assigning them are discussed in
this section, using the example policies as set out in the
"Guidelines for IANA Considerations" document [IANA-RFC].
8.1 PME Opcodes
The Pattern Matching Engine (PME) is a virtual machine, executing
RTFM rules as its instructions. The PME opcodes appear in the
'action' field of an RTFM rule. The current list of opcodes, and
their values for the PME's 'goto' and 'test' flags, are set out in
section 4.4 above ("Rules and Rulesets).
The PME opcodes are pivotal to the RTFM architecture, since they must
be implemented in every RTFM meter. Any new opcodes must therefore
be allocated through an IETF Consensus action [IANA-RFC].
Opcodes are simply non-negative integers, but new opcodes should be
allocated sequentially so as to keep the total opcode range as small
as possible.
8.2 RTFM Attributes
Attribute numbers in the range of 0-511 are globally unique and are
allocated according to an IETF Consensus action [IANA-RFC]. Appendix
C of this document allocates a basic (i.e. useful minimum) set of
attribtes; they are assigned numbers in the range 0 to 63. The RTFM
working group is working on an extended set of attributes, which will
have numbers in the range 64 to 127.
Vendor-specific attribute numbers are in the range 512-1023, and will
be allocated using the First Come FIrst Served policy [IANA-RFC].
Vendors requiring attribute numbers should submit a request to IANA
giving the attribute names: IANA will allocate them the next
available numbers.
Attribute numbers 1024 and higher are Reserved for Private Use
[IANA-RFC]. Implementors wishing to experiment with further new
attributes should use attribute numbers in this range.
Attribute numbers are simply non-negative integers. When writing
specifications for attributes, implementors must give sufficient
detail for the new attributes to be easily added to the RTFM Meter
MIB [RTFM-MIB]. In particular, they must indicate whether the new
attributes may be:
- tested in an IF statement
- saved by a SAVE statement or set by a STORE statement
- read from an RTFM meter
(IF, SAVE and STORE are statements in the SRL Ruleset Language
[RTFM-SRL]).
9 APPENDICES
9.1 Appendix A: Network Characterisation
Internet users have extraordinarily diverse requirements. Networks
differ in size, speed, throughput, and processing power, among other
factors. There is a range of traffic flow measurement capabilities
and requirements. For traffic flow measurement purposes, the
Internet may be viewed as a continuum which changes in character as
traffic passes through the following representative levels:
International |
Backbones/National ---------------
/ \
Regional/MidLevel ---------- ----------
/ \ \ / / \
Stub/Enterprise --- --- --- ---- ----
||| ||| ||| |||| ||||
End-Systems/Hosts xxx xxx xxx xxxx xxxx
Note that mesh architectures can also be built out of these
components, and that these are merely descriptive terms. The nature
of a single network may encompass any or all of the descriptions
below, although some networks can be clearly identified as a single
type.
BACKBONE networks are typically bulk carriers that connect other
networks. Individual hosts (with the exception of network management
devices and backbone service hosts) typically are not directly
connected to backbones.
REGIONAL networks are closely related to backbones, and differ only
in size, the number of networks connected via each port, and
geographical coverage. Regionals may have directly connected hosts,
acting as hybrid backbone/stub networks. A regional network is a
SUBSCRIBER to the backbone.
STUB/ENTERPRISE networks connect hosts and local area networks.
STUB/ENTERPRISE networks are SUBSCRIBERS to regional and backbone
networks.
END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above
networks.
Providing a uniform identification of the SUBSCRIBER in finer
granularity than that of end-system, (e.g. user/account), is beyond
the scope of the current architecture, although an optional attribute
in the traffic flow measurement record may carry system-specific
'user identification' labels so that meters can implement proprietary
or non-standard schemes for the attribution of network traffic to
responsible parties.
9.2 Appendix B: Recommended Traffic Flow Measurement Capabilities
Initial recommended traffic flow measurement conventions are outlined
here according to the following Internet building blocks. It is
important to understand what complexity reporting introduces at each
network level. Whereas the hierarchy is described top-down in the
previous section, reporting requirements are more easily addressed
bottom-up.
End-Systems
Stub Networks
Enterprise Networks
Regional Networks
Backbone Networks
END-SYSTEMS are currently responsible for allocating network usage to
end-users, if this capability is desired. From the Internet Protocol
perspective, end-systems are the finest granularity that can be
identified without protocol modifications. Even if a meter violated
protocol boundaries and tracked higher-level protocols, not all
packets could be correctly allocated by user, and the definition of
user itself varies widely from operating system to operating system
(e.g. how to trace network usage back to users from shared
processes).
STUB and ENTERPRISE networks will usually collect traffic data either
by end-system network address or network address pair if detailed
reporting is required in the local area network. If no local
reporting is required, they may record usage information in the exit
router to track external traffic only. (These are the only networks
which routinely use attributes to perform reporting at granularities
finer than end-system or intermediate-system network address.)
REGIONAL networks are intermediate networks. In some cases,
subscribers will be enterprise networks, in which case the
intermediate system network address is sufficient to identify the
regional's immediate subscriber. In other cases, individual hosts or
a disjoint group of hosts may constitute a subscriber. Then end-
system network address pairs need to be tracked for those
subscribers. When the source may be an aggregate entity (such as a
network, or adjacent router representing traffic from a world of
hosts beyond) and the destination is a singular entity (or vice
versa), the meter is said to be operating as a HYBRID system.
At the regional level, if the overhead is tolerable it may be
advantageous to report usage both by intermediate system network
address (e.g. adjacent router address) and by end-system network
address or end-system network address pair.
BACKBONE networks are the highest level networks operating at higher
link speeds and traffic levels. The high volume of traffic will in
most cases preclude detailed traffic flow measurement. Backbone
networks will usually account for traffic by adjacent routers'
network addresses.
9.3 Appendix C: List of Defined Flow Attributes
This Appendix provides a checklist of the attributes defined to date;
others will be added later as the Traffic Measurement Architecture is
further developed.
Note that this table gives only a very brief summary. The Meter MIB
[RTFM-MIB] provides the definitive specification of attributes and
their allowed values. The MIB variables which represent flow
attributes have 'flowData' prepended to their names to indicate that
they belong to the MIB's flowData table.
0 Null
4 SourceInterface Integer Source Address
5 SourceAdjacentType Integer
6 SourceAdjacentAddress String
7 SourceAdjacentMask String
8 SourcePeerType Integer
9 SourcePeerAddress String
10 SourcePeerMask String
11 SourceTransType Integer
12 SourceTransAddress String
13 SourceTransMask String
14 DestInterface Integer Destination Address
15 DestAdjacentType Integer
16 DestAdjacentAddress String
17 DestAdjacentMask String
18 DestPeerType Integer
19 DestPeerAddress String
20 DestPeerMask String
21 DestTransType Integer
22 DestTransAddress String
23 DestTransMask String
26 RuleSet Integer Meter attribute
27 ToOctets Integer Source-to-Dest counters
28 ToPDUs Integer
29 FromOctets Integer Dest-to-Source counters
30 FromPDUs Integer
31 FirstTime Timestamp Activity times
32 LastActiveTime Timestamp
33 SourceSubscriberID String Session attributes
34 DestSubscriberID String
35 SessionID String
36 SourceClass Integer 'Computed' attributes
37 DestClass Integer
38 FlowClass Integer
39 SourceKind Integer
40 DestKind Integer
41 FlowKind Integer
50 MatchingStoD Integer PME variable
51 v1 Integer Meter Variables
52 v2 Integer
53 v3 Integer
54 v4 Integer
55 v5 Integer
65
.. 'Extended' attributes (to be defined by the RTFM working group)
127
9.4 Appendix D: List of Meter Control Variables
Meter variables:
Flood Mark Percentage
Inactivity Timeout (seconds) Integer
'per task' variables:
Current Rule Set Number Integer
Standby Rule Set Number Integer
High Water Mark Percentage
'per reader' variables:
Reader Last Time Timestamp
9.5 Appendix E: Changes Introduced Since RFC2063
The first version of the Traffic Flow Measurement Architecture was
published as RFC2063 in January 1997. The most significant changes
made since then are summarised below.
- A Traffic Meter can now run multiple rule sets concurrently.
This makes a meter much more useful, and required only minimal
changes to the architecture.
- 'NoMatch' replaces 'Fail' as an action. This name was agreed to
at the Working Group 1996 meeting in Montreal; it better
indicates that although a particular match has failed, it may be
tried again with the packet's addresses reversed.
- The 'MatchingStoD' attribute has been added. This is a Packet
Matching Engine (PME) attribute indicating that addresses are
being matched in StoD (i.e. 'wire') order. It can be used to
perform different actions when the match is retried, thereby
simplifying some kinds of rule sets. It was discussed and agreed
to at the San Jose meeting in 1996.
- Computed attributes (Class and Kind) may now be tested within a
rule set. This lifts an unneccessary earlier restriction.
- The list of attribute numbers has been extended to define ranges
for 'basic' attributes (in this document) and 'extended'
attributes (currently being developed by the RTFM Working Group).
- The 'Security Considerations' section has been completely
rewritten. It provides an evaluation of traffic measurement
security risks and their countermeasures.
10 Acknowledgments
An initial draft of this document was produced under the auspices
of the IETF's Internet Accounting Working Group with assistance
from SNMP, RMON and SAAG working groups. Particular thanks are
due to Stephen Stibler (IBM Research) for his patient and careful
comments during the preparation of this memo.
11 References
[802-3] IEEE 802.3/ISO 8802-3 Information Processing Systems -
Local Area Networks - Part 3: Carrier sense multiple
access with collision detection (CSMA/CD) access method
and physical layer specifications, 2nd edition, September
21, 1990.
[ACT-BKG] Mills, C., Hirsch, G. and G. Ruth, "Internet Accounting
Background", RFC1272, November 1991.
[IANA-RFC] Alvestrand, H. and T. Narten, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC2434,
October 1998.
[IPPM-FRM] Paxson, V., Almes, G., Mahdavi, J. and M. Mathis,
"Framework for IP Performance Metrics", RFC2330, May
1998.
[OSI-ACT] International Standards Organisation (ISO), "Management
Framework", Part 4 of Information Processing Systems Open
Systems Interconnection Basic Reference Model, ISO 7498-4,
1994.
[RTFM-MIB] Brownlee, N., "Traffic Flow Measurement: Meter MIB", RFC
2720, October 1999.
[RTFM-NEW] Handelman, S., Stibler, S., Brownlee, N. and G. Ruth,
"RTFM: New Attributes for Traffic Flow Measurment", RFC
2724, October 1999.
[RTFM-SRL] Brownlee, N., "SRL: A Language for Describing Traffic
Flows and Specifying Actions for Flow Groups", RFC2723,
October 1999.
12 Authors' Addresses
Nevil Brownlee
Information Technology Systems & Services
The University of Auckland
Private Bag 92-019
Auckland, New Zealand
Phone: +64 9 373 7599 x8941
EMail: n.brownlee@auckland.ac.nz
Cyndi Mills
GTE Laboratories, Inc
40 Sylvan Rd.
Waltham, MA 02451, U.S.A.
Phone: +1 781 466 4278
EMail: cmills@gte.com
Greg Ruth
GTE Internetworking
3 Van de Graaff Drive
P.O. Box 3073
Burlington, MA 01803, U.S.A.
Phone: +1 781 262 4831
EMail: gruth@bbn.com
13 Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFCEditor function is currently provided by the
Internet Society.