Request for Comments: 3917 NEC Europe Ltd.
Category: Informational T. Zseby
Fraunhofer FOKUS
B. Claise
Cisco Systems
S. Zander
Swinburne University
October 2004
Requirements for IP Flow Information Export (IPFIX)
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2004).
Abstract
This memo defines requirements for the export of measured IP flow
information out of routers, traffic measurement probes, and
middleboxes.
Table of Contents
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. IP Traffic Flow. . . . . . . . . . . . . . . . . . . . 3
2.2. Observation Point. . . . . . . . . . . . . . . . . . . 4
2.3. Metering Process . . . . . . . . . . . . . . . . . . . 4
2.4. Flow Record. . . . . . . . . . . . . . . . . . . . . . 5
2.5. Exporting Process. . . . . . . . . . . . . . . . . . . 5
2.6. Collecting Process . . . . . . . . . . . . . . . . . . 5
3. Applications Requiring IP Flow Information Export . . . . . . 6
3.1. Usage-based Accounting . . . . . . . . . . . . . . . . 6
3.2. Traffic Profiling. . . . . . . . . . . . . . . . . . . 7
3.3. Traffic Engineering. . . . . . . . . . . . . . . . . . 7
3.4. Attack/Intrusion Detection . . . . . . . . . . . . . . 7
3.5. QoS Monitoring . . . . . . . . . . . . . . . . . . . . 8
4. Distinguishing Flows. . . . . . . . . . . . . . . . . . . . . 8
4.1. Encryption . . . . . . . . . . . . . . . . . . . . . . 9
4.2. Interfaces . . . . . . . . . . . . . . . . . . . . . . 9
4.3. IP Header Fields . . . . . . . . . . . . . . . . . . . 9
4.4. Transport Header Fields. . . . . . . . . . . . . . . . 10
4.5. MPLS Label . . . . . . . . . . . . . . . . . . . . . . 10
4.6. DiffServ Code Point. . . . . . . . . . . . . . . . . . 10
5. Metering Process. . . . . . . . . . . . . . . . . . . . . . . 10
5.1. Reliability. . . . . . . . . . . . . . . . . . . . . . 10
5.2. Sampling . . . . . . . . . . . . . . . . . . . . . . . 11
5.3. Overload Behavior. . . . . . . . . . . . . . . . . . . 11
5.4. Timestamps . . . . . . . . . . . . . . . . . . . . . . 12
5.5. Time Synchronization . . . . . . . . . . . . . . . . . 12
5.6. Flow Expiration. . . . . . . . . . . . . . . . . . . . 13
5.7. Multicast Flows. . . . . . . . . . . . . . . . . . . . 13
5.8. Packet Fragmentation . . . . . . . . . . . . . . . . . 13
5.9. Ignore Port Copy . . . . . . . . . . . . . . . . . . . 13
6. Data Export . . . . . . . . . . . . . . . . . . . . . . . . . 14
6.1. Information Model. . . . . . . . . . . . . . . . . . . 14
6.2. Data Model . . . . . . . . . . . . . . . . . . . . . . 16
6.3. Data Transfer. . . . . . . . . . . . . . . . . . . . . 16
6.3.1. Congestion Awareness. . . . . . . . . . . . . . 16
6.3.2. Reliability . . . . . . . . . . . . . . . . . . 17
6.3.3. Security. . . . . . . . . . . . . . . . . . . . 18
6.4. Push and Pull Mode Reporting . . . . . . . . . . . . . 18
6.5. Regular Reporting Interval . . . . . . . . . . . . . . 18
6.6. Notification on Specific Events. . . . . . . . . . . . 18
6.7. Anonymization. . . . . . . . . . . . . . . . . . . . . 18
7. Configuration . . . . . . . . . . . . . . . . . . . . . . . . 19
7.1. Configuration of the Metering Process. . . . . . . . . 19
7.2. Configuration of the Exporting Process . . . . . . . . 19
8. General Requirements. . . . . . . . . . . . . . . . . . . . . 20
8.1. Openness . . . . . . . . . . . . . . . . . . . . . . . 20
8.2. Scalability. . . . . . . . . . . . . . . . . . . . . . 20
8.3. Several Collecting Processes . . . . . . . . . . . . . 20
9. Special Device Considerations . . . . . . . . . . . . . . . . 20
10. Security Considerations . . . . . . . . . . . . . . . . . . . 23
10.1. Disclosure of Flow Information Data. . . . . . . . . . 23
10.2. Forgery of Flow Records. . . . . . . . . . . . . . . . 24
10.3. Denial of Service (DoS) Attacks. . . . . . . . . . . . 24
11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 25
12. Appendix: Derivation of Requirements from Applications. . . . 26
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 31
13.1. Normative References . . . . . . . . . . . . . . . . . 31
13.2. Informative References . . . . . . . . . . . . . . . . 31
14. Authors’ Addresses . . . . . . . . . . . . . . . . . . . . . . 32
15. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 33
1. Introduction
There are several applications that require flow-based IP traffic
measurements. Such measurements could be performed by a router while
forwarding the traffic, by a middlebox [RFC3234], or by a traffic
measurement probe attached to a line or a monitored port. This memo
defines requirements for exporting traffic flow information out of
these boxes for further processing by applications located on other
devices. They serve as input to the standardization of the IPFIX
protocol specifications.
In section 3, a selection of such applications is presented. The
following sections list requirements derived from these applications.
In its early discussions the IPFIX Working Group chose to evaluate
existing flow export protocols at the same time it was developing
this ’requirements’ document.
Flow export, however, is not performed by a protocol acting alone, it
also requires a system of co-operating processes. In producing IPFIX
requirements, therefore, the Working Group decided to specify what
was required by these various processes - the metering process, the
exporting process, etc. In these specifications we use lower-case
for the words must, may, and should, to indicate that IPFIX
implementors have some freedom as to how to meet the requirements.
The Working Group’s goal is to produce standards-track RFCs
describing the IPFIX information model and export protocol RFCs. As
well as meeting the requirements set out in this document, the
information model and protocol documents will provide a full
specification of the IPFIX system, and will use uppercase keywords as
in [RFC 2119].
2. Terminology
The following terminology is used in this document:
2.1. IP Traffic Flow
There are several definitions of the term ’flow’ being used by the
Internet community. Within this document we use the following one:
A flow is defined as a set of IP packets passing an observation point
in the network during a certain time interval. All packets belonging
to a particular flow have a set of common properties. Each property
is defined as the result of applying a function to the values of:
1. one or more packet header field (e.g., destination IP address),
transport header field (e.g., destination port number), or
application header field (e.g., RTP header fields [RFC3550])
2. one or more characteristics of the packet itself (e.g., number
of MPLS labels, etc.)
3. one or more of fields derived from packet treatment (e.g., next
hop IP address, the output interface, etc.)
A packet is defined to belong to a flow if it completely satisfies
all the defined properties of the flow.
This definition covers the range from a flow containing all packets
observed at a network interface to a flow consisting of just a single
packet between two applications with a specific sequence number.
Please note that the flow definition does not necessarily match a
general application-level end-to-end stream. However, an application
may derive properties of application-level streams by processing
measured flow data. Also, please note that although packet
properties may depend on application headers, there is no requirement
defined in this document related to application headers.
2.2. Observation Point
The observation point is a location in the network where IP packets
can be observed. Examples are a line to which a probe is attached, a
shared medium such as an Ethernet-based LAN, a single port of a
router, or a set of interfaces (physical or logical) of a router.
Note that one observation point may be a superset of several other
observation points. For example one observation point can be an
entire line card. This would be the superset of the individual
observation points at the line card’s interfaces.
2.3. Metering Process
The metering process generates flow records. Input to the process
are packet headers observed at an observation point and packet
treatment at the observation point, for example the selected output
interface. The metering process consists of a set of functions that
includes packet header capturing, timestamping, sampling,
classifying, and maintaining flow records.
The maintenance of flow records may include creating new records,
updating existing ones, computing flow statistics, deriving further
flow properties, detecting flow expiration, passing flow records to
the exporting process, and deleting flow records.
The sampling function and the classifying function may be applied
more than once with different parameters. Figure 1 shows the
sequence in which the functions are applied. Sampling is not
illustrated in the figure; it may be applied before any other
function.
packet header capturing
|
timestamping
|
v
+----->+
| |
| classifying
| |
+------+
|
maintaining flow records
|
v
Figure 1: Functions of the metering process
2.4. Flow Record
A flow record contains information about a specific flow that was
metered at an observation point. A flow record contains measured
properties of the flow (e.g., the total number of bytes of all
packets of the flow) and usually characteristic properties of the
flow (e.g., source IP address).
2.5. Exporting Process
The exporting process sends flow records to one or more collecting
processes. The flow records are generated by one or more metering
processes.
2.6. Collecting Process
The collecting process receives flow records from one or more
exporting processes. The collecting process might store received
flow records or further process them, but these actions are out of
the scope of this document.
3. Applications Requiring IP Flow Information Export
This section describes a selection of applications requiring IP flow
information export. Because requirements for flow export listed in
further sections below are derived from these applications, their
selection is crucial. The goal of this requirements document is not
to cover all possible applications with all their flow export
requirements, but to cover applications which are considered to be of
significant importance in today’s and/or future IP networks, and for
which requirements can be met with reasonable technical effort.
The list of applications should lead to a better understanding of the
requirements which is particularly important when designing or
implementing traffic flow metering functions. A detailed overview of
which requirement was derived from which application(s) is given in
the appendix.
Please note that the described applications can have a large number
of differing implementations. Requirement details or requirement
significance (required (must), recommended (should), optional (may))
could differ for specific implementations and/or for specific
application scenarios. Therefore we derive the requirements from the
general functionality of the selected applications. Some particular
cases will even mandate more stringent requirements than the ones
defined in this document. For example, usage-based accounting is
certainly the application that will probably mandate the highest
degree of reliability amongst the applications discussed below. The
reliability requirements defined in sections 5.1 and 6.3.2. are not
sufficient to guarantee the level of reliability that is needed for
many usage-based accounting systems. Particular reliability
requirements for accounting systems are discussed in [RFC2975].
3.1. Usage-based Accounting
Several new business models for selling IP services and IP-based
services are currently under investigation. Beyond flat rate
services which do not need accounting, accounting can be based on
time or volume. Accounting data can serve as input for billing
systems. Accounting can be performed per user or per user group, it
can be performed just for basic IP service or individually per high-
level service and/or per content type delivered. For advanced/future
services, accounting may also be performed per class of service, per
application, per time of day, per (label switched) path used, etc.
3.2. Traffic Profiling
Traffic profiling is the process of characterizing IP flows by using
a model that represents key parameters of the flows such as flow
duration, volume, time, and burstiness. It is a prerequisite for
network planning, network dimensioning, trend analysis, business
model development, and other activities. It depends heavily on the
particular traffic profiling objective(s), which statistics, and
which accuracy are required from the measurements. Typical
information needed for traffic profiling is the distribution of used
services and protocols in the network, the amount of packets of a
specific type (e.g., percentage of IPv6 packets) and specific flow
profiles.
Since objectives for traffic profiling can vary, this application
requires a high flexibility of the measurement infrastructure,
especially regarding the options for measurement configuration and
packet classification.
3.3. Traffic Engineering
Traffic Engineering (TE) comprises methods for measurement,
modelling, characterization and control of a network. The goal of TE
is the optimization of network resource utilization and traffic
performance [RFC2702]. Since control and administrative reaction to
measurement results requires access to the involved network nodes, TE
mechanisms and the required measurement function usually are
performed within one administrative domain. Typical parameters
required for TE are link utilization, load between specific network
nodes, number, size and entry/exit points of the active flows and
routing information.
3.4. Attack/Intrusion Detection
Capturing flow information plays an important role for network
security, both for detection of security violation, and for
subsequent defense. In case of a Denial of Service (DOS) attack,
flow monitoring can allow detection of unusual situations or
suspicious flows. In a second step, flow analysis can be performed
in order to gather information about the attacking flows, and for
deriving a defense strategy.
Intrusion detection is a potentially more demanding application which
would not only look at specific characteristics of flows, but may
also use a stateful packet flow analysis for detecting specific,
suspicious activities, or unusually frequent activities. Such
activities may be characterized by specific communication patterns,
detectable by characteristic sequences of certain packet types.
3.5. QoS Monitoring
QoS monitoring is the passive measurement of quality parameters for
IP flows. In contrast to active measurements, passive measurements
utilize the existing traffic in the network for QoS analysis. Since
no test traffic is sent, passive measurements can only be applied in
situations where the traffic of interest is already present in the
network. One example application is the validation of QoS parameters
negotiated in a service level specification. Note that
passive/active measurement is also referred to as non-
intrusive/intrusive measurement or as measurement of
observed/synthetic traffic.
Passive measurements cannot provide the kind of controllable
experiments that can be achieved with active measurements. On the
other hand passive measurements do not suffer from undesired side
effects caused by sending test traffic (e.g., additional load,
potential differences in treatment of test traffic and real customer
traffic).
QoS monitoring often requires the correlation of data from multiple
observation points (e.g., for measuring one-way metrics). This
requires proper clock synchronization of the involved metering
processes. For some measurements, flow records and/or notifications
on specific events at the different observation points must be
correlated, for example the arrival of a certain packet. For this,
the provisioning of post-processing functions (e.g., the generation
of packet IDs) at the metering processes would be useful. Since QoS
monitoring can lead to a huge amount of measurement result data, it
would highly benefit from mechanisms to reduce the measurement data,
like aggregation of results and sampling.
Please note that not all requirements for QoS monitoring are covered
by the IPFIX requirements specified in the following sections. The
IPFIX requirements are targeted at per flow information including
summaries of per-packet properties for packets within a flow, but not
per-packet information itself. For example jitter measurement
requires timestamping each packet and reporting of all timestamps of
a flow, but the IPFIX requirements only cover timestamps of first and
last packet of a flow.
4. Distinguishing Flows
Packets are mapped to flows by evaluating their properties. Packets
with common properties are considered to belong to the same flow. A
packet showing at least one difference in the set of properties is
considered to belong to a different flow.
The following subsections list a set of properties which a metering
process must, should, or may be able to evaluate for mapping packets
to flows. Please note that requiring the ability to evaluate a
certain property does not imply that this property must be evaluated
for each packet. In other words, meeting the IPFIX requirements
means that the metering process in general must be able, via its
configuration, to somehow support to distinguish flows via all the
must fields, even if in certain circumstances/for certain
applications, only a subset of the must fields is needed and
effectively used to distinguish flows.
Which combination of properties is used for distinguishing flows and
how these properties are evaluated depends on the configuration of
the metering process. The configured choice of evaluated properties
strongly depends on the environment and purpose of the measurement
and on the information required by the collecting process. But in
any case, a collecting process must be able to clearly identify, for
each received flow record, which set of properties was used for
distinguishing this flow from other ones.
For specific deployments, only a subset of the required properties
listed below can be used to distinguish flows. For example, in order
to aggregate the flow records and reduce the number of flow records
exported. On the other hand, some other deployments will require
distinguishing flows by some extra parameters, such as the TTL field
of the IP header or the BGP Autonomous System number [RFC1771] of the
IP destination address.