Request for Comments: 3954 Cisco Systems
Category: Informational October 2004
Cisco Systems NetFlow Services Export Version 9
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2004).
IESG Note
This RFC documents the NetFlow services export protocol Version 9 as
it was when submitted to the IETF as a basis for further work in the
IPFIX WG.
This RFC itself is not a candidate for any level of Internet
Standard. The IETF disclaims any knowledge of the fitness of this
RFC for any purpose, and in particular notes that it has not had
complete IETF review for such things as security, congestion control,
or inappropriate interaction with deployed protocols. The RFC Editor
has chosen to publish this document at its discretion.
Abstract
This document specifies the data export format for version 9 of Cisco
Systems’ NetFlow services, for use by implementations on the network
elements and/or matching collector programs. The version 9 export
format uses templates to provide access to observations of IP packet
flows in a flexible and extensible manner. A template defines a
collection of fields, with corresponding descriptions of structure
and semantics.
Table of Contents
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Terminology Summary Table . . . . . . . . . . . . . . . 6
3. NetFlow High-Level Picture on the Exporter. . . . . . . . . . 6
3.1. The NetFlow Process on the Exporter . . . . . . . . . . 6
3.2. Flow Expiration . . . . . . . . . . . . . . . . . . . . 7
3.3. Transport Protocol. . . . . . . . . . . . . . . . . . . 7
4. Packet Layout . . . . . . . . . . . . . . . . . . . . . . . . 8
5. Export Packet Format. . . . . . . . . . . . . . . . . . . . . 9
5.1. Header Format . . . . . . . . . . . . . . . . . . . . . 9
5.2. Template FlowSet Format . . . . . . . . . . . . . . . . 11
5.3. Data FlowSet Format . . . . . . . . . . . . . . . . . . 13
6. Options . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
6.1. Options Template FlowSet Format . . . . . . . . . . . . 14
6.2. Options Data Record Format. . . . . . . . . . . . . . . 16
7. Template Management . . . . . . . . . . . . . . . . . . . . . 17
8. Field Type Definitions. . . . . . . . . . . . . . . . . . . . 18
9. The Collector Side. . . . . . . . . . . . . . . . . . . . . . 25
10. Security Considerations . . . . . . . . . . . . . . . . . . . 26
10.1. Disclosure of Flow Information Data . . . . . . . . . . 26
10.2. Forgery of Flow Records or Template Records . . . . . . 26
10.3. Attacks on the NetFlow Collector. . . . . . . . . . . . 27
11. Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . 27
11.1. Packet Header Example . . . . . . . . . . . . . . . . . 28
11.2. Template FlowSet Example. . . . . . . . . . . . . . . . 28
11.3. Data FlowSet Example. . . . . . . . . . . . . . . . . . 29
11.4. Options Template FlowSet Example. . . . . . . . . . . . 30
11.5. Data FlowSet with Options Data Records Example. . . . . 30
12. References. . . . . . . . . . . . . . . . . . . . . . . . . . 31
12.1. Normative References. . . . . . . . . . . . . . . . . . 31
12.2. Informative References. . . . . . . . . . . . . . . . . 31
13. Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 31
15. Authors’ Addresses. . . . . . . . . . . . . . . . . . . . . . 32
16. Full Copyright Statement. . . . . . . . . . . . . . . . . . . 33
1. Introduction
Cisco Systems’ NetFlow services provide network administrators with
access to IP flow information from their data networks. Network
elements (routers and switches) gather flow data and export it to
collectors. The collected data provides fine-grained metering for
highly flexible and detailed resource usage accounting.
A flow is defined as a unidirectional sequence of packets with some
common properties that pass through a network device. These
collected flows are exported to an external device, the NetFlow
collector. Network flows are highly granular; for example, flow
records include details such as IP addresses, packet and byte counts,
timestamps, Type of Service (ToS), application ports, input and
output interfaces, etc.
Exported NetFlow data is used for a variety of purposes, including
enterprise accounting and departmental chargebacks, ISP billing, data
warehousing, network monitoring, capacity planning, application
monitoring and profiling, user monitoring and profiling, security
analysis, and data mining for marketing purposes.
This document specifies NetFlow version 9. It describes the
implementation specifications both from network element and NetFlow
collector points of view. These specifications should help the
deployment of NetFlow version 9 across different platforms and
different vendors by limiting the interoperability risks. The
NetFlow export format version 9 uses templates to provide access to
observations of IP packet flows in a flexible and extensible manner.
A template defines a collection of fields, with corresponding
descriptions of structure and semantics.
The template-based approach provides the following advantages:
- New fields can be added to NetFlow flow records without
changing the structure of the export record format. With
previous NetFlow versions, adding a new field in the flow
record implied a new version of the export protocol format and
a new version of the NetFlow collector that supported the
parsing of the new export protocol format.
- Templates that are sent to the NetFlow collector contain the
structural information about the exported flow record fields;
therefore, if the NetFlow collector does not understand the
semantics of new fields, it can still interpret the flow
record.
- Because the template mechanism is flexible, it allows the
export of only the required fields from the flows to the
NetFlow collector. This helps to reduce the exported flow data
volume and provides possible memory savings for the exporter
and NetFlow collector. Sending only the required information
can also reduce network load.
The IETF IPFIX Working Group (IP Flow Information eXport) is
developing a new protocol, based on the version 9 of Cisco Systems’
NetFlow services. Some enhancements in different domains (congestion
aware transport protocol, built-in security, etc... ) have been
incorporated in this new IPFIX protocol. Refer to the IPFIX Working
Group documents for more details.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119
[RFC2119].
2. Terminology
Various terms used in this document are described in this section.
Note that the terminology summary table in Section 2.1 gives a quick
overview of the relationships between some of the different terms
defined.
Observation Point
An Observation Point is a location in the network where IP packets
can be observed; for example, one or a set of interfaces on a network
device like a router. Every Observation Point is associated with an
Observation Domain.
Observation Domain
The set of Observation Points that is the largest aggregatable set of
flow information at the network device with NetFlow services enabled
is termed an Observation Domain. For example, a router line card
composed of several interfaces with each interface being an
Observation Point.
IP Flow or Flow
An IP Flow, also called a Flow, is defined as a set of IP packets
passing an Observation Point in the network during a certain time
interval. All packets that belong to a particular Flow have a set of
common properties derived from the data contained in the packet and
from the packet treatment at the Observation Point.
Flow Record
A Flow Record provides information about an IP Flow observed at an
Observation Point. In this document, the Flow Data Records are also
referred to as NetFlow services data and NetFlow data.
Exporter
A device (for example, a router) with the NetFlow services enabled,
the Exporter monitors packets entering an Observation Point and
creates Flows from these packets. The information from these Flows
is exported in the form of Flow Records to the NetFlow Collector.
NetFlow Collector
The NetFlow Collector receives Flow Records from one or more
Exporters. It processes the received Export Packet(s); that is, it
parses and stores the Flow Record information. Flow Records can be
optionally aggregated before being stored on the hard disk. The
NetFlow Collector is also referred to as the Collector in this
document.
Export Packet
An Export Packet is a packet originating at the Exporter that carries
the Flow Records of this Exporter and whose destination is the
NetFlow Collector.
Packet Header
The Packet Header is the first part of an Export Packet. The Packet
Header provides basic information about the packet such as the
NetFlow version, number of records contained within the packet, and
sequence numbering.
Template Record
A Template Record defines the structure and interpretation of fields
in a Flow Data Record.
Flow Data Record
A Flow Data Record is a data record that contains values of the Flow
parameters corresponding to a Template Record.
Options Template Record
An Options Template Record defines the structure and interpretation
of fields in an Options Data Record, including defining the scope
within which the Options Data Record is relevant.
Options Data Record
The data record that contains values and scope information of the
Flow measurement parameters, corresponding to an Options Template
Record.
FlowSet
FlowSet is a generic term for a collection of Flow Records that have
a similar structure. In an Export Packet, one or more FlowSets
follow the Packet Header. There are three different types of
FlowSets: Template FlowSet, Options Template FlowSet, and Data
FlowSet.
Template FlowSet
A Template FlowSet is one or more Template Records that have been
grouped together in an Export Packet.
Options Template FlowSet
An Options Template FlowSet is one or more Options Template Records
that have been grouped together in an Export Packet.
Data FlowSet
A Data FlowSet is one or more records, of the same type, that are
grouped together in an Export Packet. Each record is either a Flow
Data Record or an Options Data Record previously defined by a
Template Record or an Options Template Record.
2.1. Terminology Summary Table
+------------------+---------------------------------------------+
| | Contents |
| +--------------------+------------------------+
| FlowSet | Template Record | Data Record |
+------------------+--------------------+------------------------+
| | | Flow Data Record(s) |
| Data FlowSet | / | or |
| | | Options Data Record(s) |
+------------------+--------------------+------------------------+
| Template FlowSet | Template Record(s) | / |
+------------------+--------------------+------------------------+
| Options Template | Options Template | / |
| FlowSet | Record(s) | |
+------------------+--------------------+------------------------+
A Data FlowSet is composed of an Options Data Record(s) or Flow Data
Record(s). No Template Record is included. A Template Record defines
the Flow Data Record, and an Options Template Record defines the
Options Data Record.
A Template FlowSet is composed of Template Record(s). No Flow or
Options Data Record is included.
An Options Template FlowSet is composed of Options Template
Record(s). No Flow or Options Data Record is included.
3. NetFlow High-Level Picture on the Exporter
3.1. The NetFlow Process on the Exporter
The NetFlow process on the Exporter is responsible for the creation
of Flows from the observed IP packets. The details of this process
are beyond the scope of this document.
3.2. Flow Expiration
A Flow is considered to be inactive if no packets belonging to the
Flow have been observed at the Observation Point for a given timeout.
If any packet is seen within the timeout, the flow is considered an
active flow. A Flow can be exported under the following conditions:
1. If the Exporter can detect the end of a Flow. For example, if
the FIN or RST bit is detected in a TCP [RFC793] connection,
the Flow Record is exported.
2. If the Flow has been inactive for a certain period of time.
This inactivity timeout SHOULD be configurable at the Exporter,
with a minimum value of 0 for an immediate expiration.
3. For long-lasting Flows, the Exporter SHOULD export the Flow
Records on a regular basis. This timeout SHOULD be
configurable at the Exporter.
4. If the Exporter experiences internal constraints, a Flow MAY be
forced to expire prematurely; for example, counters wrapping or
low memory.
3.3. Transport Protocol
To achieve efficiency in terms of processing at the Exporter while
handling high volumes of Export Packets, the NetFlow Export Packets
are encapsulated into UDP [RFC768] datagrams for export to the
NetFlow Collector. However, NetFlow version 9 has been designed to
be transport protocol independent. Hence, it can also operate over
congestion-aware protocols such as SCTP [RFC2960].
Note that the Exporter can export to multiple Collectors, using
independent transport protocols.
UDP [RFC768] is a non congestion-aware protocol, so when deploying
NetFlow version 9 in a congestion-sensitive environment, make the
connection between Exporter and NetFlow Collector through a dedicated
link. This ensures that any burstiness in the NetFlow traffic
affects only this dedicated link. When the NetFlow Collector can not
be placed within a one-hop distance from the Exporter or when the
export path from the Exporter to the NetFlow Collector can not be
exclusively used for the NetFlow Export Packets, the export path
should be designed so that it can always sustain the maximum
burstiness of NetFlow traffic from the Exporter. Note that the
congestion can occur on the Exporter in case the export path speed is
too low.
4. Packet Layout
An Export Packet consists of a Packet Header followed by one or more
FlowSets. The FlowSets can be any of the possible three types:
Template, Data, or Options Template.
+--------+-------------------------------------------+
| | +----------+ +---------+ +----------+ |
| Packet | | Template | | Data | | Options | |
| Header | | FlowSet | | FlowSet | | Template | ... |
| | | | | | | FlowSet | |
| | +----------+ +---------+ +----------+ |
+--------+-------------------------------------------+
Export Packet
A FlowSet ID is used to distinguish the different types of FlowSets.
FlowSet IDs lower than 256 are reserved for special FlowSets, such as
the Template FlowSet (ID 0) and the Options Template FlowSet (ID 1).
The Data FlowSets have a FlowSet ID greater than 255.
The format of the Template, Data, and Options Template FlowSets will
be discussed later in this document. The Exporter MUST code all
binary integers of the Packet Header and the different FlowSets in
network byte order (also known as the big-endian byte ordering).
Following are some examples of export packets:
1. An Export Packet consisting of interleaved Template, Data, and
Options Template FlowSets. Example: a newly created Template is
exported as soon as possible. So if there is already an Export
Packet with a Data FlowSet that is being prepared for export, the
Template and Option FlowSets are also interleaved with this
information, subject to availability of space.
Export Packet:
+--------+--------------------------------------------------------+
| | +----------+ +---------+ +-----------+ +---------+ |
| Packet | | Template | | Data | | Options | | Data | |
| Header | | FlowSet | | FlowSet | ... | Template | | FlowSet | |
| | | | | | | FlowSet | | | |
| | +----------+ +---------+ +-----------+ +---------+ |
+--------+--------------------------------------------------------+
2. An Export Packet consisting entirely of Data FlowSets. Example:
after the appropriate Template Records have been defined and
transmitted to the NetFlow Collector device, the majority of
Export Packets consists solely of Data FlowSets.
Export Packet:
+--------+----------------------------------------------+
| | +---------+ +---------+ +---------+ |
| Packet | | Data | ... | Data | ... | Data | |
| Header | | FlowSet | ... | FlowSet | ... | FlowSet | |
| | +---------+ +---------+ +---------+ |
+--------+----------------------------------------------+
3. An Export Packet consisting entirely of Template and Options
Template FlowSets. Example: the Exporter MAY transmit a packet
containing Template and Options Template FlowSets periodically to
help ensure that the NetFlow Collector has the correct Template
Records and Options Template Records when the corresponding Flow
Data records are received.
Export Packet:
+--------+-------------------------------------------------+
| | +----------+ +----------+ +----------+ |
| Packet | | Template | | Template | | Options | |
| Header | | FlowSet | ... | FlowSet | ... | Template | |
| | | | | | | FlowSet | |
| | +----------+ +----------+ +----------+ |
+--------+-------------------------------------------------+
5. Export Packet Format
5.1. Header Format
The Packet Header format is specified as:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version Number | Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sysUpTime |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| UNIX Secs |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Packet Header Field Descriptions
Version
Version of Flow Record format exported in this packet. The
value of this field is 9 for the current version.
Count
The total number of records in the Export Packet, which is the
sum of Options FlowSet records, Template FlowSet records, and
Data FlowSet records.
sysUpTime
Time in milliseconds since this device was first booted.
UNIX Secs
Time in seconds since 0000 UTC 1970, at which the Export Packet
leaves the Exporter.
Sequence Number
Incremental sequence counter of all Export Packets sent from
the current Observation Domain by the Exporter. This value
MUST be cumulative, and SHOULD be used by the Collector to
identify whether any Export Packets have been missed.
Source ID
A 32-bit value that identifies the Exporter Observation Domain.
NetFlow Collectors SHOULD use the combination of the source IP
address and the Source ID field to separate different export
streams originating from the same Exporter.
5.2. Template FlowSet Format
One of the essential elements in the NetFlow format is the Template
FlowSet. Templates greatly enhance the flexibility of the Flow
Record format because they allow the NetFlow Collector to process
Flow Records without necessarily knowing the interpretation of all