| | | | | |
| |------->|Ingress | | |
| | | | | |
| | | Egress|--(Tunnel)-->| |
| | | | | |
+---------+ +-----------+ +---------+
Figure 3
Figure 3 shows the setup for testing the encapsulation throughput of
the DUT/SUT. One or more tunnels are created between each egress
interface of the DUT/SUT and a destination test port. Non-
Encapsulated multicast traffic will then be offered by the source
test port, encapsulated by the DUT/SUT and forwarded to the
destination test port(s).
The DUT/SUT SHOULD be configured such that the traffic across each
egress interface will consist of either:
a) A single tunnel encapsulating one or more multicast address
groups OR
b) Multiple tunnels, each encapsulating one or more multicast
address groups.
The number of multicast groups per tunnel MUST be the same when the
DUT/SUT is configured in a multiple tunnel configuration. In
addition, it is RECOMMENDED to test with the same number of tunnels
on each egress interface. All destination test ports MUST join all
multicast group addresses offered by the source test port. Each
egress interface MUST be configured with the same MTU.
Note: when offering large frames sizes, the encapsulation process may
require the DUT/SUT to fragment the IP datagrams prior to being
forwarded on the egress interface. It is RECOMMENDED to limit the
offered frame size such that no fragmentation is required by the
DUT/SUT.
A search algorithm MUST be utilized to determine the encapsulation
throughput as defined in [Du98].
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version
o Total number of multicast groups
o MTU size of DUT/SUT interfaces
o Originating un-encapsulated frame size
o Number of tunnels per egress interface
o Number of multicast groups per tunnel
o Encapsulation algorithm or format used to tunnel the
packets
The following results MUST be reflected in the test report:
o Measured Encapsulated Throughput as defined in RFC 2432
[Du98]
o Encapsulated frame size
The Encapsulated Throughput results SHOULD be reported in the form of
a table and specific to this test there SHOULD be rows for each
originating un-encapsulated frame size. Each row or iteration SHOULD
specify the offered load, encapsulation method, encapsulated frame
size, total number of offered frames, and the encapsulation
throughput.
4.4.2. Decapsulation Throughput
Objective:
To determine the maximum rate at which frames offered to one ingress
interface of a DUT/SUT are decapsulated and correctly forwarded by
the DUT/SUT on one or more egress interfaces without loss.
Procedure:
Source DUT/SUT Destination
Test Port Test Port(s)
+---------+ +-----------+ +---------+
| | | | | |
| | | Egress|------->| |
| | | | | |
| |--(Tunnel)-->|Ingress | | |
| | | | | |
| | | Egress|------->| |
| | | | | |
+---------+ +-----------+ +---------+
Figure 4
Figure 4 shows the setup for testing the decapsulation throughput of
the DUT/SUT. One or more tunnels are created between the source test
port and the DUT/SUT. Encapsulated multicast traffic will then be
offered by the source test port, decapsulated by the DUT/SUT and
forwarded to the destination test port(s).
The DUT/SUT SHOULD be configured such that the traffic across the
ingress interface will consist of either:
a) A single tunnel encapsulating one or more multicast address
groups OR
b) Multiple tunnels, each encapsulating one or more multicast
address groups.
The number of multicast groups per tunnel MUST be the same when the
DUT/SUT is configured in a multiple tunnel configuration. All
destination test ports MUST join all multicast group addresses
offered by the source test port. Each egress interface MUST be
configured with the same MTU.
A search algorithm MUST be utilized to determine the decapsulation
throughput as defined in [Du98].
When making performance comparisons between the encapsulation and
decapsulation process of the DUT/SUT, the offered frame sizes SHOULD
reflect the encapsulated frame sizes reported in the encapsulation
test (See section 4.4.1) in place of those noted in section 3.1.3.
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version
o Total number of multicast groups
o Originating encapsulation algorithm or format used to
tunnel the packets
o Originating encapsulated frame size
o Number of tunnels
o Number of multicast groups per tunnel
The following results MUST be reflected in the test report:
o Measured Decapsulated Throughput as defined in RFC 2432
[Du98]
o Decapsulated frame size
The Decapsulated Throughput results SHOULD be reported in the format
of a table and specific to this test there SHOULD be rows for each
originating encapsulated frame size. Each row or iteration SHOULD
specify the offered load, decapsulated frame size, total number of
offered frames and the decapsulation throughput.
4.4.3. Re-encapsulation Throughput
Objective:
To determine the maximum rate at which frames of one encapsulated
format offered to one ingress interface of a DUT/SUT are converted to
another encapsulated format and correctly forwarded by the DUT/SUT on
one or more egress interfaces without loss.
Procedure:
Source DUT/SUT Destination
Test Port Test Port(s)
+---------+ +---------+ +---------+
| | | | | |
| | | Egress|-(Tunnel)->| |
| | | | | |
| |-(Tunnel)->|Ingress | | |
| | | | | |
| | | Egress|-(Tunnel)->| |
| | | | | |
+---------+ +---------+ +---------+
Figure 5
Figure 5 shows the setup for testing the Re-encapsulation throughput
of the DUT/SUT. The source test port will offer encapsulated traffic
of one type to the DUT/SUT, which has been configured to re-
encapsulate the offered frames using a different encapsulation
format. The DUT/SUT will then forward the re-encapsulated frames to
the destination test port(s).
The DUT/SUT SHOULD be configured such that the traffic across the
ingress and each egress interface will consist of either:
a) A single tunnel encapsulating one or more multicast address
groups OR
b) Multiple tunnels, each encapsulating one or more multicast
address groups.
The number of multicast groups per tunnel MUST be the same when the
DUT/SUT is configured in a multiple tunnel configuration. In
addition, the DUT/SUT SHOULD be configured such that the number of
tunnels on the ingress and each egress interface are the same. All
destination test ports MUST join all multicast group addresses
offered by the source test port. Each egress interface MUST be
configured with the same MTU.
Note that when offering large frames sizes, the encapsulation process
may require the DUT/SUT to fragment the IP datagrams prior to being
forwarded on the egress interface. It is RECOMMENDED to limit the
offered frame sizes, such that no fragmentation is required by the
DUT/SUT.
A search algorithm MUST be utilized to determine the re-encapsulation
throughput as defined in [Du98].
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version
o Total number of multicast groups
o MTU size of DUT/SUT interfaces
o Originating encapsulation algorithm or format used to
tunnel the packets
o Re-encapsulation algorithm or format used to tunnel the
packets
o Originating encapsulated frame size
o Number of tunnels per interface
o Number of multicast groups per tunnel
The following results MUST be reflected in the test report:
o Measured Re-encapsulated Throughput as defined in RFC 2432
[Du98]
o Re-encapsulated frame size
The Re-encapsulated Throughput results SHOULD be reported in the
format of a table and specific to this test there SHOULD be rows for
each originating encapsulated frame size. Each row or iteration
SHOULD specify the offered load, Re-encapsulated frame size, total
number of offered frames, and the Re-encapsulated Throughput.
5. Forwarding Latency
This section presents methodologies relating to the characterization
of the forwarding latency of a DUT/SUT in a multicast environment.
It extends the concept of latency characterization presented in RFC
2544.
The offered load accompanying the latency-measured packet can affect
the DUT/SUT packet buffering, which may subsequently impact measured
packet latency. This SHOULD be a consideration when selecting the
intended load for the described methodologies below.
RFC 1242 and RFC 2544 draw a distinction between device types: "store
and forward" and "bit-forwarding." Each type impacts how latency is
collected and subsequently presented. See the related RFCs for more
information.
5.1. Multicast Latency
Objective:
To produce a set of multicast latency measurements from a single,
multicast ingress interface of a DUT/SUT through multiple, egress
multicast interfaces of that same DUT/SUT as provided for by the
metric "Multicast Latency" in RFC 2432 [Du98].
The procedures below draw from the collection methodology for latency
in RFC 2544 [Br96]. The methodology addresses two topological
scenarios: one for a single device (DUT) characterization; a second
scenario is presented or multiple device (SUT) characterization.
Procedure:
If the test trial is to characterize latency across a single Device
Under Test (DUT), an example test topology might take the form of
Figure 1 in section 3. That is, a single DUT with one ingress
interface receiving the multicast test traffic from frame-
transmitting component of the test apparatus and n egress interfaces
on the same DUT forwarding the multicast test traffic back to the
frame-receiving component of the test apparatus. Note that n
reflects the number of TESTED egress interfaces on the DUT actually
expected to forward the test traffic (as opposed to configured but
untested, non-forwarding interfaces, for example).
If the multicast latencies are to be taken across multiple devices
forming a System Under Test (SUT), an example test topology might
take the form of Figure 2 in section 3.
The trial duration SHOULD be 120 seconds to be consistent with RFC
2544 [Br96]. The nature of the latency measurement, "store and
forward" or "bit forwarding", MUST be associated with the related
test trial(s) and disclosed in the results report.
A test traffic stream is presented to the DUT. It is RECOMMENDED to
offer traffic at the measured aggregated multicast throughput rate
(Section 4.3). At the mid-point of the trial’s duration, the test
apparatus MUST inject a uniquely identifiable ("tagged") frame into
the test traffic frames being presented. This tagged frame will be
the basis for the latency measurements. By "uniquely identifiable",
it is meant that the test apparatus MUST be able to discern the
"tagged" frame from the other frames comprising the test traffic set.
A frame generation timestamp, Timestamp A, reflecting the completion
of the transmission of the tagged frame by the test apparatus, MUST
be determined.
The test apparatus will monitor frames from the DUT’s tested egress
interface(s) for the expected tagged frame(s) and MUST record the
time of the successful detection of a tagged frame from a tested
egress interface with a timestamp, Timestamp B. A set of Timestamp B
values MUST be collected for all tested egress interfaces of the
DUT/SUT. See RFC 1242 [Br91] for additional discussion regarding
store and forward devices and bit forwarding devices.
A trial MUST be considered INVALID should any of the following
conditions occur in the collection of the trial data:
o Unexpected differences between Intended Load and Offered
Load or unexpected differences between Offered Load and the
resulting Forwarding Rate(s) on the DUT/SUT egress ports.
o Forwarded test frames improperly formed or frame header
fields improperly manipulated.
o Failure to forward required tagged frame(s) on all expected
egress interfaces.
o Reception of tagged frames by the test apparatus more than
5 seconds after the cessation of test traffic by the source
test port.
The set of latency measurements, M, composed from each latency
measurement taken from every ingress/tested egress interface pairing
MUST be determined from a valid test trial:
M = { (Timestamp B(E0) - Timestamp A),
(Timestamp B(E1) - Timestamp A), ...
(Timestamp B(En) - Timestamp A) }
where (E0 ... En) represents the range of all tested egress
interfaces and Timestamp B represents a tagged frame detection event
for a given DUT/SUT tested egress interface.
A more continuous profile MAY be built from a series of individual
measurements.
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version
o Offered load
o Total number of multicast groups
The following results MUST be reflected in the test report:
o The set of all latencies with respective time units related
to the tested ingress and each tested egress DUT/SUT
interface.
The time units of the presented latency MUST be uniform and with
sufficient precision for the medium or media being tested.
The results MAY be offered in a tabular format and should preserve
the relationship of latency to ingress/egress interface for each
multicast group to assist in trending across multiple trials.
5.2. Min/Max Multicast Latency
Objective:
To determine the difference between the maximum latency measurement
and the minimum latency measurement from a collected set of latencies
produced by the Multicast Latency benchmark.
Procedure:
Collect a set of multicast latency measurements over a single test
duration, as prescribed in section 5.1. This will produce a set of
multicast latencies, M, where M is composed of individual forwarding
latencies between DUT frame ingress and DUT frame egress port pairs.
E.g.:
M = {L(I,E1),L(I,E2), ..., L(I,En)}
where L is the latency between a tested ingress interface, I, of the
DUT, and Ex a specific, tested multicast egress interface of the DUT.
E1 through En are unique egress interfaces on the DUT.
From the collected multicast latency measurements in set M, identify
MAX(M), where MAX is a function that yields the largest latency value
from set M.
Identify MIN(M), when MIN is a function that yields the smallest
latency value from set M.
The Max/Min value is determined from the following formula:
Result = MAX(M) - MIN(M)
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version
o Offered load
o Total number of multicast groups
The following results MUST be reflected in the test report:
o The Max/Min value
The following results SHOULD be reflected in the test report:
o The set of all latencies with respective time units related
to the tested ingress and each tested egress DUT/SUT
interface.
The time units of the presented latency MUST be uniform and with
sufficient precision for the medium or media being tested.
The results MAY be offered in a tabular format and should preserve
the relationship of latency to ingress/egress interface for each
multicast group.
6. Overhead
This section presents methodology relating to the characterization of
the overhead delays associated with explicit operations found in
multicast environments.
6.1. Group Join Delay
Objective:
To determine the time duration it takes a DUT/SUT to start forwarding
multicast frames from the time a successful IGMP group membership
report has been issued to the DUT/SUT.
Procedure:
The Multicast Group Join Delay measurement may be influenced by the
state of the Multicast Forwarding Database <MFDB> of the DUT/SUT. The
states of the MFDB may be described as follows:
o State 0, where the MFDB does not contain the specified
multicast group address. In this state, the delay measurement
includes the time the DUT/SUT requires to add the address to
the MFDB and begin forwarding. Delay measured from State 0
provides information about how the DUT/SUT is able to add new
addresses into MFDB.
o State 1, where the MFDB does contain the specified multicast
group address. In this state, the delay measurement includes
the time the DUT/SUT requires to update the MFDB with the
newly joined node<s> and begin forwarding to the new node<s>
plus packet replication time. Delay measured from State 1
provides information about how well the DUT/SUT is able to
update the MFDB for new nodes while transmitting packets to
other nodes for the same IP multicast address. Examples
include adding a new user to an event that is being promoted
via multicast packets.
The methodology for the Multicast Group Join Delay measurement
provides two alternate methods, based on the state of the MFDB, to
measure the delay metric. The methods MAY be used independently or
in conjunction to provide meaningful insight into the DUT/SUT ability
to manage the MFDB.
Users MAY elect to use either method to determine the Multicast Group
Join Delay; however the collection method MUST be specified as part
of the reporting format.
In order to minimize the variation in delay calculations as well as
minimize burden on the DUT/SUT, the test SHOULD be performed with one
multicast group. In addition, all destination test ports MUST join
the specified multicast group offered to the ingress interface of the
DUT/SUT.
Method A:
Method A assumes that the Multicast Forwarding Database <MFDB> of the
DUT/SUT does not contain or has not learned the specified multicast
group address; specifically, the MFDB MUST be in State 0. In this
scenario, the metric represents the time the DUT/SUT takes to add the
multicast address to the MFDB and begin forwarding the multicast
packet. Only one ingress and one egress MUST be used to determine
this metric.
Prior to sending any IGMP Group Membership Reports used to calculate
the Multicast Group Join Delay, it MUST be verified through
externally observable means that the destination test port is not
currently a member of the specified multicast group. In addition, it
MUST be verified through externally observable means that the MFDB of
the DUT/SUT does not contain the specified multicast address.
Method B:
Method B assumes that the MFDB of the DUT/SUT does contain the
specified multicast group address; specifically, the MFDB MUST be in
State 1. In this scenario, the metric represents the time the
DUT/SUT takes to update the MFDB with the additional nodes and their
corresponding interfaces and to begin forwarding the multicast
packet. One or more egress ports MAY be used to determine this
metric.
Prior to sending any IGMP Group Membership Reports used to calculate
the Group Join Delay, it MUST be verified through externally
observable means that the MFDB contains the specified multicast group
address. A single un-instrumented test port MUST be used to join the
specified multicast group address prior to sending any test traffic.
This port will be used only for insuring that the MFDB has been
populated with the specified multicast group address and can
successfully forward traffic to the un-instrumented port.
Join Delay Calculation
Once verification is complete, multicast traffic for the specified
multicast group address MUST be offered to the ingress interface