prior to the DUT/SUT receiving any IGMP Group Membership Report
messages. It is RECOMMENDED to offer traffic at the measured
aggregated multicast throughput rate (Section 4.3).
After the multicast traffic has been started, the destination test
port (See Figure 1) MUST send one IGMP Group Membership Report for
the specified multicast group.
The join delay is the difference in time from when the IGMP Group
Membership message is sent (timestamp A) and the first frame of the
multicast group is forwarded to a receiving egress interface
(timestamp B).
Group Join delay time = timestamp B - timestamp A
Timestamp A MUST be the time the last bit of the IGMP group
membership report is sent from the destination test port; timestamp B
MUST be the time the first bit of the first valid multicast frame is
forwarded on the egress interface of the DUT/SUT.
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o IGMP version
o Total number of multicast groups
o Offered load to ingress interface
o Method used to measure the join delay metric
The following results MUST be reflected in the test report:
o The group join delay time in microseconds per egress
interface(s)
The Group Join Delay results for each test MAY be reported in the
form of a table, with a row for each of the tested frame sizes per
the recommendations in section 3.1.3. Each row or iteration MAY
specify the group join delay time per egress interface for that
iteration.
6.2. Group Leave Delay
Objective:
To determine the time duration it takes a DUT/SUT to cease forwarding
multicast frames after a corresponding IGMP Leave Group message has
been successfully offered to the DUT/SUT.
Procedure:
In order to minimize the variation in delay calculations as well as
minimize burden on the DUT/SUT, the test SHOULD be performed with one
multicast group. In addition, all destination test ports MUST join
the specified multicast group offered to the ingress interface of the
DUT/SUT.
Prior to sending any IGMP Leave Group messages used to calculate the
group leave delay, it MUST be verified through externally observable
means that the destination test ports are currently members of the
specified multicast group. If any of the egress interfaces do not
forward validation multicast frames then the test is invalid.
Once verification is complete, multicast traffic for the specified
multicast group address MUST be offered to the ingress interface
prior to receipt or processing of any IGMP Leave Group messages. It
is RECOMMENDED to offer traffic at the measured aggregated multicast
throughput rate (Section 4.3).
After the multicast traffic has been started, each destination test
port (See Figure 1) MUST send one IGMP Leave Group message for the
specified multicast group.
The leave delay is the difference in time from when the IGMP Leave
Group message is sent (timestamp A) and the last frame of the
multicast group is forwarded to a receiving egress interface
(timestamp B).
Group Leave delay time = timestamp B - timestamp A
Timestamp A MUST be the time the last bit of the IGMP Leave Group
message is sent from the destination test port; timestamp B MUST be
the time the last bit of the last valid multicast frame is forwarded
on the egress interface of the DUT/SUT.
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o IGMP version
o Total number of multicast groups
o Offered load to ingress interface
The following results MUST be reflected in the test report:
o The group leave delay time in microseconds per egress
interface(s)
The Group Leave Delay results for each test MAY be reported in the
form of a table, with a row for each of the tested frame sizes per
the recommendations in section 3.1.3. Each row or iteration MAY
specify the group leave delay time per egress interface for that
iteration.
7. Capacity
This section offers a procedure relating to the identification of
multicast group limits of a DUT/SUT.
7.1. Multicast Group Capacity
Objective:
To determine the maximum number of multicast groups a DUT/SUT can
support while maintaining the ability to forward multicast frames to
all multicast groups registered to that DUT/SUT.
Procedure:
One or more destination test ports of DUT/SUT will join an initial
number of multicast groups.
After a minimum delay as measured by section 6.1, the source test
ports MUST transmit to each group at a specified offered load.
If at least one frame for each multicast group is forwarded properly
by the DUT/SUT on each participating egress interface, the iteration
is said to pass at the current capacity.
For each successful iteration, each destination test port will join
an additional user-defined number of multicast groups and the test
repeats. The test stops iterating when one or more of the egress
interfaces fails to forward traffic on one or more of the configured
multicast groups.
Once the iteration fails, the last successful iteration is the stated
Maximum Group Capacity result.
Reporting Format:
The following configuration parameters MUST be reflected in the test
report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o IGMP version
o Offered load
The following results MUST be reflected in the test report:
o The total number of multicast group addresses that were
successfully forwarded through the DUT/SUT
The Multicast Group Capacity results for each test SHOULD be reported
in the form of a table, with a row for each of the tested frame sizes
per the recommendations in section 3.1.3. Each row or iteration
SHOULD specify the number of multicast groups joined per destination
interface, number of frames transmitted and number of frames received
for that iteration.
8. Interaction
Network forwarding devices are generally required to provide more
functionality than just the forwarding of traffic. Moreover,
network-forwarding devices may be asked to provide those functions in
a variety of environments. This section offers procedures to assist
in the characterization of DUT/SUT behavior in consideration of
potentially interacting factors.
8.1. Forwarding Burdened Multicast Latency
Objective:
To produce a set of multicast latency measurements from a single
multicast ingress interface of a DUT/SUT through multiple egress
multicast interfaces of that same DUT/SUT as provided for by the
metric "Multicast Latency" in RFC 2432 [Du98] while forwarding meshed
unicast traffic.
Procedure:
The Multicast Latency metrics can be influenced by forcing the
DUT/SUT to perform extra processing of packets while multicast class
traffic is being forwarded for latency measurements.
The Burdened Forwarding Multicast Latency test MUST follow the
described setup for the Multicast Latency test in Section 5.1. In
addition, another set of test ports MUST be used to burden the
DUT/SUT (burdening ports). The burdening ports will be used to
transmit unicast class traffic to the DUT/SUT in a fully meshed
traffic distribution as described in RFC 2285 [Ma98]. The DUT/SUT
MUST learn the appropriate unicast addresses and verified through
some externally observable method.
Perform a baseline measurement of Multicast Latency as described in
Section 5.1. After the baseline measurement is obtained, start
transmitting the unicast class traffic at a user-specified offered
load on the set of burdening ports and rerun the Multicast Latency
test. The offered load to the ingress port MUST be the same as was
used in the baseline measurement.
Reporting Format:
Similar to Section 5.1, the following configuration parameters MUST
be reflected in the test report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version
o Offered load to ingress interface
o Total number of multicast groups
o Offered load to burdening ports
o Total number of burdening ports
The following results MUST be reflected in the test report:
o The set of all latencies related to the tested ingress and
each tested egress DUT/SUT interface for both the baseline
and burdened response.
The time units of the presented latency MUST be uniform and with
sufficient precision for the medium or media being tested.
The latency results for each test SHOULD be reported in the form of a
table, with a row for each of the tested frame sizes per the
recommended frame sizes in section 3.1.3, and SHOULD preserve the
relationship of latency to ingress/egress interface(s) to assist in
trending across multiple trials.
8.2. Forwarding Burdened Group Join Delay
Objective:
To determine the time duration it takes a DUT/SUT to start forwarding
multicast frames from the time a successful IGMP Group Membership
Report has been issued to the DUT/SUT while forwarding meshed unicast
traffic.
Procedure:
The Forwarding Burdened Group Join Delay test MUST follow the
described setup for the Group Join Delay test in Section 6.1. In
addition, another set of test ports MUST be used to burden the
DUT/SUT (burdening ports). The burdening ports will be used to
transmit unicast class traffic to the DUT/SUT in a fully meshed
traffic pattern as described in RFC 2285 [Ma98]. The DUT/SUT MUST
learn the appropriate unicast addresses and verified through some
externally observable method.
Perform a baseline measurement of Group Join Delay as described in
Section 6.1. After the baseline measurement is obtained, start
transmitting the unicast class traffic at a user-specified offered
load on the set of burdening ports and rerun the Group Join Delay
test. The offered load to the ingress port MUST be the same as was
used in the baseline measurement.
Reporting Format:
Similar to Section 6.1, the following configuration parameters MUST
be reflected in the test report:
o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT
o IGMP version
o Offered load to ingress interface
o Total number of multicast groups
o Offered load to burdening ports
o Total number of burdening ports
o Method used to measure the join delay metric
The following results MUST be reflected in the test report:
o The group join delay time in microseconds per egress
interface(s) for both the baseline and burdened response.
The Group Join Delay results for each test MAY be reported in the
form of a table, with a row for each of the tested frame sizes per
the recommendations in section 3.1.3. Each row or iteration MAY
specify the group join delay time per egress interface, number of
frames transmitted and number of frames received for that iteration.
9. Security Considerations
As this document is solely for the purpose of providing metric
methodology and describes neither a protocol nor a protocol’s
implementation, there are no security considerations associated with
this document specifically. Results from these methodologies may
identify a performance capability or limit of a device or system in a
particular test context. However, such results might not be
representative of the tested entity in an operational network.
10. Acknowledgements
The Benchmarking Methodology Working Group of the IETF and
particularly Kevin Dubray, Juniper Networks, are to be thanked for
the many suggestions they collectively made to help complete this
document.
11. Contributions
The authors would like to acknowledge the following individuals for
their help and participation of the compilation of this document:
Hardev Soor, Ixia, and Ralph Daniels, Spirent Communications, both
who made significant contributions to the earlier versions of this
document. In addition, the authors would like to acknowledge the
members of the task team who helped bring this document to fruition:
Michele Bustos, Tony De La Rosa, David Newman and Jerry Perser.
12. References
12.1. Normative References
[Br91] Bradner, S., "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, July 1991.
[Br96] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999.
[Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement
Levels, RFC 2119, March 1997.
[Du98] Dubray, K., "Terminology for IP Multicast Benchmarking", RFC
2432, October 1998.
[IANA1] IANA multicast address assignments,
http://www.iana.org/assignments/multicast-addresses
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, February 1998.
[Me98] Meyer, D., "Administratively Scoped IP Multicast", BCP 23,
RFC 2365, July 1998.
12.2. Informative References
[Ca02] Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A.
Thyagarajan, "Internet Group Management Protocol, Version
3", RFC 3376, October 2002.
[De89] Deering, S., "Host Extensions for IP Multicasting", STD 5,
RFC 1112, August 1989.
[Fe97] Fenner, W., "Internet Group Management Protocol, Version 2",
RFC 2236, November 1997.
[Hu95] Huitema, C., "Routing in the Internet", Prentice-Hall, 1995.
[Ka98] Kosiur, D., "IP Multicasting: the Complete Guide to
Interactive Corporate Networks", John Wiley & Sons Inc.,
1998.
[Mt98] Maufer, T., "Deploying IP Multicast in the Enterprise",
Prentice-Hall, 1998.
13. Authors’ Addresses
Debra Stopp
Ixia
26601 W. Agoura Rd.
Calabasas, CA 91302
USA
Phone: + 1 818 871 1800
EMail: debby@ixiacom.com
Brooks Hickman
Spirent Communications
26750 Agoura Rd.
Calabasas, CA 91302
USA
Phone: + 1 818 676 2412
EMail: brooks.hickman@spirentcom.com
14. Full Copyright Statement
Copyright (C) The Internet Society (2004).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/S HE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the IETF’s procedures with respect to rights in IETF Documents can
be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.