Request for Comments: 4585 Helsinki University of Technology
Category: Standards Track S. Wenger
Nokia
N. Sato
Oki
C. Burmeister
J. Rey
Matsushita
July 2006
Extended RTP Profile for
Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)
Status of This Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
Real-time media streams that use RTP are, to some degree, resilient
against packet losses. Receivers may use the base mechanisms of the
Real-time Transport Control Protocol (RTCP) to report packet
reception statistics and thus allow a sender to adapt its
transmission behavior in the mid-term. This is the sole means for
feedback and feedback-based error repair (besides a few codec-
specific mechanisms). This document defines an extension to the
Audio-visual Profile (AVP) that enables receivers to provide,
statistically, more immediate feedback to the senders and thus allows
for short-term adaptation and efficient feedback-based repair
mechanisms to be implemented. This early feedback profile (AVPF)
maintains the AVP bandwidth constraints for RTCP and preserves
scalability to large groups.
Table of Contents
1. Introduction ....................................................3
1.1. Definitions ................................................3
1.2. Terminology ................................................5
2. RTP and RTCP Packet Formats and Protocol Behavior ...............6
2.1. RTP ........................................................6
2.2. Underlying Transport Protocols .............................6
3. Rules for RTCP Feedback .........................................7
3.1. Compound RTCP Feedback Packets .............................7
3.2. Algorithm Outline ..........................................8
3.3. Modes of Operation .........................................9
3.4. Definitions and Algorithm Overview ........................11
3.5. AVPF RTCP Scheduling Algorithm ............................14
3.5.1. Initialization .....................................15
3.5.2. Early Feedback Transmission ........................15
3.5.3. Regular RTCP Transmission ..........................18
3.5.4. Other Considerations ...............................19
3.6. Considerations on the Group Size ..........................20
3.6.1. ACK Mode ...........................................20
3.6.2. NACK Mode ..........................................20
3.7. Summary of Decision Steps .................................22
3.7.1. General Hints ......................................22
3.7.2. Media Session Attributes ...........................22
4. SDP Definitions ................................................23
4.1. Profile Identification ....................................23
4.2. RTCP Feedback Capability Attribute ........................23
4.3. RTCP Bandwidth Modifiers ..................................27
4.4. Examples ..................................................27
5. Interworking and Coexistence of AVP and AVPF Entities ..........29
6. Format of RTCP Feedback Messages ...............................31
6.1. Common Packet Format for Feedback Messages ................32
6.2. Transport Layer Feedback Messages .........................34
6.2.1. Generic NACK .......................................34
6.3. Payload-Specific Feedback Messages ........................35
6.3.1. Picture Loss Indication (PLI) ......................36
6.3.2. Slice Loss Indication (SLI) ........................37
6.3.3. Reference Picture Selection Indication (RPSI) ......39
6.4. Application Layer Feedback Messages .......................41
7. Early Feedback and Congestion Control ..........................41
8. Security Considerations ........................................42
9. IANA Considerations ............................................43
10. Acknowledgements ..............................................47
11. References ....................................................48
11.1. Normative References .....................................48
11.2. Informative References ...................................48
1. Introduction
Real-time media streams that use RTP are, to some degree, resilient
against packet losses. RTP [1] provides all the necessary mechanisms
to restore ordering and timing present at the sender to properly
reproduce a media stream at a recipient. RTP also provides
continuous feedback about the overall reception quality from all
receivers -- thereby allowing the sender(s) in the mid-term (in the
order of several seconds to minutes) to adapt their coding scheme and
transmission behavior to the observed network quality of service
(QoS). However, except for a few payload-specific mechanisms [6],
RTP makes no provision for timely feedback that would allow a sender
to repair the media stream immediately: through retransmissions,
retroactive Forward Error Correction (FEC) control, or media-specific
mechanisms for some video codecs, such as reference picture
selection.
Current mechanisms available with RTP to improve error resilience
include audio redundancy coding [13], video redundancy coding [14],
RTP-level FEC [11], and general considerations on more robust media
streams transmission [12]. These mechanisms may be applied
proactively (thereby increasing the bandwidth of a given media
stream). Alternatively, in sufficiently small groups with small
round-trip times (RTTs), the senders may perform repair on-demand,
using the above mechanisms and/or media-encoding-specific approaches.
Note that "small group" and "sufficiently small RTT" are both highly
application dependent.
This document specifies a modified RTP profile for audio and video
conferences with minimal control based upon [1] and [2] by means of
two modifications/additions: Firstly, to achieve timely feedback, the
concept of Early RTCP messages as well as algorithms allowing for
low-delay feedback in small multicast groups (and preventing feedback
implosion in large ones) are introduced. Special consideration is
given to point-to-point scenarios. Secondly, a small number of
general-purpose feedback messages as well as a format for codec- and
application-specific feedback information are defined for
transmission in the RTCP payloads.
1.1. Definitions
The definitions from RTP/RTCP [1] and the "RTP Profile for Audio and
Video Conferences with Minimal Control" [2] apply. In addition, the
following definitions are used in this document:
Early RTCP mode:
The mode of operation in that a receiver of a media stream is
often (but not always) capable of reporting events of interest
back to the sender close to their occurrence. In Early RTCP mode,
RTCP packets are transmitted according to the timing rules defined
in this document.
Early RTCP packet:
An Early RTCP packet is a packet which is transmitted earlier than
would be allowed if following the scheduling algorithm of [1], the
reason being an "event" observed by a receiver. Early RTCP
packets may be sent in Immediate Feedback and in Early RTCP mode.
Sending an Early RTCP packet is also referred to as sending Early
Feedback in this document.
Event:
An observation made by the receiver of a media stream that is
(potentially) of interest to the sender -- such as a packet loss
or packet reception, frame loss, etc. -- and thus useful to be
reported back to the sender by means of a feedback message.
Feedback (FB) message:
An RTCP message as defined in this document is used to convey
information about events observed at a receiver -- in addition to
long-term receiver status information that is carried in RTCP
receiver reports (RRs) -- back to the sender of the media stream.
For the sake of clarity, feedback message is referred to as FB
message throughout this document.
Feedback (FB) threshold:
The FB threshold indicates the transition between Immediate
Feedback and Early RTCP mode. For a multiparty scenario, the FB
threshold indicates the maximum group size at which, on average,
each receiver is able to report each event back to the sender(s)
immediately, i.e., by means of an Early RTCP packet without having
to wait for its regularly scheduled RTCP interval. This threshold
is highly dependent on the type of feedback to be provided,
network QoS (e.g., packet loss probability and distribution),
codec and packetization scheme in use, the session bandwidth, and
application requirements. Note that the algorithms do not depend
on all senders and receivers agreeing on the same value for this
threshold. It is merely intended to provide conceptual guidance
to application designers and is not used in any calculations. For
the sake of clarity, the term feedback threshold is referred to as
FB threshold throughout this document.
Immediate Feedback mode:
A mode of operation in which each receiver of a media stream is,
statistically, capable of reporting each event of interest
immediately back to the media stream sender. In Immediate
Feedback mode, RTCP FB messages are transmitted according to the
timing rules defined in this document.
Media packet:
A media packet is an RTP packet.
Regular RTCP mode:
Mode of operation in which no preferred transmission of FB
messages is allowed. Instead, RTCP messages are sent following
the rules of [1]. Nevertheless, such RTCP messages may contain
feedback information as defined in this document.
Regular RTCP packet:
An RTCP packet that is not sent as an Early RTCP packet.
RTP sender:
An RTP sender is an RTP entity that transmits media packets as
well as RTCP packets and receives Regular as well as Early RTCP
(i.e., feedback) packets. Note that the RTP sender is a logical
role and that the same RTP entity may at the same time act as an
RTP receiver.
RTP receiver:
An RTP receiver is an RTP entity that receives media packets as
well as RTCP packets and transmits Regular as well as Early RTCP
(i.e., feedback) packets. Note that the RTP receiver is a logical
role and that the same RTP entity may at the same time act as an
RTP sender.
1.2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [5].
2. RTP and RTCP Packet Formats and Protocol Behavior
2.1. RTP
The rules defined in [2] also apply to this profile except for those
rules mentioned in the following:
RTCP packet types:
Two additional RTCP packet types are registered and the
corresponding FB messages to convey feedback information are
defined in Section 6 of this memo.
RTCP report intervals:
This document describes three modes of operation that influence
the RTCP report intervals (see Section 3.2 of this memo). In
Regular RTCP mode, all rules from [1] apply except for the
recommended minimal interval of five seconds between two RTCP
reports from the same RTP entity. In both Immediate Feedback and
Early RTCP modes, the minimal interval of five seconds between two
RTCP reports is dropped and, additionally, the rules specified in
Section 3 of this memo apply if RTCP packets containing FB
messages (defined in Section 4 of this memo) are to be
transmitted.
The rules set forth in [1] may be overridden by session
descriptions specifying different parameters (e.g., for the
bandwidth share assigned to RTCP for senders and receivers,
respectively). For sessions defined using the Session Description
Protocol (SDP) [3], the rules of [4] apply.
Congestion control:
The same basic rules as detailed in [2] apply. Beyond this, in
Section 7, further consideration is given to the impact of
feedback and a sender’s reaction to FB messages.
2.2. Underlying Transport Protocols
RTP is intended to be used on top of unreliable transport protocols,
including UDP and the Datagram Congestion Control Protocol (DCCP).
This section briefly describes the specifics beyond plain RTP
operation introduced by RTCP feedback as specified in this memo.
UDP: UDP provides best-effort delivery of datagrams for point-to-
point as well as for multicast communications. UDP does not
support congestion control or error repair. The RTCP-based
feedback defined in this memo is able to provide minimal support
for limited error repair. As RTCP feedback is not guaranteed to
operate on sufficiently small timescales (in the order of RTT),
RTCP feedback is not suitable to support congestion control. This
memo addresses both unicast and multicast operation.
DCCP: DCCP [19] provides for congestion-controlled but unreliable
datagram flows for unicast communications. With TCP Friendly Rate
Control (TFRC)-based [20] congestion control (CCID 3), DCCP is
particularly suitable for audio and video communications. DCCP’s
acknowledgement messages may provide detailed feedback reporting
about received and missed datagrams (and thus about congestion).
When running RTP over DCCP, congestion control is performed at the
DCCP layer and no additional mechanisms are required at the RTP
layer. Furthermore, an RTCP-feedback-capable sender may leverage
the more frequent DCCP-based feedback and thus a receiver may
refrain from using (additional) Generic Feedback messages where
appropriate.
3. Rules for RTCP Feedback
3.1. Compound RTCP Feedback Packets
Two components constitute RTCP-based feedback as described in this
document:
o Status reports are contained in sender report (SR)/received report
(RR) packets and are transmitted at regular intervals as part of
compound RTCP packets (which also include source description
(SDES) and possibly other messages); these status reports provide
an overall indication for the recent reception quality of a media
stream.
o FB messages as defined in this document that indicate loss or
reception of particular pieces of a media stream (or provide some
other form of rather immediate feedback on the data received).
Rules for the transmission of FB messages are newly introduced in
this document.
RTCP FB messages are just another RTCP packet type (see Section 4).
Therefore, multiple FB messages MAY be combined in a single compound
RTCP packet and they MAY also be sent combined with other RTCP
packets.
Compound RTCP packets containing FB messages as defined in this
document MUST contain RTCP packets in the order defined in [1]:
o OPTIONAL encryption prefix that MUST be present if the RTCP
packet(s) is to be encrypted according to Section 9.1 of [1].
o MANDATORY SR or RR.
o MANDATORY SDES, which MUST contain the CNAME item; all other SDES
items are OPTIONAL.
o One or more FB messages.
The FB message(s) MUST be placed in the compound packet after RR and
SDES RTCP packets defined in [1]. The ordering with respect to other
RTCP extensions is not defined.
Two types of compound RTCP packets carrying feedback packets are used
in this document:
a) Minimal compound RTCP feedback packet
A minimal compound RTCP feedback packet MUST contain only the
mandatory information as listed above: encryption prefix if
necessary, exactly one RR or SR, exactly one SDES with only the
CNAME item present, and the FB message(s). This is to minimize
the size of the RTCP packet transmitted to convey feedback and
thus to maximize the frequency at which feedback can be provided
while still adhering to the RTCP bandwidth limitations.
This packet format SHOULD be used whenever an RTCP FB message is
sent as part of an Early RTCP packet. This packet type is
referred to as minimal compound RTCP packet in this document.
b) (Full) compound RTCP feedback packet
A (full) compound RTCP feedback packet MAY contain any additional
number of RTCP packets (additional RRs, further SDES items, etc.).
The above ordering rules MUST be adhered to.
This packet format MUST be used whenever an RTCP FB message is
sent as part of a Regular RTCP packet or in Regular RTCP mode. It
MAY also be used to send RTCP FB messages in Immediate Feedback or
Early RTCP mode. This packet type is referred to as full compound
RTCP packet in this document.
RTCP packets that do not contain FB messages are referred to as non-
FB RTCP packets. Such packets MUST follow the format rules in [1].
3.2. Algorithm Outline
FB messages are part of the RTCP control streams and thus subject to
the RTCP bandwidth constraints. This means, in particular, that it
may not be possible to report an event observed at a receiver
immediately back to the sender. However, the value of feedback
given to a sender typically decreases over time -- in terms of the
media quality as perceived by the user at the receiving end and/or
the cost required to achieve media stream repair.
RTP [1] and the commonly used RTP profile [2] specify rules when
compound RTCP packets should be sent. This document modifies those
rules in order to allow applications to timely report events (e.g.,
loss or reception of RTP packets) and to accommodate algorithms that
use FB messages.
The modified RTCP transmission algorithm can be outlined as follows:
As long as no FB messages have to be conveyed, compound RTCP packets
are sent following the rules of RTP [1] -- except that the five-
second minimum interval between RTCP reports is not enforced. Hence,
the interval between RTCP reports is only derived from the average
RTCP packet size and the RTCP bandwidth share available to the
RTP/RTCP entity. Optionally, a minimum interval between Regular RTCP
packets may be enforced.
If a receiver detects the need to send an FB message, it may do so
earlier than the next regular RTCP reporting interval (for which it
would be scheduled following the above regular RTCP algorithm).
Feedback suppression is used to avoid feedback implosion in
multiparty sessions: The receiver waits for a (short) random
dithering interval to check whether it sees a corresponding FB
message from any other receiver reporting the same event. Note that
for point-to-point sessions there is no such delay. If a
corresponding FB message from another member is received, this
receiver refrains from sending the FB message and continues to follow
the Regular RTCP transmission schedule. In case the receiver has not
yet seen a corresponding FB message from any other member, it checks
whether it is allowed to send Early feedback. If sending Early
feedback is permissible, the receiver sends the FB message as part of
a minimal compound RTCP packet. The permission to send Early
feedback depends on the type of the previous RTCP packet sent by this
receiver and the time the previous Early feedback message was sent.
FB messages may also be sent as part of full compound RTCP packets,
which are transmitted as per [1] (except for the five-second lower
bound) in regular intervals.
3.3. Modes of Operation
RTCP-based feedback may operate in one of three modes (Figure 1) as
described below. The mode of operation is just an indication of
whether or not the receiver will, on average, be able to report all
events to the sender in a timely fashion; the mode does not influence
the algorithm used for scheduling the transmission of FB messages.
And, depending on the reception quality and the locally monitored
state of the RTP session, individual receivers may not (and do not
have to) agree on a common perception on the current mode of
operation.
a) Immediate Feedback mode: In this mode, the group size is below the
FB threshold, which gives each receiving party sufficient
bandwidth to transmit the RTCP feedback packets for the intended
purpose. This means that, for each receiver, there is enough
bandwidth to report each event by means of a virtually "immediate"
RTCP feedback packet.
The group size threshold is a function of a number of parameters
including (but not necessarily limited to): the type of feedback
used (e.g., ACK vs. NACK), bandwidth, packet rate, packet loss
probability and distribution, media type, codec, and the (worst
case or observed) frequency of events to report (e.g., frame
received, packet lost).
As a rough estimate, let N be the average number of events to be
reported per interval T by a receiver, B the RTCP bandwidth
fraction for this particular receiver, and R the average RTCP
packet size, then the receiver operates in Immediate Feedback mode
as long as N<=B*T/R.
b) Early RTCP mode: In this mode, the group size and other parameters
no longer allow each receiver to react to each event that would be
worth reporting (or that needed reporting). But feedback can
still be given sufficiently often so that it allows the sender to
adapt the media stream transmission accordingly and thereby
increase the overall media playback quality.
Using the above notation, Early RTCP mode can be roughly