Request for Comments: 4352 M. Westerlund
Category: Standards Track Ericsson
A. Lakaniemi
S. Wenger
Nokia
January 2006
RTP Payload Format for the
Extended Adaptive Multi-Rate Wideband (AMR-WB+) Audio Codec
Status of This Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
This document specifies a Real-time Transport Protocol (RTP) payload
format for Extended Adaptive Multi-Rate Wideband (AMR-WB+) encoded
audio signals. The AMR-WB+ codec is an audio extension of the AMR-WB
speech codec. It encompasses the AMR-WB frame types and a number of
new frame types designed to support high-quality music and speech. A
media type registration for AMR-WB+ is included in this
specification.
Table of Contents
1. Introduction ....................................................3
2. Definitions .....................................................4
2.1. Glossary ...................................................4
2.2. Terminology ................................................4
3. Background of AMR-WB+ and Design Principles .....................4
3.1. The AMR-WB+ Audio Codec ....................................4
3.2. Multi-rate Encoding and Rate Adaptation ....................8
3.3. Voice Activity Detection and Discontinuous Transmission ....8
3.4. Support for Multi-Channel Session ..........................8
3.5. Unequal Bit-Error Detection and Protection .................9
3.6. Robustness against Packet Loss .............................9
3.6.1. Use of Forward Error Correction (FEC) ...............9
3.6.2. Use of Frame Interleaving ..........................10
3.7. AMR-WB+ Audio over IP Scenarios ...........................11
3.8. Out-of-Band Signaling .....................................11
4. RTP Payload Format for AMR-WB+ .................................12
4.1. RTP Header Usage ..........................................13
4.2. Payload Structure .........................................14
4.3. Payload Definitions .......................................14
4.3.1. Payload Header .....................................14
4.3.2. The Payload Table of Contents ......................15
4.3.3. Audio Data .........................................20
4.3.4. Methods for Forming the Payload ....................21
4.3.5. Payload Examples ...................................21
4.4. Interleaving Considerations ...............................24
4.5. Implementation Considerations .............................25
4.5.1. ISF Recovery in Case of Packet Loss ................26
4.5.2. Decoding Validation ................................28
5. Congestion Control .............................................28
6. Security Considerations ........................................28
6.1. Confidentiality ...........................................29
6.2. Authentication and Integrity ..............................29
7. Payload Format Parameters ......................................29
7.1. Media Type Registration ...................................30
7.2. Mapping Media Type Parameters into SDP ....................32
7.2.1. Offer-Answer Model Considerations ..................32
7.2.2. Examples ...........................................34
8. IANA Considerations ............................................34
9. Contributors ...................................................34
10. Acknowledgements ..............................................34
11. References ....................................................35
11.1. Normative References .....................................35
11.2. Informative References ...................................35
1. Introduction
This document specifies the payload format for packetization of
Extended Adaptive Multi-Rate Wideband (AMR-WB+) [1] encoded audio
signals into the Real-time Transport Protocol (RTP) [3]. The payload
format supports the transmission of mono or stereo audio, aggregating
multiple frames per payload, and mechanisms enhancing the robustness
of the packet stream against packet loss.
The AMR-WB+ codec is an extension of the Adaptive Multi-Rate Wideband
(AMR-WB) speech codec. New features include extended audio bandwidth
to enable high quality for non-speech signals (e.g., music), native
support for stereophonic audio, and the option to operate on, and
switch between, several internal sampling frequencies (ISFs). The
primary usage scenario for AMR-WB+ is the transport over IP.
Therefore, interworking with other transport networks, as discussed
for AMR-WB in [7], is not a major concern and hence not addressed in
this memo.
The expected key application for AMR-WB+ is streaming. To make the
packetization process on a streaming server as efficient as possible,
an octet-aligned payload format is desirable. Therefore, a
bandwidth-efficient mode (as defined for AMR-WB in [7]) is not
specified herein; the bandwidth savings of the bandwidth-efficient
mode would be very small anyway, since all extension frame types are
octet aligned.
The stereo encoding capability of AMR-WB+ renders the support for
multi-channel transport at RTP payload format level, as specified for
AMR-WB [7], obsolete. Therefore, this feature is not included in
this memo.
This specification does not include a definition of a file format for
AMR-WB+. Instead, it refers to the ISO-based 3GP file format [14],
which supports AMR-WB+ and provides all functionality required. The
3GP format also supports storage of AMR, AMR-WB, and many other
multi-media formats, thereby allowing synchronized playback.
The rest of the document is organized as follows: Background
information on the AMR-WB+ codec, and design principles, can be found
in Section 3. The payload format itself is specified in Section 4.
Sections 5 and 6 discuss congestion control and security
considerations, respectively. In Section 7, a media type
registration is provided.
2. Definitions
2.1. Glossary
3GPP - Third Generation Partnership Project
AMR - Adaptive Multi-Rate (Codec)
AMR-WB - Adaptive Multi-Rate Wideband (Codec)
AMR-WB+ - Extended Adaptive Multi-Rate Wideband (Codec)
CN - Comfort Noise
DTX - Discontinuous Transmission
FEC - Forward Error Correction
FT - Frame Type
ISF - Internal Sampling Frequency
SCR - Source-Controlled Rate Operation
SID - Silence Indicator (the frames containing only CN
parameters)
TFI - Transport Frame Index
TS - Timestamp
VAD - Voice Activity Detection
UED - Unequal Error Detection
UEP - Unequal Error Protection
2.2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [2].
3. Background of AMR-WB+ and Design Principles
The Extended Adaptive Multi-Rate Wideband (AMR-WB+) [1] audio codec
is designed to compress speech and audio signals at low bit-rate and
good quality. The codec is specified by the Third Generation
Partnership Project (3GPP). The primary target applications are 1)
the packet-switched streaming service (PSS) [13], 2) multimedia
messaging service (MMS) [18], and 3) multimedia broadcast and
multicast service (MBMS) [19]. However, due to its flexibility and
robustness, AMR-WB+ is also well suited for streaming services in
other highly varying transport environments, for example, the
Internet.
3.1. The AMR-WB+ Audio Codec
3GPP originally developed the AMR-WB+ audio codec for streaming and
messaging services in Global System for Mobile communications (GSM)
and third generation (3G) cellular systems. The codec is designed as
an audio extension of the AMR-WB speech codec. The extension adds
new functionality to the codec in order to provide high audio quality
for a wide range of signals including music. Stereophonic operation
has also been added. A new, high-efficiency hybrid stereo coding
algorithm enables stereo operation at bit-rates as low as 6.2 kbit/s.
The AMR-WB+ codec includes the nine frame types specified for AMR-WB,
extended by new bit-rates ranging from 5.2 to 48 kbit/s. The AMR-WB
frame types can employ only a 16000 Hz sampling frequency and operate
only on monophonic signals. The newly introduced extension frame
types, however, can operate at a number of internal sampling
frequencies (ISFs), both in mono and stereo. Please see Table 24 in
[1] for details. The output sampling frequency of the decoder is
limited to 8, 16, 24, 32, or 48 kHz.
An overview of the AMR-WB+ encoding operations is provided as
follows. The encoder receives the audio sampled at, for example, 48
kHz. The encoding process starts with pre-processing and resampling
to the user-selected ISF. The encoding is performed on equally sized
super-frames. Each super-frame corresponds to 2048 samples per
channel, at the ISF. The codec carries out a number of encoding
decisions for each super-frame, thereby choosing between different
encoding algorithms and block lengths, so as to achieve a fidelity-
optimized encoding adapted to the signal characteristics of the
source. The stereo encoding (if used) executes separately from the
monophonic core encoding, thus enabling the selection of different
combinations of core and stereo encoding rates. The resulting
encoded audio is produced in four transport frames of equal length.
Each transport frame corresponds to 512 samples at the ISF and is
individually usable by the decoder, provided that its position in the
super-frame structure is known.
The codec supports 13 different ISFs, ranging from 12.8 to 38.4 kHz,
as described by Table 24 of [1]. The high number of ISFs allows a
trade-off between the audio bandwidth and the target bit-rate. As
encoding is performed on 2048 samples at the ISF, the duration of a
super-frame and the effective bit-rate of the frame type in use
varies.
The ISF of 25600 Hz has a super-frame duration of 80 ms. This is the
’nominal’ value used to describe the encoding bit-rates henceforth.
Assuming this normalization, the ISF selection results in bit-rate
variations from 1/2 up to 3/2 of the nominal bit-rate.
The encoding for the extension modes is performed as one monophonic
core encoding and one stereo encoding. The core encoding is executed
by splitting the monophonic signal into a lower and a higher
frequency band. The lower band is encoded employing either algebraic
code excited linear prediction (ACELP) or transform coded excitation
(TCX). This selection can be made once per transport frame, but must
obey certain limitations of legal combinations within the super-
frame. The higher band is encoded using a low-rate parametric
bandwidth extension approach.
The stereo signal is encoded employing a similar frequency band
decomposition; however, here the signal is divided into three bands
that are individually parameterized.
The total bit-rate produced by the extension is the result of the
combination of the encoder’s core rate, stereo rate, and ISF. The
extension supports 8 different core encoding rates, producing bit-
rates between 10.4 and 24.0 kbit/s; see Table 22 in [1]. There are
16 stereo encoding rates generating bit-rates between 2.0 and 8.0
kbit/s; see Table 23 in [1]. The frame type uniquely identifies the
AMR-WB modes, 4 fixed extension rates (see below), 24 combinations of
core and stereo rates for stereo signals, and the 8 core rates for
mono signals, as listed in Table 25 in [1]. This implies that the
AMR-WB+ supports encoding rates between 10.4 and 32 kbit/s, assuming
an ISF of 25600 Hz.
Different ISFs allow for additional freedom in the produced bit-rates
and audio quality. The selection of an ISF changes the available
audio bandwidth of the reconstructed signal, and also the total bit-
rate. The bit-rate for a given combination of frame type and ISF is
determined by multiplying the frame type’s bit-rate with the used
ISF’s bit-rate factor; see Table 24 in [1].
The extension also has four frame types which have fixed ISFs.
Please see frame types 10-13 in Table 21 in [1]. These four pre-
defined frame types have a fixed input sampling frequency at the
encoder, which can be set at either 16 or 24 kHz. Like the AMR-WB
frame types, transport frames encoded utilizing these frame types
represent exactly 20 ms of the audio signal. However, they are also
part of 80 ms super-frames. Frame types 0-13 (AMR-WB and fixed
extension rates), as listed in Table 21 in [1], do not require an
explicit ISF indication. The other frame types, 14-47, require the
ISF employed to be indicated.
The 32 different frame types of the extension, in combination with 13
ISFs, allows for a great flexibility in bit-rate and selection of
desired audio quality. A number of combinations exist that produce
the same codec bit-rate. For example, a 32 kbit/s audio stream can
be produced by utilizing frame type 41 (i.e., 25.6 kbit/s) and the
ISF of 32kHz (5/4 * (19.2+6.4) = 32 kbit/s), or frame type 47 and the
ISF of 25.6 kHz (1 * (24 + 8) = 32 kbit/s). Which combination is
more beneficial for the perceived audio quality depends on the
content. In the above example, the first case provides a higher
audio bandwidth, while the second one spends the same number of bits
on somewhat narrower audio bandwidth but provides higher fidelity.
Encoders are free to select the combination they deem most
beneficial.
Since a transport frame always corresponds to 512 samples at the used
ISF, its duration is limited to the range 13.33 to 40 ms; see Table
1. An RTP Timestamp clock rate of 72000 Hz, as mandated by this
specification, results in AMR-WB+ transport frame lengths of 960 to
2880 timestamp ticks, depending solely on the selected ISF.
Index ISF Duration(ms) Duration(TS Ticks @ 72 kHz)
------------------------------------------------------
0 N/A 20 1440
1 12800 40 2880
2 14400 35.55 2560
3 16000 32 2304
4 17067 30 2160
5 19200 26.67 1920
6 21333 24 1728
7 24000 21.33 1536
8 25600 20 1440
9 28800 17.78 1280
10 32000 16 1152
11 34133 15 1080
12 36000 14.22 1024
13 38400 13.33 960
Table 1: Normative number of RTP Timestamp Ticks for each
Transport Frame depending on ISF (ISF and Duration in
ms are rounded)
The encoder is free to change both the ISF and the encoding frame
type (both mono and stereo) during a session. For the extension
frame types with index 10-13 and 16-47, the ISF and frame type
changes are constrained to occur at super-frame boundaries. This
implies that, for the frame types mentioned, the ISF is constant
throughout a super-frame. This limitation does not apply for frame
types with index 0-9, 14, and 15; i.e., the original AMR-WB frame
types.
A number of features of the AMR-WB+ codec require special
consideration from a transport point of view, and solutions that
could perhaps be viewed as unorthodox. First, there are constraints
on the RTP timestamping, due to the relationship of the frame
duration and the ISFs. Second, each frame of encoded audio must
maintain information about its frame type, ISF, and position in the
super-frame.
3.2. Multi-rate Encoding and Rate Adaptation
The multi-rate encoding capability of AMR-WB+ is designed to preserve
high audio quality under a wide range of bandwidth requirements and
transmission conditions.
AMR-WB+ enables seamless switching between frame types that use the
same number of audio channels and the same ISF. Every AMR-WB+ codec
implementation is required to support all frame types defined by the
codec and must be able to handle switching between any two frame
types. Switching between frame types employing a different number of
audio channels or a different ISF must also be supported, but it may
not be completely seamless. Therefore, it is recommended to perform
such switching infrequently and, if possible, during periods of
silence.
3.3. Voice Activity Detection and Discontinuous Transmission
AMR-WB+ supports the same algorithms as AMR-WB for voice activity
detection (VAD) and generation of comfort noise (CN) parameters
during silence periods. However, these functionalities can only be
used in conjunction with the AMR-WB frame types (FT=0-8). This
option allows reducing the number of transmitted bits and packets
during silence periods to a minimum. The operation of sending CN
parameters at regular intervals during silence periods is usually
called discontinuous transmission (DTX) or source controlled rate
(SCR) operation. The AMR-WB+ frames containing CN parameters are
called Silence Indicator (SID) frames. More details about the VAD
and DTX functionality are provided in [4] and [5].
3.4. Support for Multi-Channel Session
Some of the AMR-WB+ frame types support the encoding of stereophonic
audio. Because of this native support for a two-channel stereophonic
signal, it does not seem necessary to support multi-channel transport
with separate codec instances, as specified in the AMR-WB RTP payload
[7]. The codec has the capability of stereo to mono downmixing as
part of the decoding process. Thus, a receiver that is only capable
of playout of monophonic audio must still be able to decode and play
signals originally encoded and transmitted as stereo. However, to
avoid spending bits on a stereo encoding that is not going to be
utilized, a mechanism is defined in this specification to signal
mono-only audio.
3.5. Unequal Bit-Error Detection and Protection
The audio bits encoded in each AMR-WB frame are sorted according to
their different perceptual sensitivity to bit errors. In cellular
systems, for example, this property can be exploited to achieve
better voice quality, by using unequal error protection and detection
(UEP and UED) mechanisms. However, the bits of the extension frame
types of the AMR-WB+ codec do not have a consistent perceptual
significance property and are not sorted in this order. Thus, UEP or
UED is meaningless with the extension frame types. If there is a
need to use UEP or UED for AMR-WB frame types, it is recommended that
RFC 3267 [7] be used.
3.6. Robustness against Packet Loss
The payload format supports two mechanisms to improve robustness
against packet loss: simple forward error correction (FEC) and frame
interleaving.
3.6.1. Use of Forward Error Correction (FEC)
Generic forward error correction within RTP is defined, for example,
in RFC 2733 [11]. Audio redundancy coding is defined in RFC 2198
[12]. Either scheme can be used to add redundant information to the
RTP packet stream and make it more resilient to packet losses, at the
expense of a higher bit rate. Please see either RFC for a discussion
of the implications of the higher bit rate to network congestion.
In addition to these media-unaware mechanisms, this memo specifies an
AMR-WB+ specific form of audio redundancy coding, which may be
beneficial in terms of packetization overhead.
Conceptually, previously transmitted transport frames are aggregated
together with new ones. A sliding window is used to group the frames
to be sent in each payload. Figure 1 below shows an example.
--+--------+--------+--------+--------+--------+--------+--------+--
| f(n-2) | f(n-1) | f(n) | f(n+1) | f(n+2) | f(n+3) | f(n+4) |
--+--------+--------+--------+--------+--------+--------+--------+--
<---- p(n-1) ---->
<----- p(n) ----->
<---- p(n+1) ---->
<---- p(n+2) ---->
<---- p(n+3) ---->
<---- p(n+4) ---->
Figure 1: An example of redundant transmission
Here, each frame is retransmitted once in the following RTP payload
packet. F(n-2)...f(n+4) denote a sequence of audio frames, and
p(n-1)...p(n+4) a sequence of payload packets.
The mechanism described does not require signaling at the session
setup. In other words, the audio sender can choose to use this
scheme without consulting the receiver. For a certain timestamp, the
receiver may receive multiple copies of a frame containing encoded
audio data or frames indicated as NO_DATA. The cost of this scheme
is bandwidth and the receiver delay necessary to allow the redundant
copy to arrive.
This redundancy scheme provides a functionality similar to the one
described in RFC 2198, but it works only if both original frames and
redundant representations are AMR-WB+ frames. When the use of other
media coding schemes is desirable, one has to resort to RFC 2198.
The sender is responsible for selecting an appropriate amount of
redundancy based on feedback about the channel conditions, e.g., in
the RTP Control Protocol (RTCP) [3] receiver reports. The sender is
also responsible for avoiding congestion, which may be exacerbated by
redundancy (see Section 5 for more details).
3.6.2. Use of Frame Interleaving
To decrease protocol overhead, the payload design allows several
audio transport frames to be encapsulated into a single RTP packet.
One of the drawbacks of such an approach is that in case of packet
loss several consecutive frames are lost. Consecutive frame loss
normally renders error concealment less efficient and usually causes
clearly audible and annoying distortions in the reconstructed audio.
Interleaving of transport frames can improve the audio quality in
such cases by distributing the consecutive losses into a number of
isolated frame losses, which are easier to conceal. However,
interleaving and bundling several frames per payload also increases