frame, and the other one consisting of two frames.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Frame Type1 | #frames = 1 | DIS1 | padd |0| Frame Type2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| #frames = 2 | DIS1 | DIS2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
4.3.3. Audio Data
Audio data of a payload consists of zero or more audio frames, as
described in the ToC of the payload.
ToC entries with FT=14 or 15 represent frame types with a length of
0. Hence, no data SHALL be placed in the audio data section to
represent frames of this type.
As already discussed, each audio frame of an extension frame type
represents an AMR-WB+ transport frame corresponding to the encoding
of 512 samples of audio, sampled with the internal sampling frequency
specified by the ISF indicator. As an exception, frame types with
index 10-13 are only capable of using a single internal sampling
frequency (25600 Hz). The encoding rates (combination of core bit-
rate and stereo bit-rate) are indicated in the frame type field of
the corresponding ToC entry. The octet length of the audio frame is
implicitly defined by the frame type field and is given in Tables 21
and 25 of [1]. The order and numbering notation of the bits are as
specified in [1]. For the AMR-WB+ extension frame types and comfort
noise frames, the bits are in the order produced by the encoder. The
last octet of each audio frame MUST be padded with zeroes at the end
if not all bits in the octet are used. In other words, each audio
frame MUST be octet-aligned.
4.3.4. Methods for Forming the Payload
The payload begins with the payload header, followed by the table of
contents, which consists of a list of ToC entries.
The audio data follows the table of contents. All the octets
comprising an audio frame SHALL be appended to the payload as a unit.
The audio frames are packetized in timestamp order within each group
of frames (per ToC entry). The groups of frames are packetized in
the same order as their corresponding ToC entries. Note that there
are no data octets in a group having a ToC entry with FT=14 or FT=15.
4.3.5. Payload Examples
4.3.5.1. Example 1: Basic Mode Payload Carrying Multiple Frames Encoded
Using the Same Frame Type
Figure 4 depicts a payload that carries three AMR-WB+ frames encoded
using 14 kbit/s frame type (FT=26) with a frame length of 280 bits
(35 bytes). The internal sampling frequency in this example is 25.6
kHz (ISF = 8). The TFI for the first frame is 2, indicating that the
first transport frame in this payload is the third in a super-frame.
Since this payload is in the basic mode, the subsequent frames of the
payload are consecutive frames in decoding order, i.e., the fourth
transport frame of the current super-frame and the first transport
frame of the next super-frame. Note that because the frames are all
encoded using the same frame type, only one ToC entry is required.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ISF = 8 | 2 |0|0| FT = 26 | #frames = 3 | f1(0...7) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f1(272...279) | f2(0...7) | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| f2(272...279) | f3(0...7) | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f3(272...279) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: An example of a basic mode payload carrying three frames
of the same frame type
4.3.5.2. Example 2: Basic Mode Payload Carrying Multiple Frames Encoded
Using Different Frame Types
Figure 5 depicts a payload that carries three AMR-WB+ frames; the
first frame is encoded using 18.4 kbit/s frame type (FT=33) with a
frame length of 368 bits (46 bytes), and the two subsequent frames
are encoded using 20 kbit/s frame type (FT=35) having frame length of
400 bits (50 bytes). The internal sampling frequency in this example
is 32 kHz (ISF = 10), implying the overall bit-rates of 23 kbit/s for
the first frame of the payload, and 25 kbit/s for the subsequent
frames. The TFI for the first frame is 3, indicating that the first
transport frame in this payload is the fourth in a super-frame.
Since this is a payload in the basic mode, the subsequent frames of
the payload are consecutive frames in decoding order, i.e., the first
and second transport frames of the current super-frame. Note that
since the payload carries two different frame types, there are two
ToC entries.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ISF=10 | 3 |0|1| FT = 33 | #frames = 1 |0| FT = 35 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| #frames = 2 | f1(0...7) | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f1(360...367) | f2(0...7) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| f2(392...399) | f3(0...7) | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f3(392...399) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: An example of a basic mode payload carrying three frames
employing two different frame types
4.3.5.3. Example 3: Payload in Interleaved Mode
The example in Figure 6 depicts a payload in interleaved mode,
carrying four frames encoded using 32 kbit/s frame type (FT=47) with
frame length of 640 bits (80 bytes). The internal sampling frequency
is 38.4 kHz (ISF = 13), implying a bit-rate of 48 kbit/s for all
frames in the payload. The TFI for the first frame is 0; hence, it
is the first transport frame of a super-frame. The displacement
fields for the subsequent frames are DIS2=18, DIS3=15, and DIS4=10,
which indicates that the subsequent frames have the TFIs of 3, 3, and
2, respectively. The long displacement field flag L in the payload
header is set to 1, which results in the use of eight bits for the
displacement fields in the ToC entry. Note that since all frames of
this payload are encoded using the same frame type, there is need
only for a single ToC entry. Furthermore, the displacement field for
the first frame (corresponding to the first ToC entry with DIS1=0)
must be ignored, since its timestamp and TFI are defined by the RTP
timestamp and the TFI found in the payload header.
The RTP timestamp values of the frames in this example are:
Frame1: TS1 = RTP Timestamp
Frame2: TS2 = TS1 + 19 * 960
Frame3: TS3 = TS2 + 16 * 960
Frame4: TS4 = TS3 + 11 * 960
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ISF=13 | 0 |1|0| FT = 47 | #frames = 4 | DIS1 = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DIS2 = 18 | DIS3 = 15 | DIS4 = 10 | f1(0...7) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f1(632...639) | f2(0...7) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f2(632...639) | f3(0...7) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f3(632...639) | f4(0...7) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | f4(632...639) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: An example of an interleaved mode payload carrying four
frames at the same frame type
4.4. Interleaving Considerations
The use of interleaving requires further considerations. As
presented in the example in Section 3.6.2, a given interleaving
pattern requires a certain amount of the deinterleaving buffer. This
buffer space, expressed in a number of transport frame slots, is
indicated by the "interleaving" media type parameter. The number of
frame slots needed can be converted into actual memory requirements
by considering the 80 bytes per frame used by the largest combination
of AMR-WB+’s core and stereo rates.
The information about the frame buffer size is not always sufficient
to determine when it is appropriate to start consuming frames from
the interleaving buffer. There are two cases in which additional
information is needed: first, when switching of the ISF occurs, and
second, when the interleaving pattern changes. The "int-delay" media
type parameter is defined to convey this information. It allows a
sender to indicate the minimal media time that needs to be present in
the buffer before the decoder can start consuming frames from the
buffer. Because the sender has full control over ISF changes and the
interleaving pattern, it can calculate this value.
In certain cases (for example, if joining a multicast session with
interleaving mid-session), a receiver may initially receive only part
of the packets in the interleaving pattern. This initial partial
reception (in frame sequence order) of frames can yield too few
frames for acceptable quality from the audio decoding. This problem
also arises when using encryption for access control, and the
receiver does not have the previous key.
Although the AMR-WB+ is robust and thus tolerant to a high random
frame erasure rate, it would have difficulties handling consecutive
frame losses at startup. Thus, some special implementation
considerations are described. In order to handle this type of
startup efficiently, it must be noted that decoding is only possible
to start at the beginning of a super-frame, and that holds true even
if the first transport frame is indicated as lost. Secondly,
decoding is only RECOMMENDED to start if at least 2 transport frames
are available out of the 4 belonging to that super-frame.
After receiving a number of packets, in the worst case as many
packets as the interleaving pattern covers, the previously described
effects disappear and normal decoding is resumed.
Similar issues arise when a receiver leaves a session or has lost
access to the stream. If the receiver leaves the session, this would
be a minor issue since playout is normally stopped. It is also a
minor issue for the case of lost access, since the AMR-WB+ error
concealment will fade out the audio if massive consecutive losses are
encountered.
The sender can avoid this type of problem in many sessions by
starting and ending interleaving patterns correctly when risks of
losses occur. One such example is a key-change done for access
control to encrypted streams. If only some keys are provided to
clients and there is a risk of their receiving content for which they
do not have the key, it is recommended that interleaving patterns not
overlap key changes.
4.5. Implementation Considerations
An application implementing this payload format MUST understand all
the payload parameters. Any mapping of the parameters to a signaling
protocol MUST support all parameters. So an implementation of this
payload format in an application using SDP is required to understand
all the payload parameters in their SDP-mapped form. This
requirement ensures that an implementation always can decide whether
it is capable of communicating.
Both basic and interleaved mode SHALL be implemented. The
implementation burden of both is rather small, and requiring both
ensures interoperability. As the AMR-WB+ codec contains the full
functionality of the AMR-WB codec, it is RECOMMENDED to also
implement the payload format in RFC 3267 [7] for the AMR-WB frame
types when implementing this specification. Doing so makes
interoperability with devices that only support AMR-WB more likely.
The switching of ISF, when combined with packet loss, could result in
concealment using the wrong audio frame length. This can occur if
packet losses result in lost frames directly after the point of ISF
change. The packet loss would prevent the receiver from noticing the
changed ISF and thereby conceal the lost transport frame with the
previous ISF, instead of the new one. Although always later
detectable, such an error results in frame boundary misalignment,
which can cause audio distortions and problems with synchronization,
as too many or too few audio samples were created. This problem can
be mitigated in most cases by performing ISF recovery prior to
concealment as outlined in Section 4.5.1.
4.5.1. ISF Recovery in Case of Packet Loss
In case of packet loss, it is important that the AMR-WB+ decoder
initiates a proper error concealment to replace the frames carried in
the lost packet. A loss concealment algorithm requires a codec
framing that matches the timestamps of the correctly received frames.
Hence, it is necessary to recover the timestamps of the lost frames.
Doing so is non-trivial because the codec frame length that is
associated with the ISF may have changed during the frame loss.
In the following, the recovery of the timestamp information of lost
frames is illustrated by the means of an example. Two frames with
timestamps t0 and t1 have been received properly, the first one being
the last packet before the loss, and the latter one being the first
packet after the loss period. The ISF values for these packets are
isf0 and isf1, respectively. The TFIs of these frames are tfi0 and
tfi1, respectively. The associated frame lengths (in timestamp
ticks) are given as L0 and L1, respectively. In this example three
frames with timestamps x1 - x3 have been lost. The example further
assumes that ISF changes once from isf0 to isf1 during the frame loss
period, as shown in the figure below.
Since not all information required for the full recovery of the
timestamps is generally known in the receiver, an algorithm is needed
to estimate the ISF associated with the lost frames. Also, the
number of lost frames needs to be recovered.
|<---L0--->|<---L0--->|<-L1->|<-L1->|<-L1->|
| Rxd | lost | lost | lost | Rxd |
--+----------+----------+------+------+------+--
t0 x1 x2 x3 t1
Example Algorithm:
Start: # check for frame loss
If (t0 + L0) == t1 Then goto End # no frame loss
Step 1: # check case with no ISF change
If (isf0 != isf1) Then goto Step 2 # At least one ISF change
If (isFractional(t1 - t0)/L0) Then goto Step 3
# More than 1 ISF change
Return recovered timestamps as
x(n) = t0 + n*L1 and associated ISF equal to isf0,
for 0 < n < (t1 - t0)/L0
goto End
Step 2:
Loop initialization: n := 4 - tfi0 mod 4
While n <= (t1-t0)/L0
Evaluate m := (t1 - t0 - n*L0)/L1
If (isInteger(m) AND ((tfi0+n+m) mod 4 == tfi1)) Then goto found;
n := n+4
endloop
goto step 3 # More than 1 ISF change
found:
Return recovered timestamps and ISFs as
x(i) = t0 + i*L0 and associated ISF equal to isf0, for 0 < i <= n
x(i) = t0 + n*L0 + (i-n)*L1 and associated ISF equal to isf1,
for n < i <= n+m
goto End
Step 3:
More than 1 ISF change has occurred. Since ISF changes can be
assumed to be infrequent, such a situation occurs only if long
sequences of frames are lost. In that case it is probably not useful
to try to recover the timestamps of the lost frames. Rather, the
AMR-WB+ decoder should be reset, and decoding should be resumed
starting with the frame with timestamp t1.
End:
The above algorithm still does not solve the issue when the receiver
buffer depth is shallower than the loss burst. In this kind of case,
where the concealment must be done without any knowledge about future
frames, the concealment may result in loss of frame boundary
alignment. If that occurs, it may be necessary to reset and restart
the codec to perform resynchronization.
4.5.2. Decoding Validation
If the receiver finds a mismatch between the size of a received
payload and the size indicated by the ToC of the payload, the
receiver SHOULD discard the packet. This is recommended because
decoding a frame parsed from a payload based on erroneous ToC data
could severely degrade the audio quality.
5. Congestion Control
The general congestion control considerations for transporting RTP
data apply; see RTP [3] and any applicable RTP profile like AVP [9].
However, the multi-rate capability of AMR-WB+ audio coding provides a
mechanism that may help to control congestion, since the bandwidth
demand can be adjusted (within the limits of the codec) by selecting
a different coding frame type or lower internal sampling rate.
The number of frames encapsulated in each RTP payload highly
influences the overall bandwidth of the RTP stream due to header
overhead constraints. Packetizing more frames in each RTP payload
can reduce the number of packets sent and hence the header overhead,
at the expense of increased delay and reduced error robustness.
If forward error correction (FEC) is used, the amount of FEC-induced
redundancy needs to be regulated such that the use of FEC itself does
not cause a congestion problem.
6. Security Considerations
RTP packets using the payload format defined in this specification
are subject to the general security considerations discussed in RTP
[3] and any applicable profile such as AVP [9] or SAVP [10]. As this
format transports encoded audio, the main security issues include
confidentiality, integrity protection, and data origin authentication
of the audio itself. The payload format itself does not have any
built-in security mechanisms. Any suitable external mechanisms, such
as SRTP [10], MAY be used.
This payload format and the AMR-WB+ decoder do not exhibit any
significant non-uniformity in the receiver-side computational
complexity for packet processing, and thus are unlikely to pose a
denial-of-service threat due to the receipt of pathological data.
6.1. Confidentiality
In order to ensure confidentiality of the encoded audio, all audio
data bits MUST be encrypted. There is less need to encrypt the
payload header or the table of contents since they only carry
information about the frame type. This information could also be
useful to a third party, for example, for quality monitoring.
The use of interleaving in conjunction with encryption can have a
negative impact on confidentiality, for a short period of time.
Consider the following packets (in brackets) containing frame numbers
as indicated: {10, 14, 18}, {13, 17, 21}, {16, 20, 24} (a popular
continuous diagonal interleaving pattern). The originator wishes to
deny some participants the ability to hear material starting at time
16. Simply changing the key on the packet with the timestamp at or
after 16, and denying that new key to those participants, does not
achieve this; frames 17, 18, and 21 have been supplied in prior
packets under the prior key, and error concealment may make the audio
intelligible at least as far as frame 18 or 19, and possibly further.
6.2. Authentication and Integrity
To authenticate the sender of the speech, an external mechanism MUST
be used. It is RECOMMENDED that such a mechanism protects both the
complete RTP header and the payload (speech and data bits).
Data tampering by a man-in-the-middle attacker could replace audio
content and also result in erroneous depacketization/decoding that
could lower the audio quality.
7. Payload Format Parameters
This section defines the parameters that may be used to select
features of the AMR-WB+ payload format. The parameters are defined
as part of the media type registration for the AMR-WB+ audio codec.
A mapping of the parameters into the Session Description Protocol