6.2.1 Start codes of ISO/IEC 14496-2).
For BIFS streams, this is the BIFSConfig() information as defined
in ISO/IEC 14496-1. Version 1 of BIFSConfig is defined in
section 9.3.5.2, and version 2 is defined in section 9.3.5.3.
The MIME format parameter objectType signals the version of
BIFSConfig.
For IPMP streams, this is either a quoted empty hexadecimal octet
string, indicating the absence of any decoder configuration
information (config=""), or the IPMPConfiguration() as will be
defined in a future MPEG-4 IPMP specification.
For Object Content Info (OCI) streams, this is the
OCIDecoderConfiguration() information of the OCI stream, as
defined in section 8.4.2.4 in ISO/IEC 14496-1.
For OD streams, Clock Reference streams and MPEG-J streams, this
is a quoted empty hexadecimal octet string (config=""), as no
information on the decoder configuration is required.
mode:
The mode in which this specification is used. The following modes
can be signaled:
mode=generic,
mode=CELP-cbr,
mode=CELP-vbr,
mode=AAC-lbr and
mode=AAC-hbr.
Other modes are expected to be defined in future RFCs. See also
section 3.3.7 and 4.2 of RFC 3640.
Optional general parameters:
objectType:
The decimal value from Table 8 in ISO/IEC 14496-1, indicating the
value of the objectTypeIndication of the transported stream. For
BIFS streams, this parameter MUST be present to signal the version
of BIFSConfiguration(). Note that objectTypeIndication may signal
a non-MPEG-4 stream and that the RTP payload format defined in
this document may not be suitable for carrying a stream that is
not defined by MPEG-4. The objectType parameter SHOULD NOT be set
to a value that signals a stream that cannot be carried by this
payload format.
constantSize:
The constant size in octets of each Access Unit for this stream.
The constantSize and the sizeLength parameters MUST NOT be
simultaneously present.
constantDuration:
The constant duration of each Access Unit for this stream,
measured with the same units as the RTP time stamp.
maxDisplacement:
The decimal representation of the maximum displacement in time of
an interleaved AU, as defined in section 3.2.3.3, expressed in
units of the RTP time stamp clock.
This parameter MUST be present when interleaving is applied.
de-interleaveBufferSize:
The decimal representation in number of octets of the size of the
de-interleave buffer, described in section 3.2.3.3. When
interleaving, this parameter MUST be present if the calculation of
the de-interleave buffer size given in 3.2.3.3 and based on
maxDisplacement and rate(max) under-estimates the size of the
de-interleave buffer. If this calculation does not under-estimate
the size of the de-interleave buffer, then the
de-interleaveBufferSize parameter SHOULD NOT be present.
Optional configuration parameters:
sizeLength:
The number of bits on which the AU-size field is encoded in the
AU-header. The sizeLength and the constantSize parameters MUST
NOT be simultaneously present.
indexLength:
The number of bits on which the AU-Index is encoded in the first
AU-header. The default value of zero indicates the absence of the
AU-Index field in each first AU-header.
indexDeltaLength:
The number of bits on which the AU-Index-delta field is encoded in
any non-first AU-header. The default value of zero indicates the
absence of the AU-Index-delta field in each non-first AU-header.
CTSDeltaLength:
The number of bits on which the CTS-delta field is encoded in the
AU-header.
DTSDeltaLength:
The number of bits on which the DTS-delta field is encoded in the
AU-header.
randomAccessIndication:
A decimal value of zero or one, indicating whether the RAP-flag is
present in the AU-header. The decimal value of one indicates
presence of the RAP-flag, the default value zero indicates its
absence.
streamStateIndication:
The number of bits on which the Stream-state field is encoded in
the AU-header. This parameter MAY be present when transporting
MPEG-4 system streams, and SHALL NOT be present for MPEG-4 audio
and MPEG-4 video streams.
auxiliaryDataSizeLength:
The number of bits that is used to encode the auxiliary-data-size
field.
Applications MAY use more parameters, in addition to those defined
above. Each additional parameter MUST be registered with IANA to
ensure that there is not a clash of names. Each additional parameter
MUST be accompanied by a specification in the form of an RFC, MPEG
standard, or other permanent and readily available reference (the
"Specification Required" policy defined in RFC 2434 [6]). Receivers
MUST tolerate the presence of such additional parameters, but these
parameters SHALL NOT impact the decoding of receivers that comply
with this specification.
Encoding considerations:
This MIME subtype is defined for RTP transport only. System
bitstreams MUST be generated according to MPEG-4 Systems
specifications (ISO/IEC 14496-1). Video bitstreams MUST be generated
according to MPEG-4 Visual specifications (ISO/IEC 14496-2). Audio
bitstreams MUST be generated according to MPEG-4 Audio specifications
(ISO/IEC 14496-3). The RTP packets MUST be packetized according to
the RTP payload format defined in RFC 3640.
Security considerations:
As defined in section 5 of RFC 3640.
Interoperability considerations:
MPEG-4 provides a large and rich set of tools for the coding of
visual objects. For effective implementation of the standard,
subsets of the MPEG-4 tool sets have been provided for use in
specific applications. These subsets, called ’Profiles’, limit the
size of the tool set a decoder is required to implement. In order to
restrict computational complexity, one or more ’Levels’ are set for
each Profile. A Profile@Level combination allows:
. a codec builder to implement only the subset of the standard
he needs, while maintaining interworking with other MPEG-4
devices that implement the same combination, and
. checking whether MPEG-4 devices comply with the standard
(’conformance testing’).
A stream SHALL be compliant with the MPEG-4 Profile@Level specified
by the parameter "profile-level-id". Interoperability between a
sender and a receiver is achieved by specifying the parameter
"profile-level-id" in MIME content. In the capability
exchange/announcement procedure, this parameter may mutually be set
to the same value.
Published specification:
The specifications for MPEG-4 streams are presented in ISO/IEC
14496-1, 14496-2, and 14496-3. The RTP payload format is described
in RFC 3640.
Applications which use this media type:
Multimedia streaming and conferencing tools.
Additional information: none
Magic number(s): none
File extension(s):
None. A file format with the extension .mp4 has been defined for
MPEG-4 content but is not directly correlated with this MIME type for
which the sole purpose is RTP transport.
Macintosh File Type Code(s): none
Person & email address to contact for further information:
Authors of RFC 3640, IETF Audio/Video Transport working group.
Intended usage: COMMON
Author/Change controller:
Authors of RFC 3640, IETF Audio/Video Transport working group.
4.2. Registration of Mode Definitions with IANA
This specification can be used in a number of modes. The mode of
operation is signaled using the "mode" MIME parameter, with the
initial set of values specified in section 4.1. New modes may be
defined at any time, as described in section 3.3.7. These modes MUST
be registered with IANA, to ensure that there is not a clash of
names.
A new mode registration MUST be accompanied by a specification in the
form of an RFC, MPEG standard, or other permanent and readily
available reference (the "Specification Required" policy defined in
RFC 2434 [6]).
4.3. Concatenation of Parameters
Multiple parameters SHOULD be expressed as a MIME media type string,
in the form of a semicolon-separated list of parameter=value pairs
(for parameter usage examples see sections 3.3.2 up to 3.3.6).
4.4. Usage of SDP
4.4.1. The a=fmtp Keyword
It is assumed that one typical way to transport the above-described
parameters associated with this payload format is via an SDP message
[5] for example transported to the client in reply to an RTSP
DESCRIBE [8] or via SAP [11]. In that case, the (a=fmtp) keyword
MUST be used as described in RFC 2327 [5], section 6, the syntax then
being:
a=fmtp:<format> <parameter name>=<value>[; <parameter name>=<value>]
5. Security Considerations
RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP
specification [2]. This implies that confidentiality of the media
streams is achieved by encryption. Because the data compression used
with this payload format is applied end-to-end, encryption may be
performed on the compressed data so there is no conflict between the
two operations. The packet processing complexity of this payload
type (i.e., excluding media data processing) does not exhibit any
significant non-uniformity in the receiver side to cause a denial-
of-service threat.
However, it is possible to inject non-compliant MPEG streams (Audio,
Video, and Systems) so that the receiver/decoder’s buffers are
overloaded, which might compromise the functionality of the receiver
or even crash it. This is especially true for end-to-end systems
like MPEG, where the buffer models are precisely defined.
MPEG-4 Systems support stream types including commands that are
executed on the terminal, like OD commands, BIFS commands, etc. and
programmatic content like MPEG-J (Java(TM) Byte Code) and MPEG-4
scripts. It is possible to use one or more of the above in a manner
non-compliant to MPEG to crash the receiver or make it temporarily
unavailable. Senders that transport MPEG-4 content SHOULD ensure
that such content is MPEG compliant, as defined in the compliance
part of IEC/ISO 14496 [1]. Receivers that support MPEG-4 content
should prevent malfunctioning of the receiver in case of non MPEG
compliant content.
Authentication mechanisms can be used to validate the sender and the
data to prevent security problems due to non-compliant malignant
MPEG-4 streams.
In ISO/IEC 14496-1, a security model is defined for MPEG-4 Systems
streams carrying MPEG-J access units that comprise Java(TM) classes
and objects. MPEG-J defines a set of Java APIs and a secure
execution model. MPEG-J content can call this set of APIs and
Java(TM) methods from a set of Java packages supported in the
receiver within the defined security model. According to this
security model, downloaded byte code is forbidden to load libraries,
define native methods, start programs, read or write files, or read
system properties. Receivers can implement intelligent filters to
validate the buffer requirements or parametric (OD, BIFS, etc.) or
programmatic (MPEG-J, MPEG-4 scripts) commands in the streams.
However, this can increase the complexity significantly.
Implementors of MPEG-4 streaming over RTP who also implement MPEG-4
scripts (subset of ECMAScript) MUST ensure that the action of such
scripts is limited solely to the domain of the single presentation in
which they reside (thus disallowing session to session communication,
access to local resources and storage, etc). Though loading static
network-located resources (such as media) into the presentation
should be permitted, network access by scripts MUST be restricted to
such a (media) download.
6. Acknowledgements
This document evolved into RFC 3640 after several revisions. Thanks
to contributions from people in the ISMA forum, the IETF AVT Working
Group and the 4-on-IP ad-hoc group within MPEG. The authors wish to
thank all people involved, particularly Andrea Basso, Stephen Casner,
M. Reha Civanlar, Carsten Herpel, John Lazaro, Zvi Lifshitz, Young-
kwon Lim, Alex MacAulay, Bill May, Colin Perkins, Dorairaj V and
Stephan Wenger for their valuable comments and support.
APPENDIX: Usage of this Payload Format
Appendix A. Interleave Analysis
A. Examples of Delay Analysis with Interleave
A.1. Introduction
Interleaving issues are discussed in this appendix. Some general
notes are provided on de-interleaving and error concealment, while a
number of interleaving patterns are examined, in particular for
determining the size of the de-interleave buffer and the maximum
displacement of access units in time. In these examples, the maximum
displacement is cited in terms of an access unit count, for ease of
reading. In actual streams, it is signaled in units of the RTP time
stamp clock.
A.2. De-interleaving and Error Concealment
This appendix does not describe any details on de-interleaving and
error concealment, as the control of the AU decoding and error
concealment process has little to do with interleaving. If the next
AU to be decoded is present and there is sufficient storage available
for the decoded AU, then decode it immediately. If not, wait. When
the decoding deadline is reached (i.e., the time when decoding must
begin in order to be completed by the time the AU is to be
presented), or if the decoder is some hardware that presents a
constant delay between initiation of decoding of an AU and
presentation of that AU, then decoding must begin at that deadline
time.
If the next AU to be decoded is not present when the decoding
deadline is reached, then that AU is lost so the receiver must take
whatever error concealment measures are deemed appropriate. The
play-out delay may need to be adjusted at that point (especially if
other AUs have also missed their deadline recently). Or, if it was a
momentary delay, and maintaining the latency is important, then the
receiver should minimize the glitch and continue processing with the
next AU.
A.3. Simple Group Interleave
A.3.1. Introduction
An example of regular interleave is when packets are formed into
groups. If the ’stride’ of the interleave (the distance between
interleaved AUs) is N, packet 0 could contain AU(0), AU(N), AU(2N),
and so on; packet 1 could contain AU(1), AU(1+N), AU(1+2N), and so
on. If there are M access units in a packet, then there are M*N
access units in the group.
An example with N=M=3 follows; note that this is the same example as
given in section 2.5 and that a fixed time duration per Access Unit
is assumed:
Packet Time stamp Carried AUs AU-Index, AU-Index-delta
P(0) T[0] 0, 3, 6 0, 2, 2
P(1) T[1] 1, 4, 7 0, 2, 2
P(2) T[2] 2, 5, 8 0, 2, 2
P(3) T[9] 9,12,15 0, 2, 2
In this example, the AU-Index is present in the first AU-header and
coded with the value 0, as required for fixed duration AUs. The
position of the first AU of each packet within the group is defined
by the RTP time stamp, while the AU-Index-delta field indicates the
position of subsequent AUs relative to the first AU in the packet.
All AU-Index-delta fields are coded with the value N-1, equal to 2 in
this example. Hence the RTP time stamp and the AU-Index-delta are
used to reconstruct the original order. See also section 3.2.3.2.
A.3.2. Determining the De-interleave Buffer Size
For the regular pattern as in this example, Figure 6 in section
3.2.3.3 shows that the de-interleave buffer stores at most 4 AUs. A
de-interleaveBufferSize value that is at least equal to the total
number of octets of any 4 "early" AUs that are stored at the same
time may be signaled.
A.3.3. Determining the Maximum Displacement
For the regular pattern as in this example, Figure 7 in section 3.3
shows that the maximum displacement in time equals 5 AU periods.
Hence, the minimum maxDisplacement value that must be signaled is 5
AU periods. In case each AU has the same size, this maxDisplacement
value over-estimates the de-interleave buffer size with one AU.
However, note that in case of variable AU sizes, the total size of
any 4 "early" AUs that must be stored at the same time may exceed
maxDisplacement times the maximum bitrate, in which case the de-
interleaveBufferSize must be signaled.
A.4. More Subtle Group Interleave
A.4.1. Introduction
Another example of forming packets with group interleave is given
below. In this example, the packets are formed such that the loss of
two subsequent RTP packets does not cause the loss of two subsequent
AUs. Note that in this example, the RTP time stamps of packet 3 and
packet 4 are earlier than the RTP time stamps of packets 1 and 2,
respectively; a fixed time duration per Access Unit is assumed.
Packet Time stamp Carried AUs AU-Index, AU-Index-delta
0 T[0] 0, 5 0, 4
1 T[2] 2, 7 0, 4
2 T[4] 4, 9 0, 4
3 T[1] 1, 6 0, 4
4 T[3] 3, 8 0, 4
5 T[10] 10, 15 0, 4
and so on ..
In this example, the AU-Index is present in the first AU-header and
coded with the value 0, as required for AUs with a fixed duration.
To reconstruct the original order, the RTP time stamp and the AU-
Index-delta (coded with the value 4) are used. See also section
3.2.3.2.
A.4.2. Determining the De-interleave Buffer Size
From Figure 8, it can be to determined that at most 5 "early" AUs are
to be stored. If the AUs are of constant size, then this value
equals 5 times the AU size. The minimum size of the de-interleave
buffer equals the maximum total number of octets of the "early" AUs
that are to be stored at the same time. This gives the minimum value
of the de-interleaveBufferSize that may be signaled.
+--+--+--+--+--+--+--+--+--+--+
Interleaved AUs | 0| 5| 2| 7| 4| 9| 1| 6| 3| 8|
+--+--+--+--+--+--+--+--+--+--+
- - 5 - 5 - 2 7 4 9
7 4 9 5
"Early" AUs 5 6
7 7
9 9
Figure 8: Storage of "early" AUs in the de-interleave buffer per
interleaved AU.
A.4.3. Determining the Maximum Displacement
From Figure 9, it can be seen that the maximum displacement in time
equals 8 AU periods. Hence the minimum maxDisplacement value to be
signaled is 8 AU periods.
+--+--+--+--+--+--+--+--+--+--+
Interleaved AUs | 0| 5| 2| 7| 4| 9| 1| 6| 3| 8|
+--+--+--+--+--+--+--+--+--+--+
Earliest not yet present AU - 1 1 1 1 1 - 3 - -
Figure 9: For each AU in the interleaving pattern, the earliest of
any earlier AUs not yet present
In case each AU has the same size, the found maxDisplacement value
over-estimates the de-interleave buffer size with three AUs.
However, in case of variable AU sizes, the total size of any 5
"early" AUs stored at the same time may exceed maxDisplacement times
the maximum bitrate, in which case de-interleaveBufferSize must be
signaled.
A.5. Continuous Interleave
A.5.1. Introduction
In continuous interleave, once the scheme is ’primed’, the number of
AUs in a packet exceeds the ’stride’ (the distance between them).
This shortens the buffering needed, smoothes the data-flow, and gives
slightly larger packets -- and thus lower overhead -- for the same
interleave. For example, here is a continuous interleave also over a
stride of 3 AUs, but with 4 AUs per packet, for a run of 20 AUs.
This shows both how the scheme ’starts up’ and how it finishes. Once
again, the example assumes fixed time duration per Access Unit.
Packet Time-stamp Carried AUs AU-Index, AU-Index-delta
0 T[0] 0 0
1 T[1] 1 4 0 2
2 T[2] 2 5 8 0 2 2
3 T[3] 3 6 9 12 0 2 2 2
4 T[7] 7 10 13 16 0 2 2 2
5 T[11] 11 14 17 20 0 2 2 2
6 T[15] 15 18 0 2
7 T[19] 19 0
In this example, the AU-Index is present in the first AU-header and
coded with the value 0, as required for AUs with a fixed duration.
To reconstruct the original order, the RTP time stamp and the
AU-Index-delta (coded with the value 2) are used. See also 3.2.3.2.
Note that this example has RTP time-stamps in increasing order.
A.5.2. Determining the De-interleave Buffer Size
For this example the de-interleave buffer size can be derived from
Figure 10. The maximum number of "early" AUs is 3. If the AUs are
of constant size, then the de-interleave buffer size equals 3 times
the AU size. Compared to the example in A.2, for constant size AUs
the de-interleave buffer size is reduced from 4 to 3 times the AU
size, while maintaining the same ’stride’.
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+-
Interleaved AUs | 0| 1| 4| 2| 5| 8| 3| 6| 9|12| 7|10|13|16|
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+-
- - - 4 - - 4 8 - - 8 12 - -