to an RTP packet using RFC 3640 conveying TTUs according to ISO/IEC
14496-17. In particular, MPEG-4 Part 17 specifies that when using
RFC 3640 for transporting timed text streams, the "streamType"
parameter value is set to 0x0D, and the value of the
"objectTypeIndication" in "config" takes the value 0x08.
+--------------------------------------+
Text samples | +--------------+ +--------------+ |
as per 3GPP | |Text Sample 1 | |Text Sample N | |
TS 26245 | +--------------+ +--------------+ |
+--------------------------------------+
\/
+-------------------------------------------------------------------+
| Partition Text Samples into units. TTU[i]= TYPE i units. |
| |
|[U R TYPE LEN][{TOTAL,THIS}SIDX{SDUR}{TLEN}{SLEN}][SampleContents] |
|{..} means present if applicable, [..] means always present |
+-------------------------------------------------------------------+
\/ \/
+-------------------------------------------------------------------+
| Aggregation (if possible) |
+-------------------------------------------------------------------+
\/ \/
+-------------------------------------------------------------------+
| RTP Entity adds and fills RTP header and Sends RTP packet, where |
| RTP packets according to this Payload Format = |
| RTP packets carrying MPEG-4 Timed Text ES over RFC 3640 |
+-------------------------------------------------------------------+
Figure 17. Relation to RFC 3640
Note: The use of RFC 3640 for transport of ISO/IEC 14496-17 data does
not require any new SDP parameters or any new mode definition.
4.9. Relation to RFC 2793
RFC 2793 [22] and its revision, RFC 4103 [23], specify a protocol for
enabling text conversation. Typical applications of this payload
format are text communication terminals and text conferencing tools.
Text session contents are specified in ITU-T Recommendation T.140
[24]. T.140 text is UTF-8 coded as specified in T.140 [24] with no
extra framing. The T140block contains one or more T.140 code
elements as specified in T.140. Code elements are control sequences
such as "New Line", "Interrupt", "String Terminator", or "Start of
String". Most T.140 code elements are single ISO 10646 [25]
characters, but some are multiple character sequences. Each
character is UTF-8 encoded [18] into one or more octets.
This payload format may also be used for conversational applications
(even for instant messaging). However, this is not its main target.
The differentiating feature of 3GPP Timed Text media format is that
it allows text decoration. This is especially useful in multimedia
presentations, karaoke, commercial banners, news tickers, clickable
text strings, and captions. T.140 text contents used in RFC 2793 do
not allow the use of text decoration.
Furthermore, the conversational text RTP payload format recommends a
method to include redundant text from already transmitted packets in
order to reduce the risk of text loss caused by packet loss. Thereby
payloads would include a redundant copy of the last payload sent.
This payload format does not describe such a method, but this is also
applicable here. As explained in Section 5, packet redundancy SHOULD
be used, whenever possible. The aggregation guidelines in Section
4.6 allow redundant payloads.
5. Resilient Transport
Apart from the basic fragmentation guidelines described in the
section above, the simplest option for packet-loss-resilient
transport is packet repetition. This mechanism may consist of a
strict window-based repetition mechanism or, simply, a repetition
mechanism in a wider sense, where new and old packets are mixed, for
example.
A server MAY decide to use repetition as a measure for packet loss
resilience. Thereby, a server MAY send the same RTP payloads or just
some of the units from the payloads.
As for the case of complete payloads, single repeated units MUST
exactly match the same units sent in the first transmission; i.e., if
fragmentation is needed, it SHALL be performed only once for each
text sample. Only then, a receiver can use the already received and
the repeated units to reconstruct the original text samples. Since
the RTP timestamp is used to group together the fragments of a
sample, care must taken to preserve the timing of units when
constructing new RTP packets.
For example, if a text sample was originally sent as a single
non-fragmented text sample (one TYPE 1 unit), a repetition of
that sample MUST be sent also as a single non-fragmented text
sample in one unit. Likewise, if the original text sample was
fragmented and spread over several RTP packets (say, a total of
3 units), then the repeated fragments SHALL also have the same
byte boundaries and use the same unit headers and bytes per
fragment.
With repetition, repeated units resolve to the same timestamp as
their originals. Where redundant units are available, only one of
them SHALL be used.
Regarding the RTP header fields:
o If the whole RTP payload is repeated, all payload-specific fields
in the RTP header (the M, TS and PT fields) MUST keep their
original values except the sequence number, which MUST be
incremented to comply with RTP (the fields TOTAL/THIS enable to
re-assemble fragments with different sequence numbers).
o In packets containing single repeated units, the general rules in
Section 3 for assigning values to the RTP header fields apply.
Keeping the value of the RTP timestamp to preserve the timing of
the units is particularly relevant here.
Apart from repetition, other mechanisms such as FEC [7],
retransmission [11], or similar techniques could be used to cope with
packet losses.
6. Congestion Control
Congestion control for RTP SHALL be implemented in accordance with
RTP [3] and the applicable RTP profile, e.g., RTP/AVP [17].
When using this payload format, mainly two factors may affect the
congestion control:
o The use of (unit) aggregation may make the payload format more
bandwidth efficient, by avoiding header overhead and thus reducing
the used bitrate.
o The use of resilient transport mechanisms: Although timed text
applications typically operate at low bitrates, the increase due to
resilient transport shall be considered for congestion control
mechanisms. This applies to all mechanisms but especially to less
efficient ones like repetition.
7. Scene Description
7.1. Text Rendering Position and Composition
In order to set up a timed text session, regardless of the stream
being stored in a 3GP file or streamed live, some initial layout
information is needed by the communicating peers.
+-------------------------------------------+
| <-> tx | +-------------+
| +-------------------------------+ |<---|Display Area |
| ^ | | | +-------------+
| : | | |
| :ty| | | +-------------+
| : | |<---------|Video track |
| : | | | +-------------+
| : | | |
| : | | |
| : | | |
| v | | |
| - | x-------------------------+ | | +-------------+
|h ^ | | |<-----------|Text Track |
|e : +---|-------------------------|-+ | +-------------+
|i : | +---------------------+ | |
|g : | | | | | +-------------+
|h : | | |<------------ |Text Box |
|t v | +---------------------+ | | +-------------+
| - +-------------------------+ |
+-------------------------------------------+
<........................>
w i d t h
Figure 18. Illustration of text rendering position and composition
The parameters used for negotiating the position and size of the text
track in the display area are shown in Figure 18. These are the
"width" and "height" of the text track, its translation values, "tx"
and "ty", and its "layer" or proximity to the user.
At the same time, the sender of the stream needs to know the
receiver’s capabilities. In this case, the maximum allowable values
for the text track height and width: "max-h" and "max-w", for the
stream the receiver shall display.
This layout information MUST be conveyed in a reliable form before
the start of the session, e.g., during session announcement or in an
Offer/Answer (O/A) exchange. An example of a reliable transport may
be the out-of-band channel used for SDP. Sections 8 and 9 provide
details on the mapping of these parameters to SDP descriptions and
their usage in O/A.
For stored content, the layout values expressing stream properties
MUST be obtained from the Track Header Box. See Section 7.3.
For live streaming, appropriate values as negotiated during session
setup shall be used.
7.2. SMIL Usage
The attributes contained in the Track Header Boxes of a 3GP file only
specify the spatial relationship of the tracks within the given 3GP
file.
If multiple 3GP files are sent, they require spatial synchronization.
For example, for a text and video stream, the positions of the text
and video tracks in Figure 18 shall be determined. For this purpose,
SMIL [9] MAY be used.
SMIL assigns regions in the display to each of those files and places
the tracks within those regions. Generally, in SMIL, the position of
one track (or stream) is expressed relative to another track. This
is different from the 3GP file, where the upper left corner is the
reference for all translation offsets. Hence, only if the position
in SMIL is relative to the video track origin, then this translation
offset has the same value as (tx, ty) in the 3GP file.
Note also that the original track header information is used for each
track only within its region, as assigned by SMIL. Therefore, even
if SMIL scene description is used, the track header information
pieces SHOULD be sent anyway, as they represent the intrinsic media
properties. See 3GPP SMIL Language Profile in [27] for details.
7.3. Finding Layout Values in a 3GP File
In a 3GP file, within the Track Header Box (tkhd):
o tx, ty: These values specify the translation offset of the
(text) track relative to the upper left corner of the video
track, if present. They are the second but last and third but
last values in the unity matrix; values are fixed-point 16.16
values, restricted to be (signed) integers (i.e., the lower 16
bits of each value shall be all zeros). Therefore, only the
first 16 bits are used for obtaining the value of the media
type parameters.
o width, height: They have the same name in the tkhd box. All
(unsigned) 32 bits are meaningful.
o layer: All (signed) 16 bits are used.
8. 3GPP Timed Text Media Type
The media subtype for the 3GPP Timed Text codec is allocated from the
standards tree. The top-level media type under which this payload
format is registered is ’video’. This registration is done using the
template defined in [29] and following RFC 3555 [28].
The receiver MUST ignore any unrecognized parameter.
Media type: video
Media subtype: 3gpp-tt
Required parameters
rate:
Refer to Section 3 in RFC 4396.
sver:
The parameter "sver" contains a list of supported
backwards-compatible versions of the timed text format
specification (3GPP TS 26.245) that the sender accepts
to receive (and that are the same that it would be
willing to send). The first value is the value
preferred to receive (or preferred to send). The first
value MAY be followed by a comma-separated list of
versions that SHOULD be used as alternatives. The order
is meaningful, being first the most preferred and last
the least preferred. Each entry has the format
Zi(xi*256+yi), where "Zi" is the number of the Release
and "xi" and "yi" are taken from the 3GPP specification
version (i.e., vZi.xi.yi). For example, for 3GPP TS
26.245 v6.0.0, Zi(xi*256+yi)=6(0), the version value is
"60". (Note that "60" is the concatenation of the
values Zi=6 and (xi*256+yi)=0 and not their product.)
If no "sver" value is available, for example, when
streaming out of a 3GP file, the default value "60",
corresponding to the 3GPP Release 6 version of 3GPP TS
26.245, SHALL be used.
Optional parameters:
tx:
This parameter indicates the horizontal translation
offset in pixels of the text track with respect to the
origin of the video track. This value is the decimal
representation of a 16-bit signed integer. Refer to TS
3GPP 26.245 for an illustration of this parameter.
ty:
This parameter indicates the vertical translation offset
in pixels of the text track with respect to the origin
of the video track. This value is the decimal
representation of a 16-bit signed integer. Refer to TS
3GPP 26.245 for an illustration of this parameter.
layer:
This parameter indicates the proximity of the text track
to the viewer. More negative values mean closer to the
viewer. This parameter has no units. This value is the
decimal representation of a 16-bit signed integer.
tx3g:
This parameter MUST be used for conveying sample
descriptions out-of-band. It contains a comma-separated
list of base64-encoded entries. The entries of this
list MAY follow any particular order and the list SHALL
NOT be empty. Each entry is the result of running
base64 encoding over the concatenation of the (static)
SIDX value as an 8-bit unsigned integer and the (static)
sample description for that SIDX, in that order. The
format of a sample description entry can be found in
3GPP TS 26.245 Release 6 and later releases. All
servers and clients MUST understand this parameter and
MUST be capable of using the sample description(s)
contained in it. Please refer to RFC 3548 [6] for
details on the base64 encoding.
width:
This parameter indicates the width in pixels of the text
track or area of the text being sent. This value is the
decimal representation of a 32-bit unsigned integer.
Refer to TS 3GPP 26.245 for an illustration of this
parameter.
height:
This parameter indicates the height in pixels of the
text track being sent. This value is the decimal
representation of a 32-bit unsigned integer. Refer to
TS 3GPP 26.245 for an illustration of this parameter.
max-w:
This parameter indicates display capabilities. This is
the maximum "width" value that the sender of this
parameter supports. This value is the decimal
representation of a 32-bit unsigned integer.
max-h:
This parameter indicates display capabilities. This is
the maximum "height" value that the sender of this
parameter supports. This value is the decimal
representation of a 32-bit unsigned integer.
Encoding considerations:
This media type is framed (see Section 4.8 in [29]) and
partially contains binary data.
Restrictions on usage:
This media type depends on RTP framing, and hence is only
defined for transfer via RTP [3]. Transport within other
framing protocols is not defined at this time.
Security considerations:
Please refer to Section 11 of RFC 4396.
Interoperability considerations:
The 3GPP Timed Text media format and its file storage is
specified in Release 6 of 3GPP TS 26.245, "Transparent end-to-
end packet switched streaming service (PSS); Timed Text Format
(Release 6)". Note also that 3GPP may in future releases
specify extensions or updates to the timed text media format in
a backwards-compatible way, e.g., new modifier boxes or
extensions to the sample descriptions. The payload format
defined in RFC 4396 allows for such extensions. For future 3GPP
Releases of the Timed Text Format, the parameter "sver" is used
to identify the exact specification used.
The defined storage format for 3GPP Timed Text format is the
3GPP File Format (3GP) [30]. 3GP files may be transferred using
the media type video/3gpp as registered by RFC 3839 [31]. The
3GPP File Format is a container file that may contain, e.g.,
audio and video that may be synchronized with the 3GPP Timed
Text.
Published specification: RFC 4396
Applications which use this media type:
Multimedia streaming applications.
Additional information:
The 3GPP Timed Text media format is specified in 3GPP TS 26.245,
"Transparent end-to-end packet switched streaming service (PSS);
Timed Text Format (Release 6)". This document and future
extensions to the 3GPP Timed Text format are publicly available
at http://www.3gpp.org.
Magic number(s): None.
File extension(s): None.
Macintosh File Type Code(s): None.
Person & email address to contact for further information:
Jose Rey, jose.rey@eu.panasonic.com
Yoshinori Matsui, matsui.yoshinori@jp.panasonic.com
Audio/Video Transport Working Group.
Intended usage: COMMON
Authors:
Jose Rey
Yoshinori Matsui
Change controller: IETF Audio/Video Transport Working Group delegated
from the IESG.
9. SDP Usage
9.1. Mapping to SDP
The information carried in the media type specification has a
specific mapping to fields in SDP [4]. If SDP is used to specify
sessions using this payload format, the mapping is done as follows:
o The media type ("video") goes in the SDP "m=" as the media name.
m=video <port number> RTP/<RTP profile> <dynamic payload type>
o The media subtype ("3gpp-tt") and the timestamp clockrate "rate"
(the RECOMMENDED 1000 Hz or other value) go in SDP "a=rtpmap" line
as the encoding name and rate, respectively:
a=rtpmap:<payload type> 3gpp-tt/1000
o The REQUIRED parameter "sver" goes in the SDP "a=fmtp" attribute by
copying it directly from the media type string as a semicolon-
separated parameter=value pair.
o The OPTIONAL parameters "tx", "ty", "layer", "tx3g", "width",
"height", "max-w" and "max-h" go in the SDP "a=fmtp" attribute by
copying them directly from the media type string as a semicolon
separated list of parameter=value(s) pairs:
a=fmtp:<dynamic payload type> <parameter
name>=<value>[,<value>][; <parameter name>=<value>]
o Any parameter unknown to the device that uses the SDP SHALL be
ignored. For example, parameters added to the media format in
later specifications MAY be copied into the SDP and SHALL be