timed text sample entry (type "tx3g").
The (unsigned) 32 bits of the "size" field in the stsd box
represent the length (in bytes) of the sample description, as
carried in TYPE 5 units. On the other hand, the LEN field of
TYPE 5 units is restricted to 16 bits. Therefore, if the
value of "size" is greater than (2^16-1-3)[bytes], then the
sample description SHALL NOT be streamed with this payload
format. There is no extension mechanism defined in this case,
since fragmentation of sample descriptions is not defined
(sample descriptions are typically up to some 200 bytes in
size). Note: The three (3) accounts for the TYPE 5 header
fields included in the LEN value.
o SDUR from the Decoding Time to Sample Box (stts). The
(unsigned) 32 bits of the "sample delta" field are used for
calculating SDUR. However, since the SDUR field is only 3
bytes long, text samples with duration values larger than
(2^24-1)/(timestamp clockrate)[seconds] cannot be streamed
directly. The solution is simple: Copies of the corresponding
text sample SHALL be sent. Thereby, the timestamp and
duration values SHALL be adjusted so that a continuous display
is guaranteed as if just one sample would have been sent.
That is, a sample with timestamp TS and duration SDUR can be
sent as two samples having timestamps TS1 and TS2 and
durations SDUR1 and SDUR2, such that TS1=TS, TS2=TS1+SDUR1,
and SDUR=SDUR1+SDUR2.
o Text sample length from the Sample Size Box (stsz). The
(unsigned) 32 bits of the "sample size" or "entry size" (one
of them, depending on whether the sample size is fixed or
variable) indicate the length (in bytes) of the 3GP text
sample. For obtaining the length of the (actual) streamed
text sample, the lengths of the text string byte count (2
bytes) and, in case of UTF-16 strings, the length the BOM
(also 2 bytes) SHALL be deducted. This is illustrated in
Figure 9.
Text Sample according to 3GPP TS 26.245
TEXT SAMPLE (length=stsz)
.--------------------------------------------------.
/ \
TEXT STRING (length=TBC)
.------------------------------------.
/ \
TBC BOM MODIFIERS
+---+---+----------------------------------+-----------+
||
|| TBC BOM -> TLEN field
|| +---+---+ U bit
||
\/
Text Sample according to this Payload Format
TEXT SAMPLE (length=SLEN w/o TBC,BOM)
.--------------------------------------------.
/ \
TEXT STRING (length=TLEN)
.--------------------------------.
/ \
TEXT STRING MODIFIERS
+----------------------------------+-----------+
KEY:
TBC = Text string Byte Count
BOM = Byte Order Mark
Figure 9. Text sample composition
Moreover, since the LEN field in TYPE 1 unit header is 16 bits
long, larger text sample sizes than (2^16-1-8) [bytes] SHALL
NOT be streamed. Also, in this case, no extension mechanism
is defined. This is because this maximum is considered enough
for the targeted streaming applications. (Note: The eight (8)
accounts for the TYPE 1 header fields included in the LEN
value).
o SIDX from the Sample to Chunk Box (stsc): The stsc Box is used
to find samples and their corresponding sample descriptions.
These are referenced by the "sample description index", a
32-bit (unsigned) integer. If possible, these indices may be
directly mapped to the SIDX field. However, there are several
cases where this may not be possible:
a) The total number of indices used is greater than
the number of indices available, i.e., if the static
sample descriptions are more than 127 or the dynamic ones
are more than 64.
b) The original SIDX value ranges do not fit in the
allowed ranges for static (129-254) or dynamic (0-127)
values.
Therefore, when assigning SIDX values to the sample
descriptions, the following guidelines are provided:
o Static sample descriptions can simply be assigned
consecutive values within the range 129-254 (closed
interval). This range should be well enough for static
sample descriptions.
o As for dynamic sample descriptions:
a) Streams that use less than 64 dynamic sample
descriptions SHOULD use consecutive values for SIDX
anywhere in the range 0-127 (closed interval).
b) For streams with more than 64 sample descriptions,
the SIDX values MUST be assigned in usage order, and if
any sample description shall be used after it has been
set inactive, it will need to be re-sent and assigned a
new SIDX value (according to the algorithm in Section
4.2.1).
Information obtained from the Media Data Box:
o Text strings, TLEN, U bit, and modifiers from the Media Data
Box (mdat). Text strings, 16-bit text string byte count, Byte
Order Mark (BOM, indicating UTF encoding), and modifier boxes
can be found here.
For TYPE 1 units, the value of TLEN is extracted from the text
string byte count that precedes the text string in the text
sample, as stored in the 3GP file. If UTF-16 encoding is
used, two (2) more bytes have to be deducted from this byte
count beforehand, in order to exclude the BOM. See Figure 9.
4.4. Fragmentation of Timed Text Samples
This section explains why text samples may have to be fragmented and
discusses some of the possible approaches to doing it. A solution is
proposed together with rules and recommendations for fragmenting and
transporting text samples.
3GPP Timed Text applications are expected to operate at low bitrates.
This fact, added to the small size of timed text samples (typically
one or two hundred bytes) makes fragmentation of text samples a rare
event. Samples should usually fit into the MTU size of the used
network path.
Nevertheless, some text strings (e.g., ending roll in a movie) and
some modifier boxes (i.e., for hyperlinks, for karaoke, or for
styles) may become large. This may also apply for future modifier
boxes. In such cases, the first option to consider is whether it is
possible to adjust the encoding (e.g., the size of sample) in such a
way that fragmentation is avoided. If it is, this is preferred to
fragmentation and SHOULD be done.
Otherwise, if this is not possible or other constraints prevent it,
fragmentation MAY be used, and the basic guidelines given in this
document MUST be followed:
o It is RECOMMENDED that text samples be fragmented as seldom as
possible, i.e., the least possible number of fragments is created
out of a text sample.
o If there is some bitrate and free space in the payload available,
sample descriptions (if at hand) SHOULD be aggregated.
o Text strings MUST split at character boundaries; see TYPE 2 header.
Otherwise, it is not possible to display the text contents of a
fragment if a previous fragment was lost. As a consequence, text
string fragmentation requires knowledge of the UTF-8/UTF-16
encoding formats to determine character boundaries.
o Unlike text strings, the modifier boxes are NOT REQUIRED to be
split at meaningful boundaries. However, it is RECOMMENDED that
this be done whenever possible. This decreases the effects of
packet loss. This payload format does not ensure that partially
received modifiers are applied to text strings. If only part of
the modifiers is received, it is an application issue how to deal
with these, i.e., whether or not to use them.
Informative note: Ensuring that partially received modifiers can
be applied to text strings in all cases (for all modifier types
and for all fragment loss constellations) would place additional
requirements on the payload format. In particular, this would
require that: a) senders understand the semantics of the
modifier boxes and b) specific fragment headers for each of the
modifier boxes are defined, in addition to the payload formats
defined below. Understanding the modifiers semantics means
knowing, e.g., where each modifier starts and ends, which text
fragments are affected, which modifiers may or may not be split,
or what the fields indicate. This is necessary to be able to
split the modifiers in such a way that each fragment can be
applied independently of previous packet losses. This would
require a more intelligent fragmentation entity and more complex
headers. Given the low probability of fragmentation and the
desire to keep the requirements low, it does not seem reasonable
to specify such modifier box specific headers.
o Modifier and text string fragments SHOULD be protected against
packet losses, i.e., using FEC [7], retransmission [11], repetition
(Section 5), or an equivalent technique. This minimizes the
effects of packet loss.
o An additional requirement when fragmenting text samples is that the
start of the modifiers MUST be indicated using the payload header
defined for that purpose, i.e., a TYPE 3 unit MUST be used (see
Section 4.1.4). This enables a receiver to detect the start of the
modifiers as long as there are not two or more consecutive packet
losses.
o Finally, sample descriptions SHALL NOT be fragmented because they
contain important information that may affect several text samples.
4.5. Reassembling Text Samples at the Receiver
The payload headers defined in this document allow reassembling
fragmented text samples. For this purpose, the standard RTP
timestamp, the duration field (SDUR), and the fields TOTAL/THIS in
the payload headers are used.
Units that belong to the same text sample MUST have the same
timestamp. TYPE 5 units do not comply with this rule since they are
not part of any particular text sample.
The process for collecting the different fragments (units) of a text
sample is as follows:
1. Search for units having the same timestamp value, i.e., units
that belong to the same text sample or sample descriptions that
shall become available at that time instant. If several units
of the same sample are repeated, only one of them SHALL be used.
Repeated units are those that have the same timestamp and the
same values for TOTAL/THIS.
Note that, as mentioned in Section 4.1.1, the receiver
SHALL ignore units with unrecognized TYPE value.
However, the RTP header fields and the rest of the units
(if any) in the payload are still useful.
2. Check within this set whether any of the units from the text
sample is missing. This is done using the TOTAL and THIS
fields; the TOTAL field indicates how many fragments were
created out of the text sample, and the THIS field indicates the
position of this fragment in the text sample. As result of this
operation, two outcomes are possible:
a. No fragment is missing. Then, the THIS field SHALL be used
to order the fragments and reassemble the text sample
before forwarding it to the decoding application. Special
care SHALL be taken when reassembling the text string as
indicated in bullet 4 below.
b. One or more fragments are missing: Check whether this
fragment belongs to the text string or to the modifiers.
TYPE 2 units identify text string fragments, and TYPE 3 and
4 identify modifier fragments:
i. If the fragment or fragments missing belong to the text
string and the modifiers were received complete, then
the received text characters may, at least, be
displayed as plain text. Some modifiers may only be
applied as long as it is possible to identify the
character numbers, e.g., if only the last text string
fragment is lost. This is the case for modifiers
defining specific font styles (’styl’), highlighted
characters (’hlit’), karaoke feature (’krok’), and
blinking characters (’blnk’). Other modifiers such as
’dlay’ or ’tbox’ can be applied without the knowledge
of the character number. It is an application issue to
decide whether or not to apply the modifiers.
ii. If the fragment missing belongs to the modifiers and
the text strings were received complete, then the
incomplete modifiers may be used. The text string
SHOULD at least be displayed as plain text. As
mentioned in Section 4.4, modifiers may split without
observing meaningful boundaries. Hence, it may not
always be possible to make use of partially received
modifiers. However, to avoid this, it is RECOMMENDED
that the modifiers do split at meaningful boundaries.
iii. A third possibility is that it is not possible to
discern whether modifiers or text strings were received
complete. For example, if the TYPE 3 unit of a sample
plus the following or preceding packet is lost, there
is no way for the RTP receiver to know if one or both
packets lost belong to the modifiers or if there are
also some missing text strings. Repetition, FEC,
retransmission, or other protection mechanisms as per
section 4.6 are RECOMMENDED to avoid this situation.
iv. Finally, if it is sure that neither text strings nor
modifiers were received complete, then the text strings
and the modifiers may be rendered partially or may be
discarded. This is an application choice.
3. Sample descriptions can be directly associated with the
reassembled text samples, via the sample description index
(SIDX).
4. Reassembling of text strings: Since the text strings transported
in RTP packets MUST NOT include any byte order mark (BOM), the
receiver MUST prepend it to the reassembled UTF-16 string before
handling it to the timed text decoder (see Figure 9). The value
of the BOM is 0xFEFF because only big endian serialization of
UTF-16 strings is supported by this payload format.
4.6. On Aggregate Payloads
Units SHOULD be aggregated to avoid overhead, whenever possible. The
aggregate payloads MUST comply with one of the following ordered
configurations:
1. Zero or more sample descriptions (TYPE 5) followed by zero or more
whole text samples (TYPE 1 units). At least one unit of either
type MUST be present.
2. Zero or more sample descriptions followed by zero or one modifier
fragment, either TYPE 3 or TYPE 4. At least one unit MUST be
present.
3. Zero or more sample descriptions, followed by zero or one text
string fragment (TYPE 2), followed by zero or one TYPE 3 unit. If
a TYPE 2 unit and a TYPE 3 unit are present, then they MUST belong
to the same text sample. At least one unit MUST be present.
Some observations:
o Different aggregates than the ones listed above SHALL NOT be used.
o Sample descriptions MUST be placed in the aggregate payload before
the occurrence of any non-TYPE 5 units.
o Correct reception of TYPE 5 units is important since their contents
may be referenced by several other units in the stream.
Receivers are unable to use text samples until their corresponding
sample descriptions are received. Accordingly, a sender SHOULD
send multiple copies of a sample description to ensure reliability
(see Section 5). Receivers MAY use payload-specific feedback
messages [21] to tell a sender that they have received a particular
sample description.
o Regarding timestamp calculation: In general, the rules for
calculating the timestamp of units in an aggregate payload depend
on the type of unit. Based on the possible constellations for
aggregate payloads, as above, we have:
o Sample descriptions MUST receive the RTP timestamp of the
packet in which they are included.
Note that for TYPE 5 units, the timestamp actually does not
represent the instant when they are played out, but instead
the instant at which they become available for use.
o For the first configuration: The first TYPE 1 unit receives
the RTP timestamp. The timestamp of any subsequent TYPE 1
unit MUST be obtained by adding sample duration and
timestamp, both of the preceding TYPE 1 unit.
o For the second and third configuration, all units, TYPE 2,
3, and 4, MUST receive the RTP timestamp.
Refer to detailed examples on the timestamp calculation
below.
o As per configuration 3 above, a payload MAY contain several
fragments of one (and only one) text sample. If it does, then
exactly one TYPE 2 unit followed by exactly one TYPE 3 unit is
allowed in the same payload. This is in line with RFC 3640 [12],
Section 2.4, which explicitly disallows combining fragments of
different samples in the same RTP payload. Note that, in this
special case, no timestamp calculation is needed. That is, the RTP
timestamp of both units is equal to the timestamp in the packet’s
RTP header.
o Finally, note that the use of empty text samples allows for
aggregating non-consecutive TYPE 1 units in the same payload. Two
text samples, with timestamps TS1 and TS3 and durations SDUR1 and
SDUR3, are not consecutive if it holds TS1+SDUR1 < TS3. A solution
for this is to include an empty TYPE 1 unit with duration SDUR2
between them, such that TS2+SDUR2 = TS1+SDUR1+SDUR2 = TS3.
Some examples of aggregate payloads are illustrated in Figure 10.
(Note: The figure is not scaled.)
N/A TS1 TS2 TS3
+------+-----+------+-----+
|TYPE5 |TYPE1|TYPE1 |TYPE1|
+------+-----+------+-----+
N/A sdur1 sdur2 sdur3
N/A TS4