Request for Comments: 4313 Cisco Systems, Inc.
Category: Informational December 2005
Requirements for Distributed Control of
Automatic Speech Recognition (ASR),
Speaker Identification/Speaker Verification (SI/SV), and
Text-to-Speech (TTS) Resources
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2005).
Abstract
This document outlines the needs and requirements for a protocol to
control distributed speech processing of audio streams. By speech
processing, this document specifically means automatic speech
recognition (ASR), speaker recognition -- which includes both speaker
identification (SI) and speaker verification (SV) -- and
text-to-speech (TTS). Other IETF protocols, such as SIP and Real
Time Streaming Protocol (RTSP), address rendezvous and control for
generalized media streams. However, speech processing presents
additional requirements that none of the extant IETF protocols
address.
Table of Contents
1. Introduction ....................................................3
1.1. Document Conventions .......................................3
2. SPEECHSC Framework ..............................................4
2.1. TTS Example ................................................5
2.2. Automatic Speech Recognition Example .......................6
2.3. Speaker Identification example .............................6
3. General Requirements ............................................7
3.1. Reuse Existing Protocols ...................................7
3.2. Maintain Existing Protocol Integrity .......................7
3.3. Avoid Duplicating Existing Protocols .......................7
3.4. Efficiency .................................................8
3.5. Invocation of Services .....................................8
3.6. Location and Load Balancing ................................8
3.7. Multiple Services ..........................................8
3.8. Multiple Media Sessions ....................................8
3.9. Users with Disabilities ....................................9
3.10. Identification of Process That Produced Media or
Control Output ............................................9
4. TTS Requirements ................................................9
4.1. Requesting Text Playback ...................................9
4.2. Text Formats ...............................................9
4.2.1. Plain Text ..........................................9
4.2.2. SSML ................................................9
4.2.3. Text in Control Channel ............................10
4.2.4. Document Type Indication ...........................10
4.3. Control Channel ...........................................10
4.4. Media Origination/Termination by Control Elements .........10
4.5. Playback Controls .........................................10
4.6. Session Parameters ........................................11
4.7. Speech Markers ............................................11
5. ASR Requirements ...............................................11
5.1. Requesting Automatic Speech Recognition ...................11
5.2. XML .......................................................11
5.3. Grammar Requirements ......................................12
5.3.1. Grammar Specification ..............................12
5.3.2. Explicit Indication of Grammar Format ..............12
5.3.3. Grammar Sharing ....................................12
5.4. Session Parameters ........................................12
5.5. Input Capture .............................................12
6. Speaker Identification and Verification Requirements ...........13
6.1. Requesting SI/SV ..........................................13
6.2. Identifiers for SI/SV .....................................13
6.3. State for Multiple Utterances .............................13
6.4. Input Capture .............................................13
6.5. SI/SV Functional Extensibility ............................13
7. Duplexing and Parallel Operation Requirements ..................13
7.1. Full Duplex Operation .....................................14
7.2. Multiple Services in Parallel .............................14
7.3. Combination of Services ...................................14
8. Additional Considerations (Non-Normative) ......................14
9. Security Considerations ........................................15
9.1. SPEECHSC Protocol Security ................................15
9.2. Client and Server Implementation and Deployment ...........16
9.3. Use of SPEECHSC for Security Functions ....................16
10. Acknowledgements ..............................................17
11. References ....................................................18
11.1. Normative References .....................................18
11.2. Informative References ...................................18
1. Introduction
There are multiple IETF protocols for establishment and termination
of media sessions (SIP [6]), low-level media control (Media Gateway
Control Protocol (MGCP) [7] and Media Gateway Controller (MEGACO)
[8]), and media record and playback (RTSP [9]). This document
focuses on requirements for one or more protocols to support the
control of network elements that perform Automated Speech Recognition
(ASR), speaker identification or verification (SI/SV), and rendering
text into audio, also known as Text-to-Speech (TTS). Many multimedia
applications can benefit from having automatic speech recognition
(ASR) and text-to-speech (TTS) processing available as a distributed,
network resource. This requirements document limits its focus to the
distributed control of ASR, SI/SV, and TTS servers.
There is a broad range of systems that can benefit from a unified
approach to control of TTS, ASR, and SI/SV. These include
environments such as Voice over IP (VoIP) gateways to the Public
Switched Telephone Network (PSTN), IP telephones, media servers, and
wireless mobile devices that obtain speech services via servers on
the network.
To date, there are a number of proprietary ASR and TTS APIs, as well
as two IETF documents that address this problem [13], [14]. However,
there are serious deficiencies to the existing documents. In
particular, they mix the semantics of existing protocols yet are
close enough to other protocols as to be confusing to the
implementer.
This document sets forth requirements for protocols to support
distributed speech processing of audio streams. For simplicity, and
to remove confusion with existing protocol proposals, this document
presents the requirements as being for a "framework" that addresses
the distributed control of speech resources. It refers to such a
framework as "SPEECHSC", for Speech Services Control.
1.1. Document Conventions
In this document, the key words "MUST", "MUST NOT", "REQUIRED",
"SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY",
and "OPTIONAL" are to be interpreted as described in RFC 2119 [3].
2. SPEECHSC Framework
Figure 1 below shows the SPEECHSC framework for speech processing.
+-------------+
| Application |
| Server |\
+-------------+ \ SPEECHSC
SIP, VoiceXML, / \
etc. / \
+------------+ / \ +-------------+
| Media |/ SPEECHSC \---| ASR, SI/SV, |
| Processing |-------------------------| and/or TTS |
RTP | Entity | RTP | Server |
=====| |=========================| |
+------------+ +-------------+
Figure 1: SPEECHSC Framework
The "Media Processing Entity" is a network element that processes
media. It may be a pure media handler, or it may also have an
associated SIP user agent, VoiceXML browser, or other control entity.
The "ASR, SI/SV, and/or TTS Server" is a network element that
performs the back-end speech processing. It may generate an RTP
stream as output based on text input (TTS) or return recognition
results in response to an RTP stream as input (ASR, SI/SV). The
"Application Server" is a network element that instructs the Media
Processing Entity on what transformations to make to the media
stream. Those instructions may be established via a session protocol
such as SIP, or provided via a client/server exchange such as
VoiceXML. The framework allows either the Media Processing Entity or
the Application Server to control the ASR or TTS Server using
SPEECHSC as a control protocol, which accounts for the SPEECHSC
protocol appearing twice in the diagram.
Physical embodiments of the entities can reside in one physical
instance per entity, or some combination of entities. For example, a
VoiceXML [11] gateway may combine the ASR and TTS functions on the
same platform as the Media Processing Entity. Note that VoiceXML
gateways themselves are outside the scope of this protocol.
Likewise, one can combine the Application Server and Media Processing
Entity, as would be the case in an interactive voice response (IVR)
platform.
One can also decompose the Media Processing Entity into an entity
that controls media endpoints and entities that process media
directly. Such would be the case with a decomposed gateway using
MGCP or MEGACO. However, this decomposition is again orthogonal to
the scope of SPEECHSC. The following subsections provide a number of
example use cases of the SPEECHSC, one each for TTS, ASR, and SI/SV.
They are intended to be illustrative only, and not to imply any
restriction on the scope of the framework or to limit the
decomposition or configuration to that shown in the example.
2.1. TTS Example
This example illustrates a simple usage of SPEECHSC to provide a
Text-to-Speech service for playing announcements to a user on a phone
with no display for textual error messages. The example scenario is
shown below in Figure 2. In the figure, the VoIP gateway acts as
both the Media Processing Entity and the Application Server of the
SPEECHSC framework in Figure 1.
+---------+
_| SIP |
_/ | Server |
+-----------+ SIP/ +---------+
| | _/
+-------+ | VoIP |_/
| POTS |___| Gateway | RTP +---------+
| Phone | | (SIP UA) |=========| |
+-------+ | |\_ | SPEECHSC|
+-----------+ \ | TTS |
\__ | Server |
SPEECHSC | |
\_| |
+---------+
Figure 2: Text-to-Speech Example of SPEECHSC
The Plain Old Telephone Service (POTS) phone on the left attempts to
make a phone call. The VoIP gateway, acting as a SIP UA, tries to
establish a SIP session to complete the call, but gets an error, such
as a SIP "486 Busy Here" response. Without SPEECHSC, the gateway
would most likely just output a busy signal to the POTS phone.
However, with SPEECHSC access to a TTS server, it can provide a
spoken error message. The VoIP gateway therefore constructs a text
error string using information from the SIP messages, such as "Your
call to 978-555-1212 did not go through because the called party was
busy". It then can use SPEECHSC to establish an association with a
SPEECHSC server, open an RTP stream between itself and the server,
and issue a TTS request for the error message, which will be played
to the user on the POTS phone.
2.2. Automatic Speech Recognition Example
This example illustrates a VXML-enabled media processing entity and
associated application server using the SPEECHSC framework to supply
an ASR-based user interface through an Interactive Voice Response
(IVR) system. The example scenario is shown below in Figure 3. The
VXML-client corresponds to the "media processing entity", while the
IVR application server corresponds to the "application server" of the
SPEECHSC framework of Figure 1.
+------------+
| IVR |
_|Application |
VXML_/ +------------+
+-----------+ __/
| |_/ +------------+
PSTN Trunk | VoIP | SPEECHSC| |
=============| Gateway |---------| SPEECHSC |
|(VXML voice| | ASR |
| browser) |=========| Server |
+-----------+ RTP +------------+
Figure 3: Automatic Speech Recognition Example
In this example, users call into the service in order to obtain stock
quotes. The VoIP gateway answers their PSTN call. An IVR
application feeds VXML scripts to the gateway to drive the user
interaction. The VXML interpreter on the gateway directs the user’s
media stream to the SPEECHSC ASR server and uses SPEECHSC to control
the ASR server.
When, for example, the user speaks the name of a stock in response to
an IVR prompt, the SPEECHSC ASR server attempts recognition of the
name, and returns the results to the VXML gateway. The VXML gateway,
following standard VXML mechanisms, informs the IVR Application of
the recognized result. The IVR Application can then do the
appropriate information lookup. The answer, of course, can be sent
back to the user using text-to-speech. This example does not show
this scenario, but it would work analogously to the scenario shown in
section Section 2.1.
2.3. Speaker Identification example
This example illustrates using speaker identification to allow
voice-actuated login to an IP phone. The example scenario is shown
below in Figure 4. In the figure, the IP Phone acts as both the
"Media Processing Entity" and the "Application Server" of the
SPEECHSC framework in Figure 1.
+-----------+ +---------+
| | RTP | |
| IP |=========| SPEECHSC|
| Phone | | TTS |
| |_________| Server |
| | SPEECHSC| |
+-----------+ +---------+
Figure 4: Speaker Identification Example
In this example, a user speaks into a SIP phone in order to get
"logged in" to that phone to make and receive phone calls using his
identity and preferences. The IP phone uses the SPEECHSC framework
to set up an RTP stream between the phone and the SPEECHSC SI/SV
server and to request verification. The SV server verifies the
user’s identity and returns the result, including the necessary login
credentials, to the phone via SPEECHSC. The IP Phone may use the
identity directly to identify the user in outgoing calls, to fetch
the user’s preferences from a configuration server, or to request
authorization from an Authentication, Authorization, and Accounting
(AAA) server, in any combination. Since this example uses SPEECHSC
to perform a security-related function, be sure to note the
associated material in Section 9.
3. General Requirements
3.1. Reuse Existing Protocols
To the extent feasible, the SPEECHSC framework SHOULD use existing
protocols.
3.2. Maintain Existing Protocol Integrity
In meeting the requirement of Section 3.1, the SPEECHSC framework
MUST NOT redefine the semantics of an existing protocol. Said
differently, we will not break existing protocols or cause
backward-compatibility problems.
3.3. Avoid Duplicating Existing Protocols
To the extent feasible, SPEECHSC SHOULD NOT duplicate the
functionality of existing protocols. For example, network
announcements using SIP [12] and RTSP [9] already define how to
request playback of audio. The focus of SPEECHSC is new
functionality not addressed by existing protocols or extending
existing protocols within the strictures of the requirement in
Section 3.2. Where an existing protocol can be gracefully extended
to support SPEECHSC requirements, such extensions are acceptable
alternatives for meeting the requirements.
As a corollary to this, the SPEECHSC should not require a separate
protocol to perform functions that could be easily added into the
SPEECHSC protocol (like redirecting media streams, or discovering
capabilities), unless it is similarly easy to embed that protocol
directly into the SPEECHSC framework.
3.4. Efficiency
The SPEECHSC framework SHOULD employ protocol elements known to
result in efficient operation. Techniques to be considered include:
o Re-use of transport connections across sessions
o Piggybacking of responses on requests in the reverse direction
o Caching of state across requests
3.5. Invocation of Services
The SPEECHSC framework MUST be compliant with the IAB Open Pluggable
Edge Services (OPES) [4] framework. The applicability of the
SPEECHSC protocol will therefore be specified as occurring between
clients and servers at least one of which is operating directly on
behalf of the user requesting the service.
3.6. Location and Load Balancing
To the extent feasible, the SPEECHSC framework SHOULD exploit
existing schemes for supporting service location and load balancing,
such as the Service Location Protocol [13] or DNS SRV records [14].
Where such facilities are not deemed adequate, the SPEECHSC framework
MAY define additional load balancing techniques.
3.7. Multiple Services
The SPEECHSC framework MUST permit multiple services to operate on a
single media stream so that either the same or different servers may
be performing speech recognition, speaker identification or
verification, etc., in parallel.
3.8. Multiple Media Sessions
The SPEECHSC framework MUST allow a 1:N mapping between session and
RTP channels. For example, a single session may include an outbound
RTP channel for TTS, an inbound for ASR, and a different inbound for
SI/SV (e.g., if processed by different elements on the Media Resource
Element). Note: All of these can be described via SDP, so if SDP is
utilized for media channel description, this requirement is met "for
free".
3.9. Users with Disabilities
The SPEECHSC framework must have sufficient capabilities to address
the critical needs of people with disabilities. In particular, the
set of requirements set forth in RFC 3351 [5] MUST be taken into
account by the framework. It is also important that implementers of
SPEECHSC clients and servers be cognizant that some interaction
modalities of SPEECHSC may be inconvenient or simply inappropriate
for disabled users. Hearing-impaired individuals may find TTS of
limited utility. Speech-impaired users may be unable to make use of
ASR or SI/SV capabilities. Therefore, systems employing SPEECHSC
MUST provide alternative interaction modes or avoid the use of speech
processing entirely.
3.10. Identification of Process That Produced Media or Control Output
The client of a SPEECHSC operation SHOULD be able to ascertain via
the SPEECHSC framework what speech process produced the output. For
example, an RTP stream containing the spoken output of TTS should be
identifiable as TTS output, and the recognized utterance of ASR
should be identifiable as having been produced by ASR processing.
4. TTS Requirements
4.1. Requesting Text Playback
The SPEECHSC framework MUST allow a Media Processing Entity or
Application Server, using a control protocol, to request the TTS
Server to play back text as voice in an RTP stream.
4.2. Text Formats
4.2.1. Plain Text
The SPEECHSC framework MAY assume that all TTS servers are capable of
reading plain text. For reading plain text, framework MUST allow the
language and voicing to be indicated via session parameters. For
finer control over such properties, see [1].
4.2.2. SSML
The SPEECHSC framework MUST support Speech Synthesis Markup Language
(SSML)[1] <speak> basics, and SHOULD support other SSML tags. The
framework assumes all TTS servers are capable of reading SSML
formatted text. Internationalization of TTS in the SPEECHSC
framework, including multi-lingual output within a single utterance,
is accomplished via SSML xml:lang tags.
4.2.3. Text in Control Channel
The SPEECHSC framework assumes all TTS servers accept text over the
SPEECHSC connection for reading over the RTP connection. The
framework assumes the server can accept text either "by value"
(embedded in the protocol) or "by reference" (e.g., by de-referencing
a Uniform Resource Identifier (URI) embedded in the protocol).
4.2.4. Document Type Indication
A document type specifies the syntax in which the text to be read is
encoded. The SPEECHSC framework MUST be capable of explicitly
indicating the document type of the text to be processed, as opposed
to forcing the server to infer the content by other means.
4.3. Control Channel
The SPEECHSC framework MUST be capable of establishing the control
channel between the client and server on a per-session basis, where a
session is loosely defined to be associated with a single "call" or
"dialog". The protocol SHOULD be capable of maintaining a long-lived
control channel for multiple sessions serially, and MAY be capable of
shorter time horizons as well, including as short as for the
processing of a single utterance.
4.4. Media Origination/Termination by Control Elements
The SPEECHSC framework MUST NOT require the controlling element
(application server, media processing entity) to accept or originate
media streams. Media streams MAY source & sink from the controlled
element (ASR, TTS, etc.).
4.5. Playback Controls
The SPEECHSC framework MUST support "VCR controls" for controlling
the playout of streaming media output from SPEECHSC processing, and
MUST allow for servers with varying capabilities to accommodate such
controls. The protocol SHOULD allow clients to state what controls
they wish to use, and for servers to report which ones they honor.
These capabilities include:
o The ability to jump in time to the location of a specific marker.
o The ability to jump in time, forwards or backwards, by a specified
amount of time. Valid time units MUST include seconds, words,