prosody header fields that define the behavior of the voice being
synthesized, as well as the actual marked-up text to be spoken. If
specific voice and prosody parameters are specified as part of the
speech markup text, it will take precedence over the values specified
in the header fields and those set using a previous SET-PARAMS
request.
When applying voice parameters, there are 3 levels of scope. The
highest precedence are those specified within the speech markup text,
followed by those specified in the header fields of the SPEAK request
and, hence, apply for that SPEAK request only, followed by the
session default values that can be set using the SET-PARAMS request
and apply for the whole session moving forward.
If the resource is idle and the SPEAK request is being actively
processed, the resource will respond with a success status code and a
request-state of IN-PROGRESS.
If the resource is in the speaking or paused states (i.e., it is in
the middle of processing a previous SPEAK request), the status
returns success and a request-state of PENDING. This means that this
SPEAK request is in queue and will be processed after the currently
active SPEAK request is completed.
For the synthesizer resource, this is the only request that can
return a request-state of IN-PROGRESS or PENDING. When the text to
be synthesized is complete, the resource will issue a SPEAK-COMPLETE
event with the request-id of the SPEAK message and a request-state of
COMPLETE.
Example:
C->S:SPEAK 543257 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543257 200 IN-PROGRESS
S->C:SPEAK-COMPLETE 543257 COMPLETE MRCP/1.0
Completion-Cause:000 normal
7.9. STOP
The STOP method from the client to the server tells the resource to
stop speaking if it is speaking something.
The STOP request can be sent with an active-request-id-list header
field to stop the zero or more specific SPEAK requests that may be in
queue and return a response code of 200(Success). If no active-
request-id-list header field is sent in the STOP request, it will
terminate all outstanding SPEAK requests.
If a STOP request successfully terminated one or more PENDING or
IN-PROGRESS SPEAK requests, then the response message body contains
an active-request-id-list header field listing the SPEAK request-ids
that were terminated. Otherwise, there will be no active-request-
id-list header field in the response. No SPEAK-COMPLETE events will
be sent for these terminated requests.
If a SPEAK request that was IN-PROGRESS and speaking was stopped, the
next pending SPEAK request, if any, would become IN-PROGRESS and move
to the speaking state.
If a SPEAK request that was IN-PROGRESS and in the paused state was
stopped, the next pending SPEAK request, if any, would become
IN-PROGRESS and move to the paused state.
Example:
C->S:SPEAK 543258 MRCP/1.0
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:STOP 543259 200 MRCP/1.0
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
7.10. BARGE-IN-OCCURRED
The BARGE-IN-OCCURRED method is a mechanism for the client to
communicate a barge-in-able event it detects to the speech resource.
This event is useful in two scenarios,
1. The client has detected some events like DTMF digits or other
barge-in-able events and wants to communicate that to the
synthesizer.
2. The recognizer resource and the synthesizer resource are in
different servers. In which case the client MUST act as a Proxy
and receive event from the recognition resource, and then send a
BARGE-IN-OCCURRED method to the synthesizer. In such cases, the
BARGE-IN-OCCURRED method would also have a proxy-sync-id header
field received from the resource generating the original event.
If a SPEAK request is active with kill-on-barge-in enabled, and the
BARGE-IN-OCCURRED event is received, the synthesizer should stop
streaming out audio. It should also terminate any speech requests
queued behind the current active one, irrespective of whether they
have barge-in enabled or not. If a barge-in-able prompt was playing
and it was terminated, the response MUST contain the request-ids of
all SPEAK requests that were terminated in its active-request-id-
list. There will be no SPEAK-COMPLETE events generated for these
requests.
If the synthesizer and the recognizer are on the same server, they
could be optimized for a quicker kill-on-barge-in response by having
them interact directly based on a common RTSP session-id. In these
cases, the client MUST still proxy the recognition event through a
BARGE-IN-OCCURRED method, but the synthesizer resource may have
already stopped and sent a SPEAK-COMPLETE event with a barge-in
completion cause code. If there were no SPEAK requests terminated as
a result of the BARGE-IN-OCCURRED method, the response would still be
a 200 success, but MUST not contain an active-request-id-list header
field.
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:BARGE-IN-OCCURRED 543259 200 MRCP/1.0
Proxy-Sync-Id:987654321
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
7.11. PAUSE
The PAUSE method from the client to the server tells the resource to
pause speech, if it is speaking something. If a PAUSE method is
issued on a session when a SPEAK is not active, the server SHOULD
respond with a status of 402 or "Method not valid in this state". If
a PAUSE method is issued on a session when a SPEAK is active and
paused, the server SHOULD respond with a status of 200 or "Success".
If a SPEAK request was active, the server MUST return an active-
request-id-list header with the request-id of the SPEAK request that
was paused.
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:PAUSE 543259 MRCP/1.0
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
7.12. RESUME
The RESUME method from the client to the server tells a paused
synthesizer resource to continue speaking. If a RESUME method is
issued on a session when a SPEAK is not active, the server SHOULD
respond with a status of 402 or "Method not valid in this state". If
a RESUME method is issued on a session when a SPEAK is active and
speaking (i.e., not paused), the server SHOULD respond with a status
of 200 or "Success". If a SPEAK request was active, the server MUST
return an active-request-id-list header with the request-id of the
SPEAK request that was resumed
Example:
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:PAUSE 543259 MRCP/1.0
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
C->S:RESUME 543260 MRCP/1.0
S->C:MRCP/1.0 543260 200 COMPLETE
Active-Request-Id-List:543258
7.13. CONTROL
The CONTROL method from the client to the server tells a synthesizer
that is speaking to modify what it is speaking on the fly. This
method is used to make the synthesizer jump forward or backward in
what it is being spoken, change speaker rate and speaker parameters,
etc. It affects the active or IN-PROGRESS SPEAK request. Depending
on the implementation and capability of the synthesizer resource, it
may allow this operation or one or more of its parameters.
When a CONTROL to jump forward is issued and the operation goes
beyond the end of the active SPEAK method’s text, the request
succeeds. A SPEAK-COMPLETE event follows the response to the CONTROL
method. If there are more SPEAK requests in the queue, the
synthesizer resource will continue to process the next SPEAK method.
When a CONTROL to jump backwards is issued and the operation jumps to
the beginning of the speech data of the active SPEAK request, the
response to the CONTROL request contains the speak-restart header.
These two behaviors can be used to rewind or fast-forward across
multiple speech requests, if the client wants to break up a speech
markup text into multiple SPEAK requests.
If a SPEAK request was active when the CONTROL method was received,
the server MUST return an active-request-id-list header with the
Request-id of the SPEAK request that was active.
Example:
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:CONTROL 543259 MRCP/1.0
Prosody-rate:fast
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
C->S:CONTROL 543260 MRCP/1.0
Jump-Size:-15 Words
S->C:MRCP/1.0 543260 200 COMPLETE
Active-Request-Id-List:543258
7.14. SPEAK-COMPLETE
This is an Event message from the synthesizer resource to the client
indicating that the SPEAK request was completed. The request-id
header field WILL match the request-id of the SPEAK request that
initiated the speech that just completed. The request-state field
should be COMPLETE indicating that this is the last Event with that
request-id, and that the request with that request-id is now
complete. The completion-cause header field specifies the cause code
pertaining to the status and reason of request completion such as the
SPEAK completed normally or because of an error or kill-on-barge-in,
etc.
Example:
C->S:SPEAK 543260 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543260 200 IN-PROGRESS
S->C:SPEAK-COMPLETE 543260 COMPLETE MRCP/1.0
Completion-Cause:000 normal
7.15. SPEECH-MARKER
This is an event generated by the synthesizer resource to the client
when it hits a marker tag in the speech markup it is currently
processing. The request-id field in the header matches the SPEAK
request request-id that initiated the speech. The request-state
field should be IN-PROGRESS as the speech is still not complete and
there is more to be spoken. The actual speech marker tag hit,
describing where the synthesizer is in the speech markup, is returned
in the speech-marker header field.
Example:
C->S:SPEAK 543261 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<mark name="here"/>
<sentence>The subject is
<prosody rate="-20%">ski trip</prosody>
</sentence>
<mark name="ANSWER"/>
</paragraph>
</speak>
S->C:MRCP/1.0 543261 200 IN-PROGRESS
S->C:SPEECH-MARKER 543261 IN-PROGRESS MRCP/1.0
Speech-Marker:here
S->C:SPEECH-MARKER 543261 IN-PROGRESS MRCP/1.0
Speech-Marker:ANSWER
S->C:SPEAK-COMPLETE 543261 COMPLETE MRCP/1.0
Completion-Cause:000 normal
8. Speech Recognizer Resource
The speech recognizer resource is capable of receiving an incoming
voice stream and providing the client with an interpretation of what
was spoken in textual form.
8.1. Recognizer State Machine
The recognizer resource is controlled by MRCP requests from the
client. Similarly, the resource can respond to these requests or
generate asynchronous events to the server to indicate certain
conditions during the processing of the stream. Hence, the
recognizer maintains states to correlate MRCP requests from the
client. The state transitions are described below.
Idle Recognizing Recognized
State State State
| | |
|---------RECOGNIZE---->|---RECOGNITION-COMPLETE-->|
|<------STOP------------|<-----RECOGNIZE-----------|
| | |
| | |-----------|
| |--------| GET-RESULT |
| START-OF-SPEECH | |---------->|
|------------| |------->| |
| | |----------| |
| DEFINE-GRAMMAR | RECOGNITION-START-TIMERS |
|<-----------| |<---------| |
| | |
| | |
|-------| | |
| STOP | |
|<------| | |
| |
|<-------------------STOP--------------------------|
|<-------------------DEFINE-GRAMMAR----------------|
8.2. Recognizer Methods
The recognizer supports the following methods.
recognizer-method = SET-PARAMS
/ GET-PARAMS
/ DEFINE-GRAMMAR
/ RECOGNIZE
/ GET-RESULT
/ RECOGNITION-START-TIMERS
/ STOP
8.3. Recognizer Events
The recognizer may generate the following events.
recognizer-event = START-OF-SPEECH
/ RECOGNITION-COMPLETE
8.4. Recognizer Header Fields
A recognizer message may contain header fields containing request
options and information to augment the Method, Response, or Event
message it is associated with.
recognizer-header = confidence-threshold ; Section 8.4.1
/ sensitivity-level ; Section 8.4.2
/ speed-vs-accuracy ; Section 8.4.3
/ n-best-list-length ; Section 8.4.4
/ no-input-timeout ; Section 8.4.5
/ recognition-timeout ; Section 8.4.6
/ waveform-url ; Section 8.4.7
/ completion-cause ; Section 8.4.8
/ recognizer-context-block ; Section 8.4.9
/ recognizer-start-timers ; Section 8.4.10
/ vendor-specific ; Section 8.4.11
/ speech-complete-timeout ; Section 8.4.12
/ speech-incomplete-timeout; Section 8.4.13
/ dtmf-interdigit-timeout ; Section 8.4.14
/ dtmf-term-timeout ; Section 8.4.15
/ dtmf-term-char ; Section 8.4.16
/ fetch-timeout ; Section 8.4.17
/ failed-uri ; Section 8.4.18
/ failed-uri-cause ; Section 8.4.19
/ save-waveform ; Section 8.4.20