SHARED SPEECH DIALOG CAPABILITIES

• US 20160041811A1
• Filed: 08/06/2014
• Published: 02/11/2016
• Est. Priority Date: 08/06/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: PARUNDEKAR, Rahul, AGOSTA, John Mark, SISBOT, Emrah Akin
• Granted Patent: US 9,389,831 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces

Abstract

The disclosure includes a speech-enabled device and method to share speech dialog capabilities of the speech-enabled device with a dumb device. The speech-enabled device includes a processor and a memory storing instructions that, when executed by the processor, cause the speech-enabled device to: receive speech dialog data of the dumb device that indicates a function of the dumb device; receive speech input; determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data; generate a command effective to invoke the function of the dumb device based on the speech dialog data; and send the command to the dumb device to invoke the function of the dumb device.

Claims

1. A computer-implemented method comprising:

a speech-enabled device receiving speech dialog data of a dumb device that indicates a function of the dumb device;

receiving speech input by the speech-enabled device;

the speech-enabled device determining the function of the dumb device to be invoked based on the speech input by using the speech dialog data;

generating a command effective to invoke the function of the dumb device based on the speech dialog data; and

sending the command to the dumb device to invoke the function of the dumb device.

2. The method of claim 1, wherein the speech dialog data indicates the command effective to invoke the function and includes speech processing data for a speech-enabled device to recognize and interpret speech input relevant to the dumb device.

3. The method of claim 2, wherein:

the speech processing data includes vocabulary, grammar, and conversation logic associated with the command;

the vocabulary includes one or more terms of speech input and/or speech output associated with invocation of the command;

the grammar includes a set of rules that constrain word sequence and sentence structure of the one or more terms of speech input and/or speech output; and

the conversation logic defines intermediate conversation logic states associated with the command.

4. The method of claim 2, further comprising:

determining updates to the speech processing data that are specific to a user that provides the speech input; and

recording the updates to the speech processing data.

5. The method of claim 1, wherein the speech input comprises first speech input received at a first time and second speech input received at a second time, the method further comprising:

determining, based on the speech dialog data and the first speech input, a conversation logic state after receiving the first speech input and prior to receiving the second speech input;

converting the conversation logic state to speech output; and

synthesizing, by a speech-enabled device, the speech output as an audible utterance,

wherein;

the speech output synthesized as the audible utterance comprises a request for a response from a user; and

the second speech input includes the response from the user.

6. The method of claim 5, wherein the conversation logic state comprises a first conversation logic state, the method further comprising:

mapping, based on the speech dialog data, the second speech input to a second conversation logic state; and

sending the second conversation logic state to the dumb device, wherein the dumb device is configured to initiate execution of the command to initiate performance of the function based on a conversation logic state corresponding to the first speech input and is configured to continue or complete execution of the command to continue or complete performance of the function based on the second conversation logic state.

7. The method of claim 6, wherein the second conversation logic state comprises a confirmation from the user.

8. The method of claim 1, further comprising, prior to receiving the speech input:

receiving a request from the dumb device to initiate a speech dialog with a user to determine whether the user wants to invoke the function of the dumb device;

determining, based on the speech dialog data and the received request, a conversation logic state;

converting the conversation logic state to speech output; and

synthesizing, by a speech-enabled device, the speech output as an audible utterance to the user,

wherein;

the audible utterance comprises a request for input from the user regarding invocation of the function; and

the speech input includes input from the user that confirms or requests invocation of the function.

9. The method of claim 1, further comprising:

estimating parameters associated with one or both of execution of the command or performance of the function based on a user profile of a user from which the speech input is received; and

sending the estimated parameters to the dumb device, wherein the dumb device is configured to execute the command and/or perform the function based on the estimated parameters.

10. The method of claim 1, further comprising:

establishing an ad hoc communication network between the dumb device and a speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device;

responsive to determining that the dumb device terminated a connection to the ad hoc communication network or to the speech-enabled device receiving a request to de-register the dumb device, deleting the speech dialog data of the dumb device from the speech-enabled device.

11. The method of claim 1, further comprising:

performing an authentication between the dumb device and a speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device; and

responsive to the authentication indicating at least that the speech dialog data of the dumb device is in a shared integration language that is common to speech dialog data of the speech-enabled device, establishing an ad hoc communication network between the dumb device and the speech-enabled device to communicate the speech dialog data of the dumb device to the speech-enabled device.

12. The method of claim 1, further comprising:

the speech-enabled device receiving speech dialog data from a plurality of dumb devices;

the speech-enabled device identifying a corresponding one of the plurality of dumb devices whose function is to be invoked and determining the function of the corresponding one of the plurality of dumb devices to be invoked based on the speech input by using the speech dialog data;

generating a command effective to invoke the function of the corresponding one of the plurality of dumb devices based on the speech dialog data; and

sending the command to the corresponding one of the plurality of dumb devices to invoke the function of the corresponding one of the plurality of dumb devices.

13. A speech-enabled device comprising:

a processor; and

a memory communicatively coupled to the processor, the memory storing instructions that, when executed by the processor, cause the speech-enabled device to;

receive speech dialog data of a dumb device that indicates a function of the dumb device;

receive speech input;

determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data;

generate a command effective to invoke the function of the dumb device based on the speech dialog data; and

send the command to the dumb device to invoke the function of the dumb device.

14. The speech-enabled device of claim 13, further comprising:

a speech input device communicatively coupled to the processor and comprising a speech-to-digital data transducer configured to generate the speech input as a digital data representation of a spoken utterance of a user; and

a speech output device communicatively coupled to the processor and comprising a digital data-to-speech transducer configured to convert speech output to an audible utterance.

15. The speech-enabled device of claim 13, wherein the speech input comprises first speech input received at a first time and second speech input received at a second time and wherein the memory further stores instructions that, when executed by the processor, cause the speech-enabled device to:

determine, based on the speech dialog data and the first speech input, a conversation logic state after receiving the first speech input and prior to receiving the second speech input;

convert the conversation logic state to an utterance; and

synthesize, by the speech-enabled device, the utterance as speech output,

wherein;

the utterance synthesized as speech output requests a response from a user; and

the second speech input includes the response from the user.

16. A computer program product comprising a non-transitory computer-usable medium including a computer-readable program, wherein the computer-readable program when executed on a computer causes the computer to:

receive speech dialog data of a dumb device that indicates a function of the dumb device;

receive speech input;

determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data;

generate a command effective to invoke the function of the dumb device based on the speech dialog data; and

send the command to the dumb device to invoke the function of the dumb device.

17. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to, prior to receipt of the speech input:

receive a request from the dumb device to initiate a speech dialog with a user to determine whether the user wants to invoke the function of the dumb device;

determine, based on the speech dialog data and the received request, a conversation logic state;

convert the conversation logic state to speech output; and

synthesize, by a speech-enabled device, the speech output as an audible utterance to the user,

wherein;

the audible utterance comprises a request for input from the user regarding invocation of the function; and

the speech input includes input from the user that confirms or requests invocation of the function.

18. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

estimate parameters associated with one or both of execution of the command or performance of the function based on a user profile of a user from which the speech input is received; and

send the estimated parameters to the dumb device, wherein the dumb device is configured to execute the command and/or perform the function based on the estimated parameters.

19. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

establish an ad hoc communication network between the dumb device and a speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device; and

responsive to a determination that the dumb device terminated a connection to the ad hoc communication network or to receipt by the speech-enabled device of a request to de-register the dumb device, delete the speech dialog data of the dumb device from the speech-enabled device.

20. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

perform an authentication between the dumb device and a speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device; and

responsive to the authentication indicating at least that the speech dialog data of the dumb device is in a shared integration language that is common to speech dialog data of the speech-enabled device, establish an ad hoc communication network between the dumb device and the speech-enabled device to communicate the speech dialog data of the dumb device to the speech-enabled device.