Sharing speech dialog capabilities of a vehicle

• US 9,389,831 B2
• Filed: 08/06/2014
• Issued: 07/12/2016
• Est. Priority Date: 08/06/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Agosta, John Mark, Parundekar, Rahul, Sisbot, Emrah Akin
• Primary Examiner(s): Pham, Tammy
• Assistant Examiner(s): Leland, III, Edwin S
• Application Number:
  US14/453,335
• Publication Number:
  US 20160041811A1
• Time in Patent Office: 706 Days
• Field of Search: 704/275
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces

Abstract

The disclosure includes a speech-enabled device and method to share speech dialog capabilities of the speech-enabled device with a dumb device. The speech-enabled device includes a processor and a memory storing instructions that, when executed by the processor, cause the speech-enabled device to: receive speech dialog data of the dumb device that indicates a function of the dumb device; receive speech input; determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data; generate a command effective to invoke the function of the dumb device based on the speech dialog data; and send the command to the dumb device to invoke the function of the dumb device.

Claims

1. A computer-implemented method for a vehicle to enable a user to provide voice commands to a dumb device that is not configured to recognize any voice commands included in a specified set of voice commands, the method comprising:

discovering, by a speech-enabled device, a dumb device, wherein the speech-enabled device is an onboard computer of a vehicle that is configured to recognize a specified set of voice commands and the dumb device is a processor-based device that is not configured to recognize the specified set of voice commands;

receiving, by the speech-enabled device, speech dialog data of the dumb device that indicates one or more functions of the dumb device, wherein the speech dialog data includes a command portfolio of the dumb device and speech processing data that includes speech recognition data configured to enable the dumb device to recognize speech and interpret user dialog, wherein the speech recognition data describes one or more voice commands that are recognizable by the dumb device;

determining, by the speech-enabled device, that the dumb device is not configured to recognize the specified set of voice commands based on analysis of the speech dialog data;

receiving, by the speech-enabled device, a speech input that verbally indicates a function to be invoked by the dumb device, wherein the speech input includes a voice command included in the specified set of voice commands that the dumb device is not configured to recognize and the voice command verbally indicates the function to be invoked by the dumb device;

determining, by the speech-enabled device, the function of the dumb device to be invoked based on the speech input;

determining, by the speech-enabled device, that the dumb device is not configured to recognize the voice command included in the speech input;

analyzing, by the speech-enabled device, the speech dialog data and the speech input to generate a dumb device command that is effective to cause the dumb device to provide the function indicated by the speech input so that the dumb device operates as if it recognized the voice command included in the specified set of voice commands that the dumb device is not configured to recognize; and

providing, by the speech-enabled device, the dumb device command to the dumb device so that the dumb device provides the function indicated by speech input and operates as if it recognized the voice command included in the specified set of voice commands that the dumb device is not configured to recognize.

2. The method of claim 1, wherein the speech dialog data indicates the dumb device command and includes speech processing data for the speech-enabled device to recognize and interpret speech input relevant to the dumb device, wherein the speech-enabled device incudes a processor-based device that is configured to recognize any voice commands included in the specified set of voice commands.

3. The method of claim 2, wherein:

the speech processing data includes vocabulary, grammar, and conversation logic associated with the dumb device command;

the vocabulary includes one or more terms of speech input and/or speech output associated with invocation of the dumb device command;

the grammar includes a set of rules that constrain word sequence and sentence structure of the one or more terms of speech input and/or speech output; and

the conversation logic defines intermediate conversation logic states associated with the dumb device command.

4. The method of claim 2, further comprising:

determining updates to the speech processing data that are specific to a user that provides the speech input; and

recording the updates to the speech processing data.

5. The method of claim 1, wherein the speech input comprises first speech input received at a first time and second speech input received at a second time, the method further comprising:

determining, based on the speech dialog data and the first speech input, a conversation logic state after receiving the first speech input and prior to receiving the second speech input;

converting the conversation logic state to speech output; and

synthesizing, by the speech-enabled device, the speech output as an audible utterance,

wherein;

the speech output synthesized as the audible utterance comprises a request for a response from a user; and

the second speech input includes the response from the user.

6. The method of claim 5, wherein the conversation logic state comprises a first conversation logic state, the method further comprising:

mapping, based on the speech dialog data, the second speech input to a second conversation logic state; and

sending the second conversation logic state to the dumb device, wherein the dumb device is configured to initiate execution of the dumb device command to initiate performance of the function based on a conversation logic state corresponding to the first speech input and is configured to continue or complete execution of the dumb device command to continue or complete performance of the function based on the second conversation logic state.

7. The method of claim 6, wherein the second conversation logic state comprises a confirmation from the user.

8. The method of claim 1, further comprising, prior to receiving the speech input:

receiving a request from the dumb device to initiate a speech dialog with a user to determine whether the user wants to invoke the function of the dumb device;

determining, based on the speech dialog data and the received request, a conversation logic state;

converting the conversation logic state to speech output; and

synthesizing, by the speech-enabled device, the speech output as an audible utterance to the user,

wherein;

the audible utterance comprises a request for input from the user regarding invocation of the function; and

the speech input includes input from the user that confirms or requests invocation of the function.

9. The method of claim 1, further comprising:

estimating parameters associated with one or both of execution of the dumb device command or performance of the function based on a user profile of a user from which the speech input is received; and

sending the estimated parameters to the dumb device, wherein the dumb device is configured to execute the dumb device command and/or perform the function based on the estimated parameters.

10. The method of claim 1, further comprising:

establishing an ad hoc communication network between the dumb device and the speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device;

responsive to determining that the dumb device terminated a connection to the ad hoc communication network or to the speech-enabled device receiving a request to de-register the dumb device, deleting the speech dialog data of the dumb device from the speech-enabled device.

11. The method of claim 1, further comprising:

performing an authentication between the dumb device and the speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device; and

responsive to the authentication indicating at least that the speech dialog data of the dumb device is in a shared integration language that is common to speech dialog data of the speech-enabled device, establishing an ad hoc communication network between the dumb device and the speech-enabled device to communicate the speech dialog data of the dumb device to the speech-enabled device.

12. The method of claim 1, further comprising:

the speech-enabled device receiving speech dialog data from a plurality of dumb devices;

the speech-enabled device identifying a corresponding one of the plurality of dumb devices whose function is to be invoked and determining the function of the corresponding one of the plurality of dumb devices to be invoked based on the speech input by using the speech dialog data;

generating a command effective to invoke the function of the corresponding one of the plurality of dumb devices based on the speech dialog data; and

sending the command to the corresponding one of the plurality of dumb devices to invoke the function of the corresponding one of the plurality of dumb devices.

13. A speech-enabled device for a vehicle to enable a user to provide voice commands to a dumb device that is not configured to recognize any voice commands included in a specified set of voice commands, the speech-enabled device comprising:

a processor of an onboard computer of a vehicle, wherein the speech-enabled device includes the onboard computer of the vehicle and the onboard computer of the vehicle is configured to recognize a specified set of voice commands; and

a memory communicatively coupled to the processor, the memory storing instructions that, when executed by the processor, cause the speech-enabled device to;

discover a dumb device, wherein the dumb device is a processor-based device that is not configured to recognize the specified set of voice commands that is recognizable to the onboard computer of the vehicle;

establish a wireless network with the dumb device;

receive speech dialog data of the dumb device that indicates a function of the dumb device;

determining that the dumb device is not configured to recognize the specified set of commands based on analysis of the speech dialog data;

receive speech input that verbally indicates a function to be invoked by the dumb device, wherein the speech input includes a voice command included in the specified set of voice commands that the dumb device is not configured to recognize and the voice command verbally indicates the function to be invoked by the dumb device;

determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data;

generate, based on the speech dialog data, a dumb device command that is effective to cause the dumb device to provide the function indicated by the speech input so that the dumb device operates as if it recognized the voice command included in the specified set of voice commands that the dumb device is not configured to recognize; and

providing, via the wireless network established with the dumb device, the dumb device command to the dumb device, wherein responsive to receiving the dumb device command via the wireless network the dumb device executes the dumb device command and the dumb device command causes the dumb device to operate as if it recognized the voice command included in the specified set of voice commands that the dumb device is not configured to recognize.

14. The speech-enabled device of claim 13, further comprising:

a speech input device communicatively coupled to the processor and comprising a speech-to-digital data transducer configured to generate the speech input as a digital data representation of a spoken utterance of a user; and

a speech output device communicatively coupled to the processor and comprising a digital data-to-speech transducer configured to convert speech output to an audible utterance.

15. The speech-enabled device of claim 13, wherein the speech input comprises first speech input received at a first time and second speech input received at a second time and wherein the memory further stores instructions that, when executed by the processor, cause the speech-enabled device to:

determine, based on the speech dialog data and the first speech input, a conversation logic state after receiving the first speech input and prior to receiving the second speech input;

convert the conversation logic state to an utterance; and

synthesize, by the speech-enabled device, the utterance as speech output,

wherein;

the utterance synthesized as speech output requests a response from a user; and

the second speech input includes the response from the user.

16. A computer program product of a vehicle to enable a user to provide voice commands to a dumb device that is not configured to recognize any voice commands included in a specified set of voice commands, the computer program product comprising a non-transitory computer-usable medium including a computer-readable program, wherein the computer-readable program when executed on an onboard computer of the vehicle causes the computer to:

discover a dumb device, wherein the onboard computer of the vehicle is a speech-enabled device that is configured to recognize a specified set of voice commands and the dumb device is a processor-based device that is not configured to recognize the specified set of voice commands that are recognized by the onboard compute of the vehicle;

receive speech dialog data of the dumb device that indicates a function of the dumb device, wherein the speech dialog data includes a command portfolio of the dumb device and speech processing data that includes speech recognition data configured to enable the dumb device to recognize speech and interpret user dialog, wherein the speech recognition data describes one or more voice commands that are recognizable by the dumb device;

determine that the dumb device is not configured to recognize the specified set of voice commands based on analysis of the speech dialog data;

receive speech input that verbally indicates a function to be invoked by the dumb device, wherein the speech input includes a voice command included in the specified set of voice commands that the dumb device is not configured to recognize and the voice command verbally indicates the function to be invoked by the dumb device;

determine the function of the dumb device to be invoked based on the speech input by using the speech dialog data;

determine that the dumb device is not configured to recognize the voice command included in the speech input;

analyzing the speech dialog data and the speech input to generate a dumb device command that is effective to cause the dumb device to provide the function indicated by the speech input so that the dumb device operates as if it recognized the voice command included in the specified set of voice commands that the dumb device is not configured to recognize; and

providing the dumb device command to the dumb device so that the dumb device provides the function indicated by speech input and operates as if it recognized the voice command included in the specified set of voice commands that the dumb device is not configured to recognize.

17. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to, prior to receipt of the speech input:

receive a request from the dumb device to initiate a speech dialog with a user to determine whether the user wants to invoke the function of the dumb device;

determine, based on the speech dialog data and the received request, a conversation logic state;

convert the conversation logic state to speech output; and

synthesize, by a speech-enabled device, the speech output as an audible utterance to the user,

wherein;

the audible utterance comprises a request for input from the user regarding invocation of the function; and

the speech input includes input from the user that confirms or requests invocation of the function.

18. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

estimate parameters associated with one or both of execution of the command or performance of the function based on a user profile of a user from which the speech input is received; and

send the estimated parameters to the dumb device, wherein the dumb device is configured to execute the command and/or perform the function based on the estimated parameters.

19. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

establish an ad hoc communication network between the dumb device and a speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device; and

responsive to a determination that the dumb device terminated a connection to the ad hoc communication network or to receipt by the speech-enabled device of a request to de-register the dumb device, delete the speech dialog data of the dumb device from the speech-enabled device.

20. The computer program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to:

perform an authentication between the dumb device and a speech-enabled device prior to the speech-enabled device receiving the speech dialog data of the dumb device; and

responsive to the authentication indicating at least that the speech dialog data of the dumb device is in a shared integration language that is common to speech dialog data of the speech-enabled device, establish an ad hoc communication network between the dumb device and the speech-enabled device to communicate the speech dialog data of the dumb device to the speech-enabled device.