MACHINE, SYSTEM AND METHOD FOR USER-GUIDED TEACHING AND MODIFYING OF VOICE COMMANDS AND ACTIONS EXECUTED BY A CONVERSATIONAL LEARNING SYSTEM

• US 20100185445A1
• Filed: 01/21/2009
• Published: 07/22/2010
• Est. Priority Date: 01/21/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Viswanathan, Mahesh, Comerford, Liam D.
• Granted Patent: US 8,407,057 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A machine, system and method for user-guided teaching and modifications of voice commands and actions to be executed by a conversational learning system. The machine includes a system bus for communicating data and control signals received from the conversational learning system to a computer system, a vehicle data and control bus for connecting devices and sensors in the machine, a bridge module for connecting the vehicle data and control bus to the system bus, machine subsystems coupled to the vehicle data and control bus having a respective user interface for receiving a voice command or input signal from a user, a memory coupled to the system bus for storing action command sequences learned for a new voice command and a processing unit coupled to the system bus for automatically executing the action command sequences learned when the new voice command is spoken.

Claims

1. A machine having a conversational system, said machine comprising:

a system bus for communicating data and control signals received from said conversational system to a computer system in said machine;

a vehicle data and control bus for connecting devices and sensors in said machine;

a bridge module for connecting said vehicle data and control bus to said system bus;

one or more respective machine subsystems coupled to said vehicle data and control bus connected to said computer system, said one or more respective machine subsystems having a respective user interface for receiving a voice command or input signal from a user for modifying a voice command set in said conversational system of said machine;

a memory coupled to said system bus for storing one or more action command sequences learned for a new voice command, wherein said one or more action command sequences comprises at least one of;

one or more voice commands, one or more actions performed on one or more physical interfaces of said one or more respective machine subsystems coupled to said vehicle data and control bus and a combination of said one or more voice commands and said one or more actions performed; and

a central processing unit coupled to said system bus for executing said one or more action command sequences learned when said new voice command is spoken.

2. The machine according to claim 1, wherein said conversational system further comprises:

a speech recognition module for decoding spoken language input received from a user using a respective spoken user interface of said one or more respective machine subsystems coupled to said data and control bus;

a gesture recognition module for decoding indications made via a respective physical user interface of said one or more respective machine subsystems coupled to said vehicle data and control bus; and

a haptic module for receiving haptic input via a respective physical user interface of said one or more respective machine subsystems coupled to said vehicle data and control bus.

3. The machine according to claim 2, wherein said conversational system further comprises:

a verification module for recording responses received from said one or more respective machine subsystems in order to detect any errors in future execution of said one or more action command sequences corresponding to said new voice command learned.

4. The machine according to claim 3, further comprising:

a command learning module for determining whether or not a new command name provided for said new voice command exists and, if said new command name provided is determined not to exist, said command name module being configured to create said new command name for initiating learning of said new voice command.

5. The machine according to claim 4, wherein said gesture recognition module observes said one or more action command sequences to be associated with performance of said new voice command for expanding said voice command set utilized by said one or more respective machine subsystems connected to said vehicle data and control bus connected to said computer system.

6. A method for operating a machine having a conversational system, said method comprising the steps of:

invoking a dialog with said conversational system for modifying a voice command set in said conversational system installed in said machine, wherein said invoking step comprises at least one of;

making an utterance and inputting a signal;

teaching one or more action command sequences for a new voice command to be learned by said conversational system for user-guided modification of said voice command set in said conversational system, wherein said one or more action command sequences comprises at least one of;

one or more voice commands, one or more actions performed on one or more physical interfaces corresponding to a plurality of devices coupled to said conversational system, and a combination of said one or more voice commands and said one or more actions performed;

saving said one or more action command sequences taught for said new voice command; and

performing said one or more action command sequences saved for said new voice command taught during execution of said new voice command.

7. The method according to claim 6, wherein said invoking step further comprises the steps of:

providing, responsive to prompting by said dialog invoked with said conversational system, a unique command name corresponding to said new voice command to be learned by said conversational system;

determining by said conversational system whether or not said unique command name provided exists; and

if said unique command name provided is determined not to exist, said conversational system creates said unique command name for invoking said dialog for user-guided learning of said new voice command.

8. The method according to claim 7, wherein said saving step further comprises the steps of:

observing, by said conversational system, said one or more action command sequences to be associated with performance of said new voice command taught; and

building a command list for utilization of said plurality of devices connected to said system bus of said conversational system for execution of said one or more action command sequences in response to utterance of said new voice command.

9. The method according to claim 8, wherein said observing step further comprises the step of:

monitoring, using a vehicle data and control bus adjustments made to one or more physical controls controlling one or more of said plurality of devices connected to said vehicle data and control bus for performing said one or more action command sequences corresponding to said new voice command taught.

10. The method according to claim 9, further comprising the steps of:

recording responses received from said plurality of devices for detecting any errors in future execution of said one or more action command sequences corresponding to said new voice command taught.

11. A computer program product for modifying a command vocabulary set of a conversational system, said computer program product comprising:

a computer readable storage medium;

first program instructions to initiate a dialog request with said conversational system for modifying said command vocabulary set of said conversational system, wherein said dialog request includes uttering via a user interface of said conversational system a unique command phrase corresponding to a new command phrase to be learned;

second program instructions to save, responsive to said dialog request initiated, one or more action command sequences taught for said new command phrase in a command list for utilization of one or more subsystems connected to a vehicle data and control bus for communicating with said conversational system;

third program instructions to execute said one or more action command sequences saved for said new command phrase learned upon utterance of said new command phrase learned, and wherein said first, second and third program instructions are recorded on said computer readable storage medium.

12. The computer program product according to claim 11, further comprising:

fourth program instructions to record responses from said one or more subsystems for detecting errors in execution of said one or more action command sequences corresponding to said new command phrase learned, wherein said fourth program instructions are recorded on said computer readable storage medium.

13. The computer program product according to claim 12, wherein said first program instructions further comprise instructions to determine whether or not said unique command phrase uttered exists and, if said unique command phrase uttered is determined not to exist, said first program instructions further comprise instructions to create said unique command phrase for initiating learning of said new command phrase.

14. The computer program product according to claim 13, wherein said second program instructions further comprise instructions to observe said one or more action command sequences to be associated with said new command phrase uttered, and to build said command list for utilization of said one or more subsystems connected to said vehicle data and control bus coupled to said conversational system for execution of said one or more action command sequences in response to utterance of said new command phrase.

15. The computer program product according to claim 14, wherein said second program instructions further comprise instructions to monitor adjustments made to one or more physical controls controlling one or more subsystems linked to said vehicle data and control bus coupled to said conversational system for performing said one or more action command sequences corresponding to said new voice command learned.

16. A computer system for expanding an existing voice command set used by one or more subsystems coupled to said computer system, said computer system comprising:

first program instructions to initiate a dialog request with a conversational system having a command modification tool and coupled to said computer system for modifying said voice command set of said conversational system, wherein said dialog request includes uttering via a user interface of said conversational system a unique command phrase corresponding to a new command phrase to be learned;

second program instructions to save, responsive to said dialog request initiated, one or more action command sequences taught for said new command phrase in a command list for utilization of one or more subsystems connected to a vehicle data and control bus for communicating with said conversational system;

third program instructions to execute said one or more action command sequences saved for said new command phrase learned upon utterance of said new command phrase learned;

a computer readable storage medium for storing each of said first, second and third program instructions; and

a central processing unit for executing each of said first, second and third program instructions.

17. The computer system according to claim 16, further comprising:

fourth program instructions to record responses from said one or more subsystems for detecting errors in execution of said one or more action command sequences corresponding to said new command phrase learned, wherein said fourth program instructions are stored on said computer readable storage medium for execution by said central processing unit.

18. The computer system according to claim 17, wherein said first program instructions further comprise instructions to determine whether or not said unique command phrase uttered exists and, if said unique command phrase uttered is determined not to exist, said first program instructions further comprise instructions to create said unique command phrase for initiating learning of said new command phrase.

19. The computer system according to claim 18, wherein said second program instructions further comprise instructions to observe said one or more action command sequences to be associated with execution of said new command phrase uttered, and to build said command list for utilization of said one or more subsystems connected to said vehicle data and control bus coupled to said conversational system for execution of said one or more action command sequences in response to utterance of said new command phrase.

20. The computer system according to claim 19, wherein said second program instructions further comprise instructions to monitor adjustments made to one or more physical controls controlling one or more subsystems linked to said vehicle data and control bus coupled to said conversational system for executing said one or more action command sequences corresponding to said new voice command learned.

21. A process for deploying computing infrastructure comprising integrating computer-readable code into a computing system of a machine, wherein said code in combination with said computing system is capable of modifying a command vocabulary set in a conversational system deployed on said machine, said process comprising the steps of:

invoking a dialog request, using a command modification tool of said conversational system deployed on said machine, for modifying a voice command set in said conversational system, wherein said invoking step comprises at least one of;

making an utterance and inputting a signal;

teaching one or more action command sequences for a new voice command to be learned by said conversational system for user-guided modification of said voice command set in said conversational system, wherein said one or more action command sequences comprises at least one of;

one or more voice commands, one or more actions performed on one or more physical interfaces corresponding to a plurality of devices coupled to said conversational system, and a combination of said one or more voice commands and said one or more actions performed;

saving said one or more action command sequences taught for said new voice command; and

executing said one or more action command sequences saved for said new voice command taught during execution of said new voice command.

22. The process according to claim 21, wherein said initiating step further comprises the steps of:

determining by said conversational system whether or not said unique command name uttered exists; and

if said unique command name uttered is determined not to exist, said conversational system creates said unique command name for initiating learning of said new voice command.

23. The process according to claim 22, wherein said saving step further comprises the steps of:

observing, by said conversational system, said one or more action command sequences to be associated with execution of said new voice command uttered; and

building said command list for utilization of said one or more subsystems connected to a vehicle data and control bus coupled to said conversational system for execution of said one or more action command sequences in response to utterance of said new voice command.

24. The process according to claim 23, further comprising the step of:

recording responses from said one or more subsystems for detecting errors in execution of said one or more action command sequences corresponding to said new voice command learned.

25. The process according to claim 24, wherein said observing step further comprises the step of:

monitoring, using a vehicle data and control bus, adjustments made to one or more physical controls controlling one or more of said plurality of devices connected to said vehicle data and control bus for execution of said one or more action command sequences corresponding to said new voice command taught.