System and method for adding speech recognition to GUI applications

• US 7,328,158 B1
• Filed: 04/11/2003
• Issued: 02/05/2008
• Est. Priority Date: 04/11/2003
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Walker, William D., Burridge, Richard N., Korn, Peter A., Braun, Oliver
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): Han; Qi
• Application Number:
  US10/412,432
• Time in Patent Office: 1,761 Days
• Field of Search: 704/255, 704/231, 704/270, 704/275
• US Class Current: 704/270
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A system and method for adding speech recognition to graphics user interface based applications for a computer system is described. The speech recognition system provides a profile generation module that enables users to create editable profiles of grammar and corresponding actions that may be used by a speech agent and in combination with an accessibility framework to drive that application based on the user'"'"'s spoken commands.

Claims

1. A computer system comprising:

a processor;

a memory storage unit;

a plurality of user applications;

an operating system comprising a kernel, said kernel comprising a plurality of kernel modules; and

a speech recognition system for dynamically encoding a plurality of editable profile information representative of audio signals to be translated into text to be added to said plurality of user applications, the plurality of editable profile information is in a form that is user accessible and modifiable, such that modifications of the editable profile information does not require access to underlying speech recognition source code to add specific speech features for any one of the plurality of user applications,

wherein a user provides a custom override profile information to replace a subset of grammar rules in one of the plurality of editable profile information that is associated with a combination of one of the plurality of user applications and the user.

2. The computer system of claim 1, wherein said plurality of user applications are graphic user interface applications.

3. The computer system of claim 2, further comprising a speech agent for processing said editable profile information.

4. The computer system of claim 3, wherein said speech agent further reads said editable profile information for a particular one of said plurality of user applications to drive that application based on a user provided spoken commands.

5. The computer system of claim 4, further comprising accessibility framework to enable the speech recognition system perform applications-specific speech operations.

6. The computer system of claim 5, wherein said accessibility framework further permits said speech agent to determine what speech actions may be implemented by each one of said plurality of user applications and events required to cause said actions.

7. The computer system of claim 6, wherein said speech recognition system comprises a profile generation module to generate the editable profile information.

8. The computer system of claim 7, wherein said editable profile information is user defined to include a list of words and phrases specific to said user'"'"'s use.

9. The computer system of claim 8, wherein said list of words and phrases define application-specific speech grammar.

10. The computer system of claim 9, wherein said user defined profile information further comprises a list of actions to be taken in an application when said user utters a speech command.

11. The computer system of claim 10, wherein said speech recognition system further comprises a grammar customization module for enabling said user customize said profile information to make words spoken by said, user more understandable.

12. The computer system as recited in claim 1, wherein the operating system is in communication with a speech subsystem to provide speech recognition capability to the user.

13. The computer system as recited in claim 12, wherein the speech recognition system interfaces with the speech subsystem without a change in programming code contained in any one of the application plurality of applications.

14. The computer system as recited in claim 13, further comprising as audio input device to enable a user to interact with the computer system through an audio command.

15. The computer system as recited in claim 14, wherein the speech recognition system further comprising a custom module to create a customized speech recognition profile by customizing the editable profile information so as to make the audio command more understandable to the computer system.

16. The computer system as recited in claim 15, wherein the user has a plurality of customized speech recognition profiles to interact with the plurality of user applications.

17. The computer system as recited in claim 16, wherein the audio command is inputted to the operating system through the audio input device, the speech subsystem transfers the audio command to the speech recognition system including a conversion module to convert the audio command to a text command using a selected one of the plurality of customized speech recognition profiles.

18. The computer system as recited in claim 17, wherein the customized speech recognition profile includes a list of words and phrases specific to the user.

19. A computer system including a processor, a memory storage unit, and an application, comprising:

an operating system in communication with a speech subsystem to provide speech recognition capabilities to a user of the computer system;

a speech recognition system that provides speech-enabling capability to the application, wherein the speech recognition system interfaces with the speech subsystem to provide the application access to the speech subsystem without a change in a programming code of the application, the speech recognition system including,

a grammar generation module to generate speech recognition grammar and speech action, wherein the speech action is an operation executed in the application when the user utters an audio command;

a profile generation module to generate a speech recognition profiles, that is representative of audio signals to be translated into text, including the speech recognition grammar and the speech action for the application, wherein a custom override speech recognition profile is provided by the user to replace a subset of grammar rules defined in the speech recognition profile and the speech recognition profile is tied to a combination of the user and the application;

a speech agent to read the speech recognition profile for a combination of the user and the application; and

an accessibility framework to determine that the speech action is permissible by the application and to control the application based on the audio command.

20. The computer system as recited in claim 19, wherein the accessibility framework further comprises a logic to permit the speech agent to determine what speech actions may be implemented by the application.

21. The computer system as recited in claim 20, wherein the accessibility framework further comprises a logic to permit the speech agent to determine what events required to cause the speech action.

22. The computer system as recited in claim 19, wherein the speech recognition profile is customized to make spoken words unique to the user understandable to the computer system.

23. The computer system as recited in claim 22, further comprising a custom module to create a customized speech recognition profile by customizing the speech recognition profile so as to make the audio command more understandable to the computer system.

24. The computer system as recited in claim 23, wherein the user has a plurality of customized speech recognition profiles to interact with the application.

25. The computer system as recited in claim 24, wherein the audio command is inputted to the operating system through an audio input device, the operating system speech subsystem transfers the command to the speech recognition system including a conversion module to convert the audio command to a text command using a selected one of the plurality of customized speech recognition profiles.

26. The computer system as recited in claim 23, wherein the customized speech recognition profile include a list of words and phrases specific to the user.