Speech recognition interface system suitable for window systems and speech mail systems

• US 5,632,002 A
• Filed: 12/28/1993
• Issued: 05/20/1997
• Est. Priority Date: 12/28/1992
• Status: Expired due to Term

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Seto, Shigenobu, Hashimoto, Hideki, Yamaguchi, Koji, Nagata, Yoshifumi, Shinchi, Hideaki, Takebayashi, Yoichi
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Dorvil, Richemond
• Application Number:
  US08/178,731
• Time in Patent Office: 1,239 Days
• Field of Search: 395/12, 395/2.4, 395/2.84, 395/2.79, 395/2.67, 364/419.08
• US Class Current: 704/231
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/533 Voice mail systems

Abstract

A speech recognition interface system capable of handling a plurality of application programs simultaneously, and realizing convenient speech input and output modes which are suitable for the applications in the window systems and the speech mail systems. The system includes a speech recognition unit for carrying out a speech recognition processing for a speech input made by a user to obtain a recognition result; a program management table for managing program management data indicating a speech recognition interface function required by each application program; and a message processing unit for exchanging messages with the plurality of application programs in order to specify an appropriate recognition vocabulary to be used in the speech recognition processing of the speech input to the speech recognition unit, and to transmit the recognition result for the speech input obtained by the speech recognition unit by using the appropriate recognition vocabulary to appropriate ones of the plurality of application programs, according to the program management data managed by the program management table.

Claims

1. A speech recognition interface system for enabling a speech control of a plurality of application programs executed on a computer, comprising:

speech recognition means for carrying out a speech recognition processing for a speech input made by a user to obtain a recognition result;

program management means for managing program management data indicating a speech recognition interface function required by each application program; and

message processing means for exchanging messages with said plurality of application programs in order to specify an appropriate recognition vocabulary to be used in the speech recognition processing of the speech input to the speech recognition means, and to transmit the recognition result for the speech input obtained by the speech recognition means by using the appropriate recognition vocabulary to appropriate ones of said plurality of application programs, according to the program management data managed by the program management means.

2. The system of claim 1, wherein the message processing means also exchanges messages with said plurality of application programs in order to update the program management data in the program management means according to an internal state of each application program.

3. The system of claim 1, wherein the program management data include an ID of each application program, input masks specifying types of the messages to be exchanged by each application program with the message processing means, a speech input flag indicating a presence/absence of a speech focus on each application program, and a recognition vocabulary list specifying the appropriate recognition vocabulary for each application program.

4. The system of claim 1, further comprising a window system connected with said plurality of application programs for providing graphical user interface functions to said plurality of application programs.

5. The system of claim 4, wherein the said plurality of application program include a recognition vocabulary display program for displaying the appropriate recognition vocabulary currently set by the message processing means to the speech recognition means, through the window system.

6. The system of claim 4, wherein the message processing means changes the appropriate recognition vocabulary to be used in the speech recognition processing of the speech input specified to the speech recognition means, according to a currently working position specified on the window system.

7. The system of claim 4, wherein the window system provides manually controllable input means for inputting user inputs, and the message processing means sets a speech focus for the speech input independently from an input focus of the manually controllable input means.

8. The system of claim 7, wherein said plurality of application programs include a task management program for displaying the speech focus currently set by the message processing means, through the window system.

9. The system of claim 8, wherein the task management program also displays the appropriate recognition vocabulary currently set by the message processing means to the speech recognition means.

10. The system of claim 1, wherein the message processing means shifts a speech focus for the speech input to a speech focus shifting target specified by the speech input.

11. The system of claim 10, wherein the program management data specifies a local/global attribute to each recognition vocabulary registered for each application program, such that a local attributed recognition vocabulary is recognizable only when said each application program for which said each recognition vocabulary is registered is speech focused while the global attributed recognition vocabulary including a name of said each application program is always recognizable.

12. The system of claim 1, wherein the message processing means sets a speech focus to more than one of said plurality of application programs simultaneously.

13. The system of claim 1, wherein each application program exchanges the messages with the message processing means, in order to specify expected operations of the speech recognition interface system according to an internal state of said each application program.

14. The system of claim 1, further comprising data acquisition means for acquiring learning data for learning vocabulary, and dictionary production means for producing a recognition dictionary to be used by the speech recognition means according to the learning data acquired by the data acquisition means.

15. The system of claim 14, wherein the data acquisition means displays the candidates for the learning vocabulary according to the appropriate recognition vocabulary currently set by the message processing means to the speech recognition means.

16. The system of claim 14, wherein the data acquisition means displays a leaning guide display for guiding the user to utter appropriate learning vocabulary for as many times as necessary.

17. The system of claim 14, wherein the dictionary production means carries out a dictionary production processing on a background, while the speech recognition interface system is operated to carry out other processing unrelated to the dictionary production processing.

18. The system of claim 14, wherein the dictionary production means carries out a dictionary production processing according to a dictionary production management table registering dictionary production requests in a desired order in which the dictionary production processing is to be carried out by the dictionary production means.

19. The system of claim 14, wherein the dictionary production means provides an indication of at least one of a progress and a speed of a dictionary production processing.

20. The system of claim 14, wherein the data acquisition means includes:

speech data confirmation means for outputting a confirmation speech output for a learning vocabulary speech input uttered by the user, and

data validity input means for allowing the user to input a validity of the learning vocabulary speech input.

21. The system of claim 20, wherein the speech data confirmation means produces the confirmation speech output by adding known additional sounds before and after a detected speech section of the learning vocabulary speech input.

22. The system of claim 20, wherein the speech data confirmation means produces the confirmation speech output by adding click sounds at starting and ending edges of a detected speech section of the learning vocabulary speech input.

23. The system of claim 20, wherein the speech data confirmation means produces the confirmation speech output in a form of a reproduction of an entire learning vocabulary speech input followed by a reproduction of a detected speech section of the learning vocabulary speech input.

24. The system of claim 1, further comprising mode switching means for switching operation modes of the speech recognition interface system among a normal recognition mode for carrying out the speech recognition processing for the speech input by using the appropriate recognition vocabulary set by the message processing means, and an special mode in which only a specific keyword is recognizable by the speech recognition means, where the normal recognition mode is switched to the specific mode when the speech input is not given by the user for a predetermined period of time, and the specific mode is switched to the normal recognition mode when the speech input of the specific keyword is made by the user.

25. The system of claim 1, wherein said plurality of application programs include a speech mail tool.

26. The system of claim 25, wherein the speech mail tool selectively presents received mails of a category specified by the speech input made by the user.

27. The system of claim 1, wherein the speech input made by the user represents a control command for controlling operations of at least one of said plurality of application programs.

28. The system of claim 1, further comprising:

speech synthesis means for outputting speech outputs from said plurality of application programs;

wherein the message processing means also exchanges messages with said plurality of application programs in order to control the speech outputs at the speech synthesis means.

29. The system of claim 28, wherein the speech synthesis means includes:

speech output management means for managing speech output management data indicating a speech output function required by each application program;

waveform synthesis means for synthesizing speech output waveform data for each speech output of each application program; and

waveform superposition means for superposing synthesized speech output waveform data for said plurality of application programs and outputting the speech outputs given by the superposed and synthesized speech output waveform data, according to the speech output management data managed by the speech output management means.

30. The system of claim 28, wherein the speech output management data and the program management data include speech output priority data specifying required priority settings for each speech output of each application program.

31. The system of claim 30, wherein the required priority settings specified by the speech output priority data includes a presence/absence of an output interruption processing for each speech output to interrupt other speech outputs while outputting said each speech output.

32. The system of claim 28, wherein said plurality of application programs includes a speech mail tool.

33. The system of claim 32, wherein the speech input made by the user represents a control command for controlling mail production operations at the speech mail tool.

34. The system of claim 32, wherein the speech input made by the user represents a control command for controlling mail reading operations at the speech mail tool.

35. The system of claim 34, wherein the speech synthesis means outputs the speech outputs resulting from the mail reading operations, by using a controllable speech synthesized reading voice suitably controlled for each mail read by the mail reading operations.

36. The system of claim 35, wherein the controllable speech synthesized reading voice is controlled according to control codes contained within each mail read by the mail reading operations.

37. The system of claim 32, wherein the speech mail tool includes:

task priority management means for managing a task priority level assigned to each operation that can be executed in said plurality of application programs; and

mail processing means for notifying a reception of a new mail to the user only when a priority level assigned to the new mail received at the speech mail tool is higher than the task priority level assigned to a currently executed operation, according to the task priority management means.

38. The system of claim 32, wherein the speech mail tool includes mail summary means for producing a summary of each mail received at the speech mail tool, which can be read by mail reading operations at the speech mail tool instead of said each mail itself.

39. The system of claim 32, wherein the speech input is inputted by the user through a telephone, and the speech input made by the user represents a control command for controlling mail production and forwarding operations at the speech mail tool.

40. The system of claim 39, wherein the mail production and forwarding operations include at least one of a recording of a mail speech message, a reproducing of a recorded mail speech message, a specifying of a mail subject, and a specifying of a forwarding mail address.

41. The system of claim 1, wherein said plurality of application programs include a speech interface management system for interfacing the speech recognition interface system and general application programs which are not directly connected with the speech recognition interface system.

42. The system of claim 41, wherein the speech interface management system sets a dummy speech focus indicating an intended input target of the speech input to one of said plurality of application programs and the general application programs Identified by a program name obtained as the recognition result by the speech recognition interface system.

43. The system of claim 42, further comprising a window system connected with said plurality of application programs and the general application programs for providing graphical user interface functions using windows to said plurality of application programs and the general application programs, wherein the program management means also manages the dummy speech focus and recognition vocabulary attributes with respect to each window, and the message processing means specifies the appropriate recognition vocabulary for the speech input and transmits the recognition result for the speech input to appropriate ones of said plurality of application programs and the general application programs, according to the program management data, the dummy speech focus, and the recognition vocabulary attributes.

44. The system of claim 43, wherein the recognition vocabulary attributes include a local attribute, a window attribute, and a global attribute, where a local attributed recognition vocabulary is recognizable only when the window for which said local attributed recognition vocabulary is registered is dummy speech focused, a window attributed recognition vocabulary is recognizable when one of the windows assigned to an identical application program as the window for which said window attributed recognition vocabulary is registered is dummy speech focused, and a global attributed recognition vocabulary is always recognizable.

45. The system of claim 43, wherein the program management means also manages a grouping relationship among the windows, such that the recognition vocabulary registered for any one of grouped windows is recognizable whenever one of grouped windows is dummy speech focused.

46. The system of claim 43, wherein the program management means also manages a parent-child relationship among the windows, such that the recognition vocabulary registered for a child window is recognizable whenever a parent window is dummy speech focused.

47. The system of claim 42, wherein the speech interface management system comprises:

speech interface management means for managing speech interface management data indicating a message conversion function required by each general application program; and

message conversion means for converting the recognition result obtained at the speech recognition interface system into the operation command for the general application programs according to the speech interface management data.

48. The system of claim 47, wherein the speech interface management system further comprises:

operation registration means for registering the operation command to be managed in correspondence to each recognition vocabulary of each general application program in the speech interface management means.

49. The system of claim 1, further comprising dictionary editing means for editing a recognition dictionary containing recognition vocabularies of said plurality of application programs.

50. The system of claim 1, further comprising:

speech response management means for managing speech responses of said plurality of application programs, to be outputted in response to the speech input for said plurality of application programs; and

speech response registration means for registering the speech responses to be managed in correspondence to operations of said plurality of application programs resulting from an execution of the operation command specified by the speech input in the speech response management means.

51. A method of speech recognition interface for enabling a speech control of a plurality of application programs executed on a computer, comprising the steps of:

(a) managing program management data indicating a speech recognition interface function required by each application programs;

(b) specifying an appropriate recognition vocabulary to be used in a speech recognition processing of a speech input made by a user according to the program management data managed at the step (a);

(c) carrying out the speech recognition processing for the speech input by using the appropriate recognition vocabulary specified at the step (b), to obtain a recognition result; and

(d) transmitting the recognition result for the speech input obtained at the step (c) to appropriate ones of said plurality of application programs according to the program management data managed at the step (a).

52. The method of claim 51, wherein the steps (b) and (d) are realized by exchanging messages between said plurality of application programs and a speech recognition interface system handling the steps (a) and (c).

53. The method of claim 51, further comprising the step of:

(e) updating the program management data managed at the step (a) according to a state of each application programs.

54. The method of claim 51, wherein the program management data include an ID of each application program, input masks;

specifying types of the messages to be exchanged by each application program with the message processing means, a speech input flag indicating a presence/absence of a speech focus on each application program, and a recognition vocabulary list specifying the appropriate recognition vocabulary for each application program.

55. The method of claim 51, further comprising the step of:

(f) controlling speech outputs from said plurality of application programs.

56. The method of claim 51, wherein said plurality of application programs include a speech mail tool.