Erroneous input processing method and apparatus in information processing system using composite input

• US 5,864,808 A
• Filed: 09/22/1997
• Issued: 01/26/1999
• Est. Priority Date: 04/25/1994
• Status: Expired due to Term

• Current Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Original Assignee: Hitachi, Ltd.
• Inventors: Kikuchi, Hideaki, Oheda, Shigeto, Ando, Haru, Hasegawa, Tsukasa, Hataoka, Nobuo, Matsuda, Yasumasa
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Chawan, Vijay B.
• Application Number:
  US08/935,299
• Time in Patent Office: 491 Days
• Field of Search: 704/275, 704/276, 704/251, 704/231, 704/211
• US Class Current: 704/251
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A user inputs voice through a voice recognition program, a microphone and an A/D converter while pointing by use of a pointing gesture, touch pen or the like with reference to an image displayed on a display unit. For the result of recognition of the inputted voice, a processing or display indicated by a candidate having the first rank of reliability of recognition is performed and an indication showing a plurality of candidates having the second rank and the lower ranks than that is displayed in a menu form on a display screen. In the case where there is an error (that is, in the case where the processing or display indicated by the candidate having the first rank is not a processing intended by the user or the user makes an erroneous input), the error is corrected in such a manner that a correct input candidate is selected by a finger, pen or the like from the displayed menu of candidates having the second rank and the lower ranks than that and a processing operation or display associated with the selected candidate is performed again.

Claims

1. An information processing system using voice comprising:

at least information displaying means for displaying information;

position information inputting means for inputting continuous position information by a user through a pointing gesture;

voice information inputting means for inputting speech information;

storing means for storing the position information inputted by said position information inputting means and the voice information inputted by said voice information inputting means;

standard pattern information storing means for storing at least one of a voice standard pattern, word information, and grammar information;

voice information analyzing means for determining the reliability of voice inputted by said voice information inputting means by use of the at least one of voice standard pattern, word information, and grammar information; and

error processing means by which in the case where the information processing system first performs a processing or display determined by a candidate having a first rank of reliability of recognition as the result of recognition of speech while a menu of plural candidates having a second rank of reliability of recognition and lower ranks than the second rank as the result of recognition of speech is displayed on a display screen, and by which if the processing or display determined by the candidate having the first rank of reliability is erroneous or the user makes an erroneous input, a correct input candidate is selected by a pointing device from the displayed menu of candidates having the second rank of reliability of recognition and the lower ranks than the second rank so that the information processing system again performs a processing operation or display associated with the selected candidate.

2. An information processing system using voice according to claim 1, wherein said error processing means stores pointing information having already been inputted by the pointing device in order that it is not required that information other than a voice input and inclusive of pointing information be inputted again at the time of correction in the case where the candidate of recognition of speech is selected by the pointing device.

3. An information processing system using voice according to claim 1, wherein in the case where there is no correct candidate in the displayed menu, said error processing means makes it possible to make correction by inputting only necessary information by voice again.

4. An information processing system using voice according to claim 1, wherein in a case where a voice input is made again, the candidate having the first rank of reliability of recognition as the result of recognition of voice and the menu-displayed candidates having the second rank of reliability of recognition and the lower ranks than the second rank are eliminated from being becoming an object of recognition.

5. An information processing system using voice according to claim 1, wherein the system has a function of performing the processing or display determined by the candidate having the first rank of reliability of recognition and displaying the content of the result of recognition itself on the display screen or outputting the content of the result of recognition by speech.

6. An information processing system using voice in an image editing system which includes the information processing system using voice according to claim 1 and in which an editing operation is performed by inputting an editing command through voice by a user and indicating an object or a moving position, by a pointing device, the system being provided with error processing means by which in the case where the input of information is followed by performing an operation based on a command which is a candidate having the first rank of reliability of recognition of speech while displaying a menu of candidates having the second rank of reliability of recognition of speech and the lower ranks than the second rank and the operation based on the candidate having the first rank of reliability of recognition of speech is erroneous or the user makes an erroneous input, information other than voice and inclusive of pointing information having already been inputted is stored so that a selection to the menu-displayed plural candidates having the second rank of reliability of recognition and the lower ranks than the second rank is only made by the pointing device to perform a processing operation or display associated with the selection again.

7. An erroneous input correcting method in a composite input information processing system, comprising the steps of:

inputting an editing instruction indicative of the modification in shape or change in position of a displayed object by voice while designating the displayed object directly;

storing position information of the object;

recognizing the inputted speech and performing an editing instruction having a first rank of reliability of recognition based on the recognized inputted speech;

displaying the result of performance and a menu of editing instructions having a second rank of reliability of recognition as the result of recognition of speech and lower ranks than the second rank;

selecting a processing instruction of the menu of editing instructions having the second rank of reliability of recognition and the lower ranks than the second rank; and

executing said processing instruction for said position information.

8. An information processing system using voice comprising:

information displaying means for displaying information;

position information inputting means for inputting continuous position information by a user through a pointing gesture;

voice information inputting means for inputting voice information representing a desired processing or a desired display;

storing means for storing the position information inputted by the position information inputting means and the voice information inputted by the voice information inputting means;

standard pattern information storing means for storing at least one of a voice standard pattern, word information, and grammar information;

voice information analyzing means for determining a reliability indicating a likelihood of representing the voice information inputted by the voice information inputting means by use of the at least one of a voice standard pattern, word information, and grammar information; and

error processing means which

(1) executes first a processing or a display indicated by a candidate having a first rank of reliability as a result of speech recognition of the voice information while displaying a menu of candidates having a second rank and lower ranks of reliability on a display screen of the information displaying means, and

(2) when the processing or the display indicated by the candidate having the first rank of reliability is erroneous, or when the user makes an erroneous input, selects a correct candidate by a human finger or a writing pen from the menu of candidates having the second rank and lower ranks of reliability, and executes a processing or a display indicated by the selected correct candidate.

9. An information processing system using voice according to claim 8, wherein the error processing means stores information already inputted by a pointing gesture of the user other than information inputted by voice so that when the processing indicated by the candidate having the first rank of reliability is erroneous, it is only necessary to select the correct candidate from the displayed menu and execute the processing indicated by the selected candidate without inputting again the information inputted by the pointing gesture of the user.

10. An information processing system using voice according to claim 9, wherein when there is no correct candidate in the displayed menu, the error processing means makes it possible to correct an erroneous processing executed due to the indication by the candidate having the first rank of reliability by inputting only necessary information by voice again to obtain a new candidate as a result of the speech recognition of the inputted voice information.

11. An information processing system using voice according to claim 10, wherein when the voice information is inputted again, the candidate having the first rank of reliability and the candidates having the second rank and lower ranks of reliability in the displayed menu are eliminated from candidates obtained as a result of the speech recognition of the voice information inputted again.

12. An information processing system using voice according to claim 8, further including means for executing the processing or the display indicated by the candidate having the first rank of reliability, and at the same time, displaying the content of the result of the speech recognition itself on the display screen and outputting the content of the result of the speech recognition by voice.

13. An information processing system using voice in an editing system, the information processing system using voice comprising:

information displaying means for displaying information;

editing information inputting means for inputting editing information indicating an editing object and a moving position by a human finger or a writing pen;

voice information inputting means for inputting voice information representing editing commands including at least "move" and "copy";

storing means for storing the editing information inputted by the editing information inputting means and the voice information inputted by the voice information inputting means;

standard pattern information storing means for storing at least one of a voice standard pattern, word information, and grammar information;

voice information analyzing means for determining a reliability indicating a likelihood of representing the voice information inputted by the voice information inputting means by use of the at least one of a voice standard pattern, word information, and grammar information; and

error processing means which

(1) performs an editing operation, when voice information and editing information are inputted, based on an editing command which is a first candidate of speech recognition, and displays a menu of candidates which are second and following candidates of the speech recognition on a display screen of the information displaying means, and

(2) when the editing operation performed based on the editing command which is the first candidate of the speech recognition is erroneous, or when the user makes an erroneous input, stores information already inputted, including the editing information and excluding the voice information, selects an editing command from the menu of the second and following candidates of the speech recognition with the human finger or the writing pen, and executes an editing operation based on the selected editing command.

14. An erroneous input correcting method in a composite input information processing system, the method comprising the steps of:

inputting by voice an editing instruction indicative of a modification in shape or a change in position of a displayed object while designating the displayed object directly;

storing position information of the object;

performing speech recognition of the editing instruction inputted by voice, and executing an editing instruction having a first rank of reliability of recognition which is an editing instruction which is most likely to be representative of the inputted editing instruction;

displaying (1) a result of executing the editing instruction having the first rank of reliability of recognition, and (2) a menu of editing instructions having a second rank and lower ranks of reliability of recognition as a result of recognition of the editing instruction inputted by voice;

selecting an editing instruction from the menu of editing instructions having the second rank and lower ranks of reliability of recognition; and

executing the editing instruction selected from the menu for the position information of the object.

15. An information processing system comprising:

speech recognition processing means for recognizing an editing instruction inputted by voice, and determining a plurality of candidates of recognition;

processing means for executing, with respect to an editing object designated by pointing, an editing instruction which is a candidate of recognition having a first rank of reliability among the candidates of recognition; and

display means for displaying (1) a processing result of the editing instruction which is the candidate of recognition having the first rank of reliability, and (2) candidates of recognition having a second rank and lower ranks of reliability;

wherein when the processing result of the editing instruction which is the candidate of recognition having the first rank of reliability is erroneous, or when the user makes an erroneous input, the processing means executes, with respect to the editing object, an editing instruction selected from the candidates of recognition having the second rank and lower ranks of reliability displayed by the display means.

16. An information processing system according to claim 15, wherein the processing means stores information of the editing object until the processing result of the executed editing instruction is confirmed to be correct.

17. An information processing system according to claim 16, wherein when the editing instruction is again inputted by voice, the speech recognition processing means excludes the candidate of recognition having the first rank of reliability and the candidates of recognition having the second rank and lower ranks of reliability from candidates of recognition of the editing instruction inputted again, and outputs new candidates of recognition.

18. An information processing method using a speech recognition processing, the method comprising the steps of:

(a) inputting by voice an instruction sentence containing an editing instruction, an editing object, and position information required for editing;

(b) specifying an arbitrary object and an arbitrary position on a display;

(c) performing speech recognition of the instruction sentence, and determining a plurality of candidates of recognition of the editing instruction;

(d) using the object and the position specified in step (b) as the editing object and the position information indicated in step (a), and executing an editing instruction which is a candidate of recognition having a first rank of reliability among the candidates of recognition;

(e) displaying on the display (1) a result of executing the editing instruction which is a candidate of recognition having the first rank of reliability, and (2) the candidates of recognition;

(f) when the result of executing the editing instruction which is the candidate of recognition having the first rank of reliability is erroneous, selecting an editing instruction from the candidates of recognition displayed on the display other than the candidate of recognition having the first rank of reliability; and

(g) executing again the instruction sentence inputted in step (a) based on the editing instruction selected in step (f) and the object and the position specified in step (b).

19. An information processing method according to claim 18, wherein in step (f), when the result of executing the editing instruction which is the candidate of recognition having the first rank of reliability is erroneous and there is no desired candidate in the candidates of recognition displayed on the display, a new editing instruction is inputted by voice, and the instruction sentence inputted in step (a) is executed again based on a result of speech recognition of the new editing instruction inputted by voice and the object and the position specified in step (b).

20. An information processing method according to claim 19, wherein at a time of performing speech recognition of the new editing instruction inputted by voice, the candidates of recognition determined in step (c) are excluded from candidates of recognition of the new editing instruction inputted by voice.

21. An information processing system using voice comprising:

a display;

a pointing device; and

a control device which

recognizes a plurality of candidates of recognition from inputted voice information representing a desired processing by performing voice recognition on the inputted voice information, the candidates of recognition including a first candidate of recognition and other candidates of recognition,

executes a processing indicated by the first candidate of recognition while displaying the other candidates of recognition in a menu, and

if the processing indicated by the first candidate of recognition is erroneous, executes a correct processing indicated by a correct candidate of recognition selected from the other candidates of recognition displayed in the menu.

22. An information processing system according to claim 21, wherein the first candidate of recognition is a candidate of recognition having a first rank of reliability of recognition of the inputted voice information;

and

wherein the other candidates of recognition are candidates of recognition having second and lower ranks of reliability of recognition of the inputted voice information.