Speech recognition apparatus and method and a computer usable medium for selecting an application in accordance with the viewpoint of a user

• US 6,076,061 A
• Filed: 09/08/1995
• Issued: 06/13/2000
• Est. Priority Date: 09/14/1994
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kawasaki, Katsuhiko, Komori, Yasuhiro, Ohora, Yasunori
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Chawan, Vijay B.
• Application Number:
  US08/524,949
• Time in Patent Office: 1,740 Days
• Field of Search: 395/2.79, 395/2.84, 395/2.81
• US Class Current: 704/270
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A viewpoint of a user is detected in a viewpoint detecting process, and how long the detected viewpoint has stayed in an area is determined. The obtained viewpoint and its trace is displayed on a display unit. In a recognition information controlling process, the relationship between the viewpoint (in an area) and/or its movement, and recognition information (words, sentences, grammar, etc.) is obtained as weight P(). When the user pronounces a word (or sentence), the speech is inputted and A/D converted via a speech input unit. Next, in a speech recognition process, a speech recognition probability PS() is obtained. Finally, speech recognition is performed on the basis of a product of the weight P() and the speech recognition probability PS(). Accordingly, classes of the recognition information are controlled in accordance with the movement of the user'"'"'s viewpoint, thereby improving the speech recognition probability and the speed of recognition.

Claims

1. A speech recognition apparatus comprising:

detecting means for detecting a viewpoint of a user;

selecting means for selecting both an application from a plurality of applications and a recognition process from a plurality of recognition processes in accordance with the viewpoint detected by said detecting means at the time speech is pronounced; and

recognizing means for recognizing an input command in pronounced speech to be subjected to the application selected by said selecting means, by executing the speech recognition corresponding to the recognition process selected by said selecting means, for the pronounced speech.

2. The speech recognition apparatus according to claim 1, wherein said changing means changes classes of recognition information to be used for the speech recognition in accordance with the viewpoint detected by said detecting means.

3. The speech recognition apparatus according to claim 2, wherein said changing means sets a plurality of areas each of which is related to a different class of the recognition information, selects one of the plurality of areas in accordance with the viewpoint detected by said detecting means, and determines a class of the recognition information, related to the selected area, to be used in the speech recognition.

4. The speech recognition apparatus according to claim 3, wherein one of the plurality of areas is set for each respective image, shown on a display screen, representing an application and has a predetermined size in which the image is included.

5. The speech recognition apparatus according to claim 3, wherein said changing means detects a position where the viewpoint stays on the basis of the viewpoint detected by said detecting means and selects one of the plurality of areas which includes said position.

6. The speech recognition apparatus according to claim 1, wherein said changing means controls weight for each class of the recognition information to be used in the speech recognition in accordance with the viewpoint detected by said detecting means.

7. The speech recognition apparatus according to claim 6, wherein said changing means sets a plurality of areas each of which is related to a different class of recognition information and controls the weight for each class of the recognition information in accordance with relationship between the viewpoint detected by said detecting means and each of the plurality of areas.

8. The speech recognition apparatus according to claim 7, wherein one of the plurality of areas is set for each respective image, shown on a display screen, representing an application and has a predetermined size in which the image is included.

9. The speech recognition apparatus according to claim 7, wherein said changing means counts a number of times a viewpoint has stayed in the plurality of areas during a predetermined time period on the basis of the viewpoint detected by said detecting means, and controls weight for each class of the recognition information to be used in the speech recognition in accordance with the counted number.

10. The speech recognition apparatus according to claim 7, wherein said changing means detects a position where the viewpoint stays on the basis of the viewpoint detected by said detecting means, and controls the weight for each class of the recognition information in accordance with relationship between the position and each of the plurality of areas.

11. The speech recognition apparatus according to claim 7, wherein said changing means assigns a maximum value to weight for a class of the recognition information related to one of the plurality of areas in which the viewpoint stays, and assigns a minimum value to weight for other classes of the recognition information.

12. The speech recognition apparatus according to claim 11, wherein, in a case where the viewpoint exits from the area where it has stayed, said changing means changes the weight for the class of the recognition information related to the area in which the viewpoint had stayed to the minimum value after a predetermined time period has elapsed.

13. The speech recognition apparatus according to claim 11, wherein, in a case where the viewpoint exits from the area where it has stayed, said changing means gradually decrease the weight for the class of the recognition information related to the area in which the viewpoint had stayed toward the minimum value as time elapses.

14. The speech recognition apparatus according to claim 13, wherein rate of weight changes differ from each other depending upon different classes of the recognition information.

15. The speech recognition apparatus according to claim 11, wherein, in a case where the viewpoint enters one of the plurality of areas, said changing means gradually increases the weight for a class of the recognition information related to the area which the viewpoint entered toward the maximum value as time elapses.

16. The speech recognition apparatus according to claim 15, wherein rate of weight changes differ from each other depending upon classes of the recognition information each of which is related to one of the plurality of areas.

17. The speech recognition apparatus according to claim 7, wherein each of the plurality of areas is divided into a plurality of fields, and said changing means controls weight for each class of the recognition information in accordance with the field in which the viewpoint detected by said detecting means is.

18. The speech recognition apparatus according to claim 1, wherein said changing means changes recognition processes on the basis of a viewpoint detected by said detecting means just before the speech recognition is performed by said recognizing means.

19. The speech recognition apparatus according to claim 1, wherein said selecting means selects an application for performing an input command based on the viewpoint detected by said detecting means and selects recognition processes for speech recognition in accordance with the selected application.

20. The speech recognition apparatus according to claim 1, wherein said selecting means sets a plurality of areas each of which is related to a different application, selects one of the plurality of areas in accordance with the viewpoint detected by said detecting means, and selects an application related to the selected area.

21. A speech recognition method comprising:

a detecting step of detecting a viewpoint of a user;

a selecting step of selecting both an application from a plurality of applications and a recognition process from a plurality of recognition processes in accordance with the viewpoint detected in said detecting step; and

a recognizing step for recognizing an input command to be subjected to the application selected by said selecting step, by executing the speech recognition corresponding to the recognition process selected in said selecting step.

22. The speech recognition method according to claim 21, wherein, in said selecting step, classes of recognition information to be used for the speech recognition are selected in accordance with the viewpoint detected in said detecting step.

23. The speech recognition method according to claim 22, wherein, in said selecting step, a plurality of areas each of which is related to a different application and a different class of the recognition information are set, and one of the plurality of areas is selected in accordance with the viewpoint detected in said detecting step, and a class of the recognition information related to the selected area is selected.

24. The speech recognition method according to claim 21, wherein one of the plurality of areas is set for each respective image, shown on a display screen, representing an application and has a predetermined size in which the image is included.

25. The speech recognition method according to claim 23, wherein, in said selecting step, a position where the viewpoint stays is detected on the basis of the viewpoint detected in said detecting step and one of the plurality of areas which includes said position is selected.

26. The speech recognition method according to claim 21, further comprising the step of controlling the weight for each class of recognition information to be used in the recognition process is controlled in accordance with the viewpoint detected in said detecting step and executing the speech recognition using classes of the recognition,

wherein said selecting step selects a class of the recognition information based on the weight and recognition results of said executing step and selects an application corresponding to the selected recognition information, and wherein said recognizing step determines the recognition result of the speech recognition corresponding to the selected class of the recognition information as an input command to be subjected to the selected application.

27. The speech recognition method according to claim 26, wherein, in said selecting step, a plurality of areas each of which is related to a different class of recognition information are set and the weight for each class of the recognition information is controlled in accordance with relationship between the viewpoint detected in said detecting step and each of the plurality of areas.

28. The speech recognition method according to claim 25, wherein one of the plurality of areas is set for each respective image, shown on a display screen, representing an application and has a predetermined size in which the image is included.

29. The speech recognition method according to claim 27, wherein, in said selecting step, a number of times a viewpoint has stayed in the plurality of areas is counted during a predetermined time period on the basis of the viewpoint detected in said detecting step, and the weight for each class of the recognition information to be used in the speech recognition is controlled in accordance with the counted number.

30. The speech recognition method according to claim 27, wherein, in said selecting step, a position where the viewpoint stays is detected on the basis of the viewpoint detected in said detecting step, and the weight for each class of the recognition information is controlled in accordance with relationship between the position and each of the plurality of areas.

31. The speech recognition method according to claim 27, wherein, in said selecting step, a maximum value is assigned to the weight for a class of the recognition information related to one of the plurality of areas in which the viewpoint stays, and a minimum value is assigned to the weight for other classes of the recognition information.

32. The speech recognition method according to claim 31, wherein, in a case where the viewpoint exits from the area where it has stayed, the weight for the class of the recognition information related to the area in which the viewpoint had stayed is changed to the minimum value, after a predetermined time period has elapsed, in said selecting step.

33. The speech recognition method according to claim 31, wherein, in a case where the viewpoint exits from the area where it has stayed, the weight for the class of the recognition information related to the area in which the viewpoint had stayed is gradually decreased, toward the minimum value as time elapses, in said selecting step.

34. The speech recognition method according to claim 31, wherein rate of weight changes differ from each other depending upon different classes of the recognition information.

35. The speech recognition method according to claim 31, wherein, in a case where the viewpoint enters one of the plurality of areas, the weight for a class of the recognition information related to the area which the viewpoint entered is gradually increased, toward the maximum value as time elapses, in said selecting step.

36. The speech recognition method according to claim 33, wherein rate of weight changes differ from each other depending upon classes of the recognition information each of which is related to one of the plurality of areas.

37. The speech recognition method according to claim 27, wherein each of the plurality of areas is divided into a plurality of fields, and, in said selecting step, the weight for each class of the recognition information is controlled in accordance with the field to which the viewpoint detected in said detecting step is directed.

38. The speech recognition method according to claim 21, wherein, in said selecting step, recognition processes on the basis of a viewpoint detected in said detecting step are selected just before the speech recognition is performed in said recognizing step.

39. The speech recognition method according to claim 21, wherein said selecting step selects an application for performing an input command based on the viewpoint detected by said detecting step an selects recognition processes for speech recognition in accordance with the selected application.

40. The speech recognition method according to claim 21, wherein said selecting step sets a plurality of areas each of which is related to a different application, selects one of the plurality of areas in accordance with the viewpoint detected by said detecting step, and selects an application related to the selected area.

44. The information processing apparatus according to claim 37, further comprising display means for displaying images representing each of a plurality of applications,

wherein said changing means sets a plurality of areas each of which includes each of the plurality of images and changes recognition processes in accordance with the relationship between the viewpoint detected by said detecting means and each of the plurality of areas.

41. An information processing apparatus comprising:

detecting means for detecting a viewpoint of a user;

selecting means for selecting both an application from a plurality of applications and a recognition process from a plurality of recognition processes in accordance with the viewpoint detected by said detecting means at the time speech is pronounced;

recognizing means for recognizing an input command, in the pronounced speech, to be subjected to the application selected by said selecting means, by executing the speech recognition corresponding to the recognition process selected by said selecting means, for the pronounced speech; and

executing means for executing the application to perform the input command recognized by said recognizing means.

42. The speech recognition apparatus according to claim 41, wherein said selecting means selects a class of recognition information to be used for the speech recognition in accordance with the viewpoint detected by said detecting means, and selects an application in accordance with the selected class of the recognition information.

43. The speech recognition apparatus according to claim 41, further comprising control means for controlling the weight for each class of the recognition information to be used in the speech recognition in accordance with the viewpoint detected by said detecting means, and execution means for executing the speech recognition using classes of the recognition,

wherein said selecting means selects a class of the recognition information based on the weight and recognition results of said execution means, and selects an application corresponding to the selected recognition information, and said recognizing means determines the recognition result of the speech recognition corresponding to the selected class of the recognition information as an input command to be subjected to the selected application.

45. A computer program product comprising a computer usable medium having computer readable program code means embodied in said medium for speech recognition processing, said product including:

first computer readable program code means for causing a computer to detect a viewpoint of a user;

second computer readable program code means for causing a computer to select both an application from a plurality of applications and a recognition process from a plurality of recognition processes in accordance with the viewpoint, at the time speech is pronounced, caused to be detected by said first computer readable program code means; and

third computer readable program code means for causing a computer to recognize an input command, in the pronounced speech, to be subjected to the application caused to be selected by said second computer readable program code means, by causing the computer to execute the speech recognition corresponding to the recognition process, for the pronounced speech, caused to be selected by said second computer readable program code means.

46. A computer program product according to claim 45, wherein said second computer readable program code means causes the computer to select a class of recognition information to be used for the speech recognition in accordance with the viewpoint caused to be detected by said first computer readable program code means.

47. A computer program product according to claim 45, further comprising fourth computer readable program code means for causing the computer to control the weight of each class of the recognition information to be used in the speech recognition in accordance with the viewpoint caused to be detected by said first computer readable program code means and causing the computer to execute the speech recognition using classes of the recognition,

wherein said second computer readable program code means causes the computer to select a class of the recognition information based on the controlled weights and recognition results caused to be obtained by said fourth computer readable program code means, and selects an application corresponding to the selected recognition information, and said second computer readable program code means causes the computer to determine the recognition result of the speech recognition corresponding to the selected class of the recognition information as an input command to be subjected to the selected application.

48. A computer program product according to claim 46 further comprising fourth computer readable program code means for causing the computer to select one of the plurality of applications in accordance with a class of the recognition information to be used in the recognizing operation and to transmit a recognized result in said recognition process to the selected application.