Control apparatus

• US 20030120494A1
• Filed: 12/18/2002
• Published: 06/26/2003
• Est. Priority Date: 12/20/2001
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Shao, Yuan, Jost, Uwe Helmut
• Granted Patent: US 7,212,971 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A control apparatus is provided for enabling a user to communicate by speech with processor-controlled apparatus (1) having a display (44) for displaying screens to a user, an audio input (42) for receiving speech data from the user and a pointing device (40) for providing data for enabling the screen location at which the user'"'"'s attention is directed to be determined. The control apparatus has a screen data provider (100) providing screen data for a display screen, the screen data including access data, location data defining a location within the screen associated with the access data and speech input data for assisting a user in formulating a spoken command or commands to initiate use of the access data; a screen data displayer (100) for causing the display (44) to display a screen representing the screen data to the user; a location determiner (100) for determining from position data provided by the position data provider the screen location at which the user'"'"'s attention is directed; and

a prompter (100) for, when the determined location corresponds to the location defined by the location data, using the speech input data to provide the user with a prompt to assist the user in formulating a spoken command that initiates use of the access data.

Claims

1. Control apparatus for enabling a user to communicate by speech with processor-controlled apparatus having a display for displaying screens to a user, an audio input means for receiving speech data from the user and a position data providing means for providing data for enabling the screen location at which the user'"'"'s attention is directed to be determined, the control apparatus comprising:

screen data providing means providing screen data for a display screen, the screen data including access data, location data defining a location within the screen associated with the access data and speech input data for enabling a user to formulate a spoken command or commands to initiate use of the access data;

screen data display control means for causing the display to display a screen representing the screen data to the user;

location determining means for determining from position data provided by the position data providing means the screen location at which the user'"'"'s attention is directed; and

prompt means for, when the determined location corresponds to the location defined by the location data, using the speech input data to provide the user with a prompt to assist the user in formulating a spoken command that initiates use of the access data.

2. Control apparatus according to claim 1, wherein the screen data providing means provides as the speech input data data representing at least one of:

hints for formulating a speech command and a set of available speech commands.

3. Control apparatus according to claim 1, wherein the prompt means is operable to cause the display to display the prompt to the user.

4. Control apparatus according to claim 1, wherein the prompt means is operable to cause the display to display the prompt to the user as one of a drop down menu and a list.

5. Control apparatus according to claim 1, wherein the prompt means is operable to provide the prompt as an audio output where the processor-controlled apparatus has speech producing means.

6. Control apparatus according to claim 1, wherein the screen data includes indicator data defining an indicator to be displayed on the screen at the screen location associated with the access data and the screen data display control means is operable to cause the indicator to be displayed at the screen location associated with the access data.

7. Control apparatus according to claim 6, wherein the indicator data defines an indicator that indicates that speech command input is available to initiate use of the access data.

8. Control apparatus according to claim 7, wherein the indicator data defines the indicator as one or more wavy lines.

9. Control apparatus according to claim 1, further comprising cursor control means for causing the display to display on the screen a cursor representing the screen location at which the user'"'"'s attention is directed, the cursor control means being operable to cause the cursor to be changed when the cursor is at the screen location associated with the access data to indicate to the user the presence of the access data.

10. Control apparatus according to claim 9, wherein the cursor control means is operable to cause the cursor to be changed to one of a default cursor for identifying a screen location associated with access data and a cursor specified by the screen data, such as a representation of a mouth when the cursor is at the screen location associated with the access data.

11. Control apparatus according to claim 1, wherein the location determining means is operable to determine the screen location at which the user'"'"'s attention is directed by obtaining from image data provided by a camera forming the position data providing means data representing the direction of the user'"'"'s gaze.

12. A user interface having a display for displaying screens to a user, an audio input means for receiving speech data from the user, a position data providing means for providing data for enabling the screen location at which the user'"'"'s attention is directed to be determined and control apparatus in accordance with claim 1.

13. A method of enabling a user to communicate by speech with processor-controlled apparatus having a display for displaying screens to a user, an audio input means for receiving speech data from the user and a position data providing means for providing data for enabling the screen location at which the user'"'"'s attention is directed to be determined, the method comprising processor means carrying out the steps of:

providing screen data for a display screen, the screen data including access data, location data defining a location within the screen associated with the access data and speech input data for enabling a user to formulate a spoken command or commands;

causing the display to display a screen representing the screen data to the user;

determining from position data provided by the position data providing means the screen location at which the user'"'"'s attention is directed; and

when the determined location corresponds to the location defined by the location data, using the speech input data to provide the user with a prompt to assist the user in formulating a spoken command that initiates use of the access data.

14. A method according to claim 13, wherein the screen data providing step provides as the speech input data data representing at least one of:

hints for formulating a speech command and a set of available speech commands.

15. A method according to claim 13, wherein the step of providing the prompt to display the prompt to the user.

16. A method according to claim 13, wherein the step of providing the prompt displays the prompt to the user as one of a drop down menu and a list.

17. A method according to claim 13, wherein the step of providing the prompt provides the prompt as an audio output where the processor-controlled apparatus has speech producing means.

18. A method according to claim 13, wherein the screen data includes indicator data defining an indicator to be displayed on the screen at the screen location associated with the access data and the step of causing the display to display the screen data causes the indicator to be displayed at the screen location associated with the access data.

19. A method according to claim 18, wherein the indicator data defines an indicator that indicates that speech command input is available to initiate use of the access data.

20. A method according to claim 19, wherein the indicator data defines the indicator as one or more wavy lines.

21. A method according to claim 13, further comprising the processor means carrying out the steps of causing the display to display on the screen a cursor representing the screen location at which the user'"'"'s attention is directed and to cause the displayed cursor to change when the cursor is at the screen location associated with the access data to indicate to the user the presence of the access data.

22. A method according to claim 21, wherein the processor means causes the cursor to change to a default cursor for identifying the presence of the access data or a cursor specified by the screen data, such as a representation of a mouth when the cursor is at the screen location associated with the access data.

23. A method according to claim 13, wherein the processor means determines the screen location at which the user'"'"'s attention is directed by obtaining from image data provided by a camera forming the position data providing means data representing the direction of the user'"'"'s gaze.

24. A signal comprising processor implementable instructions and/or data for causing processor means to carry out a method in accordance with claim 13.

25. A storage medium comprising processor implementable instructions and/or data for causing processor means to carry out a method in accordance with claim 13.

26. Control apparatus for enabling a user to communicate by speech with processor-controlled apparatus having a display for displaying screens to a user and an audio input means for receiving speech data from the user, the control apparatus comprising:

screen data display control means for causing the display to display a screen representing screen data to the user; and

prompt means for using speech data input by the user to provide the user with a prompt to assist the user in formulating a spoken command that initiates use of access data.

27. A method of enabling a user to communicate by speech with processor-controlled apparatus having a display for displaying screens to a user and an audio input means for receiving speech data from the user, the method comprising processor means carrying out the steps of:

causing the display to display a screen representing screen data to the user; and

using speech data input by the user to provide the user with a prompt to assist the user in formulating a spoken command that initiates use of access data.

28. Program instructions for programming a processor to enable a user to communicate by speech with processor-controlled apparatus having a display for displaying screens to a user and an audio input means for receiving speech data from the user, the program instructions comprising:

screen data display control code for causing the display to display a screen representing screen data to the user; and

prompt code for using speech data input by the user to provide the user with a prompt to assist the user in formulating a spoken command that initiates use of access data.