Display system capable of accepting user commands by use of voice and gesture inputs
First Claim
1. A method of accepting multimedia operation commands wherein, while pointing to either of a display object and a display position on a display screen of a graphics display system through a pointing input device, a user commands the graphics display system to cause an event on a graphics display, through a voice input device;
- comprising;
a first step of allowing the user to enter a string of coordinate points which surround one area for either of the display object or any desired display position by performing a pointing gesture along the string of coordinate points;
a second step of allowing said user to give a voice command together with said pointing gesture;
a third step of recognizing a command content of said voice command by a speech recognition process in response to said voice command;
a fourth step of recognizing a command content of said pointing gesture in accordance with a recognized result of said third step; and
a fifth step of executing the event on the graphics display in accordance with the command contents of said voice command and said pointing gesture.
4 Assignments
0 Petitions
Accused Products
Abstract
A method of accepting multimedia operation commands wherein, while pointing to either of a display object or a display position on a display screen of a graphics display system through a pointing input device, a user commands the graphics display system to cause an event on a graphics display, through a voice input device; comprising a first step of allowing the user to perform the pointing gesture so as to enter a string of coordinate points which surround one area for either of the display object and any desired display position; a second step of allowing the user to give the voice command together with the pointing gesture; a third step of recognizing a command content of the voice command by a speech recognition process in response to the voice command; a fourth step of recognizing a command content of the pointing gesture in accordance with the recognized result of the third step; and a fifth step of executing the event on the graphics display in accordance with the command contents of the voice command and the pointing gesture. Thus, the method provides a man-machine interface which utilizes the plurality of media of the voice and the pointing gesture, which offers a high operability to the user, and with which an illustration etc. can be easily edited.
-
Citations
18 Claims
-
1. A method of accepting multimedia operation commands wherein, while pointing to either of a display object and a display position on a display screen of a graphics display system through a pointing input device, a user commands the graphics display system to cause an event on a graphics display, through a voice input device;
- comprising;
a first step of allowing the user to enter a string of coordinate points which surround one area for either of the display object or any desired display position by performing a pointing gesture along the string of coordinate points; a second step of allowing said user to give a voice command together with said pointing gesture; a third step of recognizing a command content of said voice command by a speech recognition process in response to said voice command; a fourth step of recognizing a command content of said pointing gesture in accordance with a recognized result of said third step; and a fifth step of executing the event on the graphics display in accordance with the command contents of said voice command and said pointing gesture. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
- comprising;
-
12. A display system which is commanded by a user to cause an event concerning a display object on a graphics display, by the use of a voice command and a pointing gesture;
- comprising;
pointing input means for entering a string of coordinate points which surround one area for either of the display object on the graphics display and a display position of said display object; a pointing area table which stores therein said string of coordinate points entered by said pointing input means; bit map data memory means for storing therein bit map data of various display parts that constitute said display object, and standard maximum widths and standard maximum lengths of said display parts; a drawing table which stores therein identifiers of said display parts selected from within said bit map data memory means and displayed on said graphics display, widthwise and lengthwise scale-up/down ratios of said display parts relative to the standard maximum widths and lengths on said graphics display, and positional information of said display parts; a display parts dictionary which holds therein speech-recognizable names of the individual display parts stored in said bit map data memory means; voice command input means for entering the voice command of the user; speech recognition means for recognizing said voice command entered by salad voice command input means, with reference to said display parts dictionary; display parts extraction means for extracting said display parts on said graphics display as designated on the basis of said string of coordinate points in said pointing area table; target point calculation means for calculating a target point designated on the basis of said string of coordinate points in said pointing area table; scale-up/down ratio calculation means for calculating the widthwise and lengthwise scale-up/down ratio information of said display parts on the basis of said string of coordinate points in said pointing area table; and control means for selectively activating at least one of said display parts extraction means, said target point calculation means and said scale-up/down ratio calculation means in accordance with a result of the speech recognition, and for rewriting said drawing table on the basis of a result of the activating. - View Dependent Claims (13, 14, 15, 16, 17)
- comprising;
-
18. A display system which is commanded by a user to cause an event concerning a display object on a graphics display, by the use of a voice command and a pointing gesture;
- comprising;
pointing input means for entering a string of coordinate points which surround an area on the graphics display; voice command input means for entering the voice command of the user; speech recognition means for recognizing said voice command entered by said voice command input means; execution means for executing the event on the graphics display in accordance with the string of coordinate points entered by the pointing input means and the voice command entered by the voice command input means; and wherein a plurality of display objects share common areas with said area represented by the string of coordinate points, one of the display objects which has the largest common area with the area represented by the string of coordinate points is selected as the display object.
- comprising;
Specification