Multimodal input method
First Claim
Patent Images
1. A multimodal input method comprising:
- a receiving step of receiving first input information acquired by a speech input source during a speech input period and second input information indicating at least one pointed item acquired by a pointing input source during the speech input period;
a speech recognition step of recognizing a plurality of recognition candidates based on the first input information;
a determining step of determining whether the number of unknown items included in a respective recognition candidate is equal to the number of the pointed items acquired by the pointing input source;
if it is determined in the determining step that the number unknown items included in the respective recognition candidate is not equal to the number of the pointed items acquired by the pointing input source, repeating the determining step with a different respective recognition candidate in order to obtain a matching recognition candidate which is a respective recognition candidate whose number of unknown items is equal to the number of the at least one pointed items; and
an output step of outputting a recognition result in which the at least one unknown item included in the matching recognition candidate is replaced with the at least one pointed item acquired by the pointing input source.
1 Assignment
0 Petitions
Accused Products
Abstract
In a multimodal input method, input information input from at least two input sources is received, control of the recognition of input from a second input source is performed based on the number of inputs from a first input source, and a recognition result is output.
-
Citations
7 Claims
-
1. A multimodal input method comprising:
-
a receiving step of receiving first input information acquired by a speech input source during a speech input period and second input information indicating at least one pointed item acquired by a pointing input source during the speech input period; a speech recognition step of recognizing a plurality of recognition candidates based on the first input information; a determining step of determining whether the number of unknown items included in a respective recognition candidate is equal to the number of the pointed items acquired by the pointing input source; if it is determined in the determining step that the number unknown items included in the respective recognition candidate is not equal to the number of the pointed items acquired by the pointing input source, repeating the determining step with a different respective recognition candidate in order to obtain a matching recognition candidate which is a respective recognition candidate whose number of unknown items is equal to the number of the at least one pointed items; and an output step of outputting a recognition result in which the at least one unknown item included in the matching recognition candidate is replaced with the at least one pointed item acquired by the pointing input source. - View Dependent Claims (2, 3)
-
-
4. A computer-readable medium having stored thereon a control program for causing a computer to execute a multimodal input method comprising:
-
a receiving step of receiving first input information acquired by a speech input source during a speech input period and second input information indicating at least one pointed item acquired by a pointing input source during the speech input period; a speech recognition step of recognizing a plurality of recognition candidates based on the first input information; a determining step of determining whether the number of unknown items included in a respective recognition candidate is equal to the number of the pointed items acquired by the pointing input source; if it is determined in the determining step that the number unknown items included in the respective recognition candidate is not equal to the number of the pointed items acquired by the pointing input source, repeating the determining step with a different respective recognition candidate in order to obtain a matching recognition candidate which is a respective recognition candidate whose number of unknown items is equal to the number of the at least one pointed items; and an output step of outputting a recognition result in which the at least one unknown item included in the matching recognition candidate is replaced with the at least one pointed item acquired by the pointing input source. - View Dependent Claims (5)
-
-
6. A multimodal input apparatus comprising:
-
a receiving unit configured to receive first input information acquired by a speech input source during a speech input period and second input information indicating at least one pointed item acquired by a pointing input source during the speech input period; a speech recognition unit configured to recognize a plurality of recognition candidates based on the first input information; a determining unit configured to determine whether the number of unknown items included in a respective recognition candidate is equal to the number of the pointed items acquired by the pointing input source; a control unit configured to control the determining unit to repeat the determining with a different respective recognition candidate in order to obtain a matching recognition candidate which is a respective recognition candidate whose number of unknown items is equal to the number of the at least one pointed items if it is determined that the number unknown items included in the respective recognition candidate is not equal to the number of the pointed items acquired by the pointing input source; and an output unit configured to output a result of the recognition in which the unknown item included in the matching recognition candidate obtained in the obtaining step is replaced with the pointed item acquired by the pointing input source. - View Dependent Claims (7)
-
Specification