System and method for querying information using a flexible multi-modal interface
First Claim
1. A method of interacting with a user on a computer device, the computer device being capable of receiving a plurality of types of user input and being capable of presenting information in a plurality of types of device output, the method comprising:
- (1) receiving a user query in one of the plurality of types of user input;
(2) presenting data to the user related to the user query;
(3) receiving a second user query associated with the presented data in one of the plurality of types of user input; and
(4) presenting a response to the user query or the second user query.
1 Assignment
0 Petitions
Accused Products
Abstract
A system and method of providing information to a user via interaction with a computer device is disclosed. The computer device is capable of receiving user input via speech, pen or multi-modally. The device receives a user query regarding a business or other entity within an area such as a city. The user query is input in speech, pen or multi-modally. The computer device responds with information associated with the request using a map on the computer device screen. The device receives further user input in speech, pen or multi-modally, and presents a response to the user query. The multi-modal input can be any combination of speech, handwriting pen input and/or gesture pen input.
176 Citations
39 Claims
-
1. A method of interacting with a user on a computer device, the computer device being capable of receiving a plurality of types of user input and being capable of presenting information in a plurality of types of device output, the method comprising:
-
(1) receiving a user query in one of the plurality of types of user input;
(2) presenting data to the user related to the user query;
(3) receiving a second user query associated with the presented data in one of the plurality of types of user input; and
(4) presenting a response to the user query or the second user query. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A method of providing information associated with a map to a user via interaction with a computer device, the computer device being capable of receiving a plurality of types of user input comprising speech, pen or multi-modally, the method comprising:
-
(1) receiving a user query in speech, pen or multi-modally;
(2) presenting data to the user related to the user query;
(3) receiving a second user query associated with the presented data in one of the plurality of types of user input; and
(4) presenting a response to the user query or the second user query. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21, 22, 23, 24)
-
-
25. A method of providing information to a user via interaction with a computer device, the computer device being capable of receiving user input via speech, pen or multi-modally, the method comprising:
-
(1) receiving a user business entity query in speech, pen or multi-modally, the user business entity query including a query related to a business location; and
(2) presenting a response to the user business entity query. - View Dependent Claims (26, 27, 28, 29, 30, 31, 32, 33)
-
-
34. A method of providing business-related information to a user on a computer device, the computer device being capable of receiving input either via speech, pen, or multi-modally, the method comprising:
-
(1) receiving a user query regarding a business either via speech, pen or multi-modally, the user query including a location component; and
(2) in response to the user query, presenting on a map display information associated with the user query. - View Dependent Claims (35, 36, 37)
-
-
38. An apparatus for interacting with a user, the apparatus storing a multi-modal recognition module using a finite-state machine to build a single meaning representation from a plurality of types of user input, the apparatus comprising:
-
(1) means for receiving a user query in one of the plurality of types of user input;
(2) means for presenting information on a map display related to the user query;
(3) means for receiving further user input in one of the plurality of types of user input; and
(4) means for presenting a response to the user query.
-
-
39. An apparatus for receiving multi-modal input from a user, the apparatus comprising:
-
a user interface module;
a speech recognition module;
a gesture recognition module;
an integrator module;
a facilitator module that communicates with the user interface module, the speech recognition module, the gesture recognition module and the integrator module, wherein the apparatus receives user input as speech through the speech recognition module, gestures through the gesture recognition module, or a combination of speech and gestures through the integrator module, processes the user input, and generates a response to the user input through the facilitator module and the user interface module.
-
Specification