Method and system for providing information services related to multimodal inputs
First Claim
Patent Images
1. A method comprising:
- receiving at a system server a first input comprising visual information from a client device, wherein the visual information comprises a plurality of character sequence groups;
receiving at the system server a second input comprising audio information from the client device, wherein the audio information comprises vocals;
extracting the plurality of character sequence groups from the visual information using an optical character recognition engine;
converting the vocals to text using a speech recognition engine;
generating a plurality of contexts wherein a first context comprises a first character sequence group from the plurality of character sequence groups and a first portion of the textidentifying a first context from the plurality of contexts based on the first input;
identifying a second context from the plurality of contexts based on the second input;
querying a database using the first and second contexts to generate a first list comprising at least one information service;
identifying a third context based on the first list of at least one information service;
querying the database using the third context to generate a second list of information services;
displaying the second list of information services on a screen of the client device;
mapping the third context to a first information service in the list of information services, wherein the mapping the third context further comprises using rules stored at the system server;
identifying the first information service as a first type or a second type;
if a user accesses the first information service identified as a first type, debiting an account of the user; and
if the user accesses the first information service identified as a second type, debiting an account of a provider of the information service.
2 Assignments
0 Petitions
Accused Products
Abstract
A system and method provides information services related to multimodal inputs. Several different types of data used as multimodal inputs are described. Also described are various methods involving the generation of contexts using multimodal inputs, synthesizing context-information service mappings and identifying and providing information services.
54 Citations
20 Claims
-
1. A method comprising:
-
receiving at a system server a first input comprising visual information from a client device, wherein the visual information comprises a plurality of character sequence groups; receiving at the system server a second input comprising audio information from the client device, wherein the audio information comprises vocals; extracting the plurality of character sequence groups from the visual information using an optical character recognition engine; converting the vocals to text using a speech recognition engine; generating a plurality of contexts wherein a first context comprises a first character sequence group from the plurality of character sequence groups and a first portion of the text identifying a first context from the plurality of contexts based on the first input; identifying a second context from the plurality of contexts based on the second input; querying a database using the first and second contexts to generate a first list comprising at least one information service; identifying a third context based on the first list of at least one information service; querying the database using the third context to generate a second list of information services; displaying the second list of information services on a screen of the client device; mapping the third context to a first information service in the list of information services, wherein the mapping the third context further comprises using rules stored at the system server; identifying the first information service as a first type or a second type; if a user accesses the first information service identified as a first type, debiting an account of the user; and if the user accesses the first information service identified as a second type, debiting an account of a provider of the information service. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification