Mobile systems and methods of supporting natural language human-machine interactions
First Claim
1. A mobile device for processing speech and non-speech communications, comprising:
- an identification module that identifies a user that communicates with the mobile device;
a matching module that matches the identified user to one or more profiles;
a transcription module that transcribes the speech and the non-speech communications to create a speech-based textual message and a non-speech-based textual message, wherein the transcription module accesses the one or more profiles;
a merging module that merges the speech-based textual message and the non-speech-based textual message to generate a query;
a search module that searches the query for text combinations;
a comparison module that compares the text combinations to entries in a context description grammar;
a plurality of domain agents that are associated with the context description grammar;
a scoring module that provides relevance scores based on results from the comparison module;
a domain agent selector that selects domain agents based on results from the scoring module; and
a response generating module that communicates with the selected domain agents to obtain content that is gathered by the selected domain agents and that generates a response from the content, wherein the content is arranged in a selected order based on results from the scoring module.
6 Assignments
0 Petitions
Accused Products
Abstract
A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.
732 Citations
49 Claims
-
1. A mobile device for processing speech and non-speech communications, comprising:
-
an identification module that identifies a user that communicates with the mobile device;
a matching module that matches the identified user to one or more profiles;
a transcription module that transcribes the speech and the non-speech communications to create a speech-based textual message and a non-speech-based textual message, wherein the transcription module accesses the one or more profiles;
a merging module that merges the speech-based textual message and the non-speech-based textual message to generate a query;
a search module that searches the query for text combinations;
a comparison module that compares the text combinations to entries in a context description grammar;
a plurality of domain agents that are associated with the context description grammar;
a scoring module that provides relevance scores based on results from the comparison module;
a domain agent selector that selects domain agents based on results from the scoring module; and
a response generating module that communicates with the selected domain agents to obtain content that is gathered by the selected domain agents and that generates a response from the content, wherein the content is arranged in a selected order based on results from the scoring module. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. The mobile device, comprising:
-
an object that is accessible by the mobile device;
an input device that receives speech and non-speech annotations corresponding to the object;
a transcription module that transcribes the speech and the non-speech annotations to create a speech-based textual message and a non-speech-based textual message; and
an association module that associates the object with the speech-based textual message and the non-speech-based textual message. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30)
-
-
31. A system, comprising:
-
a plurality of mobile devices that support multimodal communications, wherein two or more mobile devices are associated with a user; and
a context module that is coupled to the plurality of mobile devices, wherein a same context is maintained at the two or more mobile devices that are associated with the user. - View Dependent Claims (32, 33, 34, 35, 36, 37)
-
-
38. A method of processing speech and non-speech communications in a multimodal environment, comprising:
-
receiving the speech and non-speech communications;
identifying a user that provides the speech and non-speech communications;
matching the identified user to one or more profiles;
transcribing the speech and non-speech communications to create a speech-based textual message and a non-speech-based textual message, wherein the transcribing includes accessing the one or more profiles;
merging the speech-based textual message and the non-speech-based textual message to generate a query;
searching the query for text combinations;
comparing the text combinations to entries in a context description grammar;
accessing a plurality of domain agents that are associated with the context description grammar;
generating a relevance score based on results from comparing the text combinations to entries in the context description grammar;
selecting one or more domain agents based on results from the relevance score;
obtaining content that is gathered by the selected domain agents; and
generating a response from the content, wherein the content is arranged in a selected order based on results from the relevance score. - View Dependent Claims (39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49)
-
Specification