×

MOBILE SYSTEMS AND METHODS OF SUPPORTING NATURAL LANGUAGE HUMAN-MACHINE INTERACTIONS

  • US 20110231182A1
  • Filed: 04/11/2011
  • Published: 09/22/2011
  • Est. Priority Date: 08/29/2005
  • Status: Active Grant
First Claim
Patent Images

1. A mobile device for processing multi-modal natural language inputs, comprising:

  • a conversational voice user interface that receives a multi-modal natural language input from a user, the multi-modal natural language input including a natural language utterance and a non-speech input, the conversational voice user interface coupled to a transcription module that transcribes the non-speech input to create a non-speech-based transcription;

    a conversational speech analysis engine that identifies the user that provided the multi-modal natural language input, the conversational speech analysis engine using a speech recognition engine and a semantic knowledge-based model to create a speech-based transcription of the natural language utterance, wherein the semantic knowledge-based model includes a personalized cognitive model derived from one or more prior interactions between the identified user and the mobile device, a general cognitive model derived from one or more prior interactions between a plurality of users and the mobile device, and an environmental model derived from an environment of the identified user and the mobile device;

    a merging module that merges the speech-based transcription and the non-speech-based transcription to create a merged transcription;

    a knowledge-enhanced speech recognition engine that identifies one or more entries in a context stack matching information contained in the merged transcription and determines a most likely context for the multi-modal natural language input based on the identified entries; and

    a response generating module that identifies a domain agent associated with the most likely context for the multi-modal input, communicates a request to the identified domain agent, and generates a response to the user from content provided by the identified domain agent as a result of processing the request.

View all claims
  • 5 Assignments
Timeline View
Assignment View
    ×
    ×