System and method for multi-modal context-sensitive applications in home network environment

• US 7,584,280 B2
• Filed: 11/12/2004
• Issued: 09/01/2009
• Est. Priority Date: 11/14/2003
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kwak, Ji Young, Kim, Doo Hyun, Lee, Eun Ryung, Vinod, Joseph C., Ahn, Sung Ho, Kim, Ji Yong, Sul, Dong Myung
• Primary Examiner(s): Hu; Jinsong
• Application Number:
  US10/987,870
• Publication Number:
  US 20050283532A1
• Time in Patent Office: 1,754 Days
• Field of Search: 709/202, 709/223, 709/224, 709/226, 710/36, 711/150
• US Class Current: 709/226
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 67/306 User profiles

Abstract

A system and method for multi-modal context-sensitive applications in a home network are provided. The system includes an input processing agent classifying inputs from a predetermined input device based on at least one criterion selected from the group consisting of an input block, an input type, a priority, and ranking and then outputting the classified inputs; an input collaboration agent performing real-time analysis on each of the inputs fed by the input processing agent to obtain a pre-defined output action for an input type corresponding to each input based on a user'"'"'s context and performing a probabilistic inference on the inputs fed by the input processing agent to obtain a possible output action based on the user'"'"'s context; and an agent server obtaining an output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the output action obtained by the input collaboration agent.

Claims

1. A system for providing a dynamic user configurable output for multi-modal context-sensitive applications in a home network, the system comprising:

a processor;

a memory couple to the processor;

an input processing agent unit classifying inputs from a predetermined input device based on at least one criterion selected from the group consisting of an input block, an input type, a priority, and ranking and then outputting an input with the highest priority and the highest ranking among the classified inputs;

an input collaboration agent unit obtaining a pre-defined output action for an input type corresponding to each input based on a user'"'"'s context by performing real-time analysis on each of the inputs fed by the input processing agent unit and obtaining a possible output action based on the user'"'"'s context by performing a probabilistic inference on the inputs fed by the input processing agent unit;

a user manager agent unit to transmit the current user'"'"'s profile and a detected user context to the input collaboration agent through an agent server and obtaining a user interface in accordance with an output resource corresponding to the output action determined by the input collaboration agent, wherein the output resource, which is available in the home network and corresponds to the current user'"'"'s context, is based on the current user'"'"'s profile and the detected user context; and

the agent server obtaining the output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the output action obtained by comparing the output actions from the input collaboration agent.

2. The system of claim 1, wherein the input processing agent unit feeds the inputs received from an input agent unit comprising at least one among a speech input sub-agent, a camera input sub-agent, sketch input sub-agent, and a text input sub-agent to the input collaboration agent sequentially according to priorities assigned to the inputs and rankings determined based on quality of the inputs.

3. The system of claim 1, wherein the input collaboration agent unit comprises:

a real-time analysis agent unit obtaining a pre-defined output action for an input type corresponding to each input based on a current input category, a current input value, a previous input category, a previous input value, a previous output action, a current context, and a previous context;

a multi-modal input analysis agent unit performing the probabilistic inference on the inputs fed by the input processing agent to obtain the possible output action to be taken for a sequence of the inputs, a previous action, a similar previous sequence of inputs, and current and previous contexts associated with the inputs; and

a hybrid collaboration architecture accepting the inputs based on an input block, an input category, an input type, a current context, and a corresponding input string from the multi-modal input analysis agent and performing a reactive inference and a probabilistic inference on the accepted inputs to obtain the possible output action for the inputs.

4. The system of claim 3, wherein the hybrid collaboration architecture comprising:

a reactive inference unit performing reactive inference based on the inputs from the multi-modal input analysis agent;

a probabilistic inference unit performing probabilistic inference corresponding to the inputs from the multi-modal input analysis agent based on the inference result of the reactive inference unit;

a query best action unit performing query a best possible output action to the inputs from the multi-modal input analysis agent based on the inference result obtained by the sequential inference operation of the reactive inference unit and the probabilistic inference unit; and

a similar input collaboration unit ascertaining the best possible output action for the inputs from the multi-modal input analysis agent and outputting the best possible output action to the agent server.

5. The system of claim 1, further comprising:

a security agent unit authenticating a current user and detecting a profile and a context that correspond to the authenticated current user.

6. The system of claim 1 or 5, further comprising a device discovery agent unit discovering devices connected to the home network and obtaining configuration information regarding the devices.

7. The system of claim 1 or 5, further comprising a user interface agent unit storing the user'"'"'s profile and context and adding a new profile and context based on the user'"'"'s needs.

8. A method of providing a dynamic user configurable output for multi-modal context-sensitive applications in a home network, the method comprising:

(a) classifying inputs from a predetermined input device based on at least one criterion selected from the group consisting of an input block, an input type, a priority, and ranking and then outputting an input with the highest priority and the highest ranking among the classified inputs;

(b) obtaining a pre-defined output action for an input type corresponding to each input based on a user'"'"'s context by performing real-time analysis on each of the inputs and obtaining a possible output action based on the user'"'"'s context by performing a probabilistic inference on the inputs using a computer;

(c) obtaining an output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the output action obtained by comparing the output actions obtained in the step (b);

(d) authenticating the user and detecting a user'"'"'s profile and a user'"'"'s context that correspond to the authenticated user; and

(e) obtaining a user interface in accordance with the output resource, which is available in the home network, wherein the output resource corresponds to the user'"'"'s context, based on the detected user'"'"'s profile and the detected user'"'"'s context.

9. The method of claim 8, wherein the predetermined input device comprises at least one among a speech input agent, a camera input agent, sketch input agent, and a text input agent, and

the outputting of the classified inputs comprises sequentially outputting the inputs according to priorities assigned to the inputs and rankings determined based on quality of the inputs.

10. The method of claim 8, further comprising (f) discovering devices connected to the home network and obtaining configuration information regarding the devices.

11. The method of claim 8, further comprising (g) storing the user'"'"'s profile and context and adding a new profile and context based on the user'"'"'s needs.

12. The method of claim 8, wherein the step (b) comprising:

(b1) performing reactive inference based on the inputs from the input device;

(b2) performing probabilistic inference corresponding to the inputs from the input device based on the inference result of the reactive inference;

(b3) performing query a best possible output action to the inputs from the input device based on the inference result obtained by the reactive and probabilistic inference operation performed sequentially; and

(b4) ascertaining the best possible output action for the inputs from the input device and outputting the best possible output action.