System and method for multi-modal context-sensitive applications in home network environment

• US 20050283532A1
• Filed: 11/12/2004
• Published: 12/22/2005
• Est. Priority Date: 11/14/2003
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Ji Yong, Kwak, Ji Young, Lee, Eun Ryung, Vinod, Joseph C., Ahn, Sung Ho, Kim, Doo Hyun, Sul, Dong Myung
• Granted Patent: US 7,584,280 B2
• Time in Patent Office: Days
• US Class Current: 709/225
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 67/306 User profiles

Abstract

A system and method for multi-modal context-sensitive applications in a home network are provided. The system includes an input processing agent classifying inputs from a predetermined input device based on at least one criterion selected from the group consisting of an input block, an input type, a priority, and ranking and then outputting the classified inputs; an input collaboration agent performing real-time analysis on each of the inputs fed by the input processing agent to obtain a pre-defined output action for an input type corresponding to each input based on a user'"'"'s context and performing a probabilistic inference on the inputs fed by the input processing agent to obtain a possible output action based on the user'"'"'s context; and an agent server obtaining an output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the output action obtained by the input collaboration agent.

Claims

1. A system for providing a dynamic user configurable output for multi-modal context-sensitive applications in a home network, the system comprising:

an input processing agent classifying inputs from a predetermined input device based on at least one criterion selected from the group consisting of an input block, an input type, a priority, and ranking and then outputting the classified inputs;

an input collaboration agent performing real-time analysis on each of the inputs fed by the input processing agent to obtain a pre-defined output action for an input type corresponding to each input based on a user'"'"'s context and performing a probabilistic inference on the inputs fed by the input processing agent to obtain a possible output action based on the user'"'"'s context; and

an agent server obtaining an output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the output action obtained by the input collaboration agent.

2. The system of claim 1, wherein the input processing agent feeds the inputs received from an input agent comprising at least one among a speech input sub-agent, a camera input sub-agent, sketch input sub-agent, and a text input sub-agent to the input collaboration agent sequentially according to priorities assigned to the inputs and rankings determined based on quality of the inputs.

3. The system of claim 1, wherein the input collaboration agent comprises:

a real-time analysis agent obtaining a pre-defined output action for an input type corresponding to each input based on a current input category, a current input value, a previous input category, a previous input value, a previous output action, a current context, and a previous context;

a multi-modal input analysis agent performing the probabilistic inference on the inputs fed by the input processing agent to obtain the possible output action to be taken for a sequence of the inputs, a previous action, a similar previous sequence of inputs, and current and previous contexts associated with the inputs; and

a hybrid collaboration architecture accepting the inputs based on an input block, an input category, an input type, a current context, and a corresponding input string from the multi-modal input analysis agent and performing a reactive inference and a probabilistic inference on the accepted inputs to obtain the possible output action for the inputs.

4. The system of claim 3, wherein the hybrid collaboration architecture comprising:

a reactive inference unit performing reactive inference based on the inputs from the multi-modal input analysis agent;

a probabilistic inference unit performing probabilistic inference corresponding to the inputs from the multi-modal input analysis agent based on the inference result of the reactive inference unit;

a query best action unit performing query a best possible output action to the inputs from the multi-modal input analysis agent based on the inference result obtained by the sequential inference operation of the reactive inference unit and the probabilistic inference unit; and

a similar input collaboration unit ascertaining the best possible output action for the inputs from the multi-modal input analysis agent and outputting the best possible output action to the agent server.

5. The system of claim 1, further comprising:

a security agent authenticating a current user and detecting a profile and a context that correspond to the authenticated current user; and

a user manager agent transmitting the current user'"'"'s profile and context detected by the security agent to the input collaboration agent through the agent server and obtaining a user interface in accordance with the output resource, which is available in the home network and corresponds to the current user'"'"'s context, based on the current'"'"'s user'"'"'s profile and context.

6. The system of claim 1 or 5, further comprising a device discovery agent discovering devices connected to the home network and obtaining configuration information regarding the devices.

7. The system of claim 1 or 5, further comprising a user interface agent storing the user'"'"'s profile and context and adding a new profile and context based on the user'"'"'s needs.

8. A method of providing a dynamic user configurable output for multi-modal context-sensitive applications in a home network, the method comprising:

(a) classifying inputs from a predetermined input device based on at least one criterion selected from the group consisting of an input block, an input type, a priority, and ranking and then outputting the classified inputs;

(b) performing real-time analysis on each of the inputs to obtain a pre-defined output action for an input type corresponding to each input based on a user'"'"'s context or performing a probabilistic inference on the inputs to obtain a possible output action based on the user'"'"'s context; and

(c) obtaining an output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the obtained output action.

9. The method of claim 8, wherein the predetermined input device comprises at least one among a speech input agent, a camera input agent, sketch input agent, and a text input agent, and

the outputting of the classified inputs comprises sequentially outputting the inputs according to priorities assigned to the inputs and rankings determined based on quality of the inputs.

10. The method of claim 8, further comprising:

(d) authenticating the user and detecting a profile and a context that correspond to the authenticated user; and

(e) obtaining a user interface in accordance with the output resource, which is available in the home network and corresponds to the user'"'"'s context, based on the detected user'"'"'s profile and context.

11. The method of claim 8 or 10, further comprising (f) discovering devices connected to the home network and obtaining configuration information regarding the devices.

12. The method of claim 8 or 10, further comprising (g) storing the user'"'"'s profile and context and adding a new profile and context based on the user'"'"'s needs.

13. The method of claim 8, wherein the step (b) comprising:

(b1) performing reactive inference based on the inputs from the input device;

(b2) performing probabilistic inference corresponding to the inputs from the input device based on the inference result of the reactive inference;

(b3) performing query a best possible output action to the inputs from the input device based on the inference result obtained by the reactive and probabilistic inference operation performed sequentially; and

(b4) ascertaining the best possible output action for the inputs from the input device and outputting the best possible output action.