APPARATUS AND METHOD FOR TRANSFORMING APPLICATION FOR MULTI-MODAL INTERFACE

• US 20080120447A1
• Filed: 11/20/2007
• Published: 05/22/2008
• Est. Priority Date: 11/21/2006
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: PARK, Dong-Hwan, SON, Young-Sung, MOON, Kyeong-Deok, KU, Tai-Yeon, PARK, Jun-Hee
• Granted Patent: US 8,881,001 B2
• Time in Patent Office: Days
• US Class Current: 710/65
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

Provided are an apparatus and method for transforming an application for a multi-modal interface. The apparatus includes: a parsing means for parsing an original application to generate a parsed structure of the original application; an application transforming/creating means for transforming the parsed structure generated by the parsing means; a multi-modal synthesizing means for synthesizing the contents created by the application transforming/creating means with multi-modal information of the user terminal; a multi-modal detecting means for detecting multi-modal information received from the user terminal and transforming the detected information into contents that can be used by the device; and an action mapping means for mapping the contents, which are received from the multi-modal detecting means and can be used in the device, to action information contained in the original application to generate an event of the device according to the action information.

Claims

1. A multi-modal application transform apparatus connected to a multi-modal user terminal to manage an original application of a device in a home network, the multi-modal application transform apparatus comprising:

a parsing means for parsing an original application to generate a parsed structure of the original application;

an application transforming/creating means for transforming the parsed structure generated by the parsing means according to the output characteristics of the user terminal to create contents that can be used by the user terminal;

a multi-modal synthesizing means for synthesizing the contents created by the application transforming/creating means with multi-modal information of the user terminal;

a multi-modal detecting means for detecting multi-modal information received from the user terminal and transforming the detected information into contents that can be used by the device; and

an action mapping means for mapping the contents, which are received from the multi-modal detecting means and can be used in the device, to action information contained in the original application to generate an event of the device according to the action information.

2. The multi-modal application transform apparatus of claim 1, wherein the action mapping means includes:

a content transforming unit for transforming multi-modal information detected by the multi-modal detecting means into contents that can be used by the device and extracting action information of the original application corresponding to the multi-modal information;

a scoring unit for giving, using the multi-modal information detected by the multi-modal detecting means, a score to the action information extracted by the content transforming unit on the basis of the current state; and

an action transforming unit for transforming, according to the score result received from the scoring unit, the action information into multi-modal information that can be detected by the device, and providing the multi-modal information to the device.

3. The multi-modal application transform apparatus of claim 1, wherein the application transforming/creating means pre-stores and uses a policy describing a transform of the original application according to the output characteristics of the user terminal.

4. The multi-modal application transform apparatus of claim 1, wherein the application transforming/creating means creates contents that can be used by the user terminal in a text format.

5. The multi-modal application transform apparatus of claim 1, wherein the multi-modal detecting means transforms contents that can be used by the device into a text format.

6. A multi-modal application transform method for transforming a multi-modal application provided to a multi-modal user terminal to manage an original application of a device in a home network, the multi-modal application transform method comprising the steps of:

a) transforming the original application according to the output characteristics of the user terminal to create contents that can be used by the user terminal;

b) synthesizing the contents that can be used by the user terminal with multi-modal information of the user terminal and providing the resulting data to the user terminal;

c) transforming multi-modal information detected by the user terminal into contents that can be used by the device and extracting action information of the original application corresponding to the multi-modal information;

d) giving, according to the multi-modal information detected by the user terminal, a score to the action information extracted from the original application corresponding to the multi-modal information; and

e) transforming, according to the score given to the action information, the action information into multi-modal information that can be detected by the device, and providing the multi-modal information to the device.

7. The multi-modal application transform method of claim 6, wherein the contents that can be used by the user terminal are created in a text format in the step a).

8. The multi-modal application transform method of claim 6, wherein the multi-modal information detected by the user terminal is transformed into text contents in the step c).