METHOD FOR PROVIDING SERVICE BASED ON MULTIMODAL INPUT AND ELECTRONIC DEVICE THEREOF

• US 20140365215A1
• Filed: 06/05/2014
• Published: 12/11/2014
• Est. Priority Date: 06/05/2013
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KIM, Kyung-Tae
• Application Number:
  US14/297,042
• Publication Number:
  US 20140365215A1
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method for operating an electronic device is provided. The method includes receiving a voice signal and an input, extracting data corresponding to the input, converting the voice signal to text data, setting an association relationship between the converted text data and the extracted data, and generating a response for the voice signal based on the converted text data, the extracted data, and the set association relationship.

Claims

1. A method in an electronic device, the method comprising:

receiving a voice signal and an input;

extracting data corresponding to the input;

converting the voice signal to text data;

setting an association relationship between the converted text data and the extracted data; and

generating a response for the voice signal based on the converted text data, the extracted data, and the set association relationship.

2. The method of claim 1, further comprising:

acquiring first time information at which the voice signal is received; and

acquiring second time information at which the input is received,

wherein the association relationship between the text data and the extracted data is set using the first time information and the second time information.

3. The method of claim 2, wherein setting the association relationship between the text data and the extracted data comprises:

identifying at least one sub-text data requiring additional information from the text data;

extracting third time information corresponding to the at least one sub-text data from the first time information at which the voice signal is received; and

setting the association relationship between the at least one sub-text data and the extracted data based on the second time information and the third time information.

4. The method of claim 3, wherein setting the association relationship between the at least one sub-text data and the extracted data based on the second time information and the third time information comprises:

determining data associated with the at least one sub-text data among the extracted data based on the second time information and the third time information; and

setting the determined data as the additional data associated with the at least one sub-text data.

5. The method of claim 1, wherein setting the association relationship between the text data and the extracted data comprises:

identifying at least one sub-text data requiring additional information from the text data; and

setting an association relationship between the at least one sub-text data and the extracted data.

6. The method of claim 5, wherein the sub-text data includes at least one word representing a demonstrative pronoun and an order.

7. The method of claim 6, wherein setting the association relationship between the at least one sub-text data and the extracted data comprises:

determining an order of data corresponding to the input when the sub-text data is a word representing an order; and

setting an association relationship between the sub-text and the extracted data based on the determined order.

8. The method of claim 1, wherein extracting the data corresponding to the input comprises:

identifying an application during display in the electronic device; and

extracting data of the application corresponding to the input.

9. The method of claim 1, wherein generating the response for the voice signal based on the text data, the extracted data, and the set association relationship comprises:

analyzing the text data;

determining a meaning of the text data based on the extracted data when there is text data having a plurality of meanings in the text data;

determining an intent of the text data based on the determined meaning, the text data, the extracted data, and the set association relationship; and

generating a response corresponding to the determined intention.

10. A method for operating an electronic device, comprising:

receiving a voice signal;

storing first time information at which the voice signal is received;

receiving an input;

storing second time information corresponding to the input;

extracting data corresponding to the input;

converting the voice signal to text data; and

generating a response for the voice signal based on the text data, the first time information, the extracted data, and the second time information.

11. An electronic device comprising:

a voice input device configured to receive a voice signal;

another input device configured to receive an input; and

a control unit configured to extract data corresponding to the input, convert the voice signal to text data, set an association relationship between the text data and the extracted data, and generate a response for the voice signal based on the text data, the extracted data, and the set association relationship.

12. The electronic device of claim 11, wherein

the control unit acquires first time information at which the voice signal is received, and second time information at which the input is received, and

the association relationship between the text data and the extracted data is set using the first time information and the second time information.

13. The electronic device of claim 12, wherein the control unit identifies at least one sub-text data requiring additional information from the text data, extracts third time information corresponding to the at least one sub-text data from the first time information at which the voice signal is received, and sets the association relationship between the at least one sub-text data and the extracted data based on the second time information and the third time information.

14. The electronic device of claim 13, wherein the control unit determines data associated with the at least one sub-text data among the extracted data based on the second time information and the third time information, and sets the determined data as the additional data associated with the at least one sub-text data.

15. The electronic device of claim 11, wherein the control unit identifies at least one sub-text data requiring additional information from the text data, and sets an association relationship between the at least one sub-text data and the extracted data.

16. The electronic device of claim 15, wherein the sub-text data includes at least one of words representing a demonstrative pronoun and an order.

17. The electronic device of claim 16, wherein the control unit determines an order of data corresponding to the input when the sub-text data is a word representing an order, and sets an association relationship between the sub-text data and the extracted data based on the determined order.

18. The electronic device of claim 11, wherein the control unit identifies an application during display in the electronic device, and extracts data of the application corresponding to the input.

19. The electronic device of claim 11, wherein the control unit analyzes the text data, determines a meaning of the text data based on the extracted data when there is text data having a plurality of meanings in the text data, determines an intent of the text data based on the determined meaning, the text data, the extracted data, and the set association relationship, and generates a response corresponding to the determined intention.

20. An electronic device comprising:

a voice input device configured to receive a voice signal;

an input device configured to receive an input; and

a control unit configured to acquire first time information at which the voice signal is received and second time information at which the input is received, extract data corresponding to the input, convert the voice signal to text data, and generate a response for the voice signal based on the text data, the first time information, the extracted data, and the second time information.