Displaying additional data about outputted media data by a display device for a speech search command

• US 9,547,716 B2
• Filed: 07/29/2013
• Issued: 01/17/2017
• Est. Priority Date: 08/29/2012
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Yongsin
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US13/953,313
• Publication Number:
  US 20140067402A1
• Time in Patent Office: 1,268 Days
• Field of Search: 704/231, 704/251, 704/252, 704/254, 704/270, 707/725, 707/758, 707/760, 725/53
• US Class Current: 1/1
• CPC Class Codes: G06F 16/632 Query formulation
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612

Abstract

A speech search method performed by a display device, the method including outputting media data including audio data, receiving a speech search command for additional data about the outputted media data from a user, the speech search command including at least one query word, determining whether the at least one query word matches a query term that is full and searchable, when the at least one query word matches the query term that is full and searchable, performing a search for the additional data using the query term, and when the at least one query word does not match the query term that is full and searchable, determining the query term from a predetermined amount of the audio data prior to receiving the speech search command and performing the search for the additional data using the query term.

Claims

1. A speech search method performed by a display device, the method comprising:

outputting media data including audio data;

receiving a speech search command for additional data about the outputted media data from a user, the speech search command including at least one query word;

determining whether the at least one query word matches a query term that is full and searchable based on pronunciation patterns of the user;

when the at least one query word matches the query term that is full and searchable and the pronunciation pattern is a first pattern, performing a search for the additional data using the query term; and

when the at least one query word does not match the query term that is full and searchable and the pronunciation pattern is a second pattern, determining the query term from a predetermined amount of the audio data prior to receiving the speech search command and performing the search for the additional data using the query term,

wherein the first pattern means at least one of that the user'"'"'s accent is clear and that there is no mumbling in the user'"'"'s pronunciation, and

wherein the second pattern means at least one of that the user'"'"'s accent is unclear and that there is mumbling in the user'"'"'s pronunciation.

2. The method of claim 1, wherein the context information comprises at least one of title information, genre information, character information, scene information, schedule information, audio related information, and image related information of the media data.

3. The method of claim 1, wherein the step of determining the query term from the predetermined amount of the audio data comprises:

voice-recognizing the predetermined amount of the audio data;

extracting at least one query term candidate from a corresponding voice recognition result; and

determining the at least one query word that matches the query term that is full and searchable from the at least one query term candidate.

4. The method of claim 3, the method further comprising:

outputting the at least one query term candidate when the at least one query term candidate does not match the at least one query word.

5. The method of claim 1, wherein the step of determining the query term from the predetermined amount of the audio data comprises:

voice-recognizing the predetermined amount of the audio data;

extracting at least one query term candidate from a corresponding voice recognition result;

outputting the at least one query term candidate; and

receiving a command for selecting the query term from the at least one query term candidate.

6. The method of claim 5, wherein the outputting step of the at least one query term candidate comprises

providing the user with the at least one query term candidate in a chronological order;

and

providing the user with an image of the media data while the at least one query term candidate being outputted.

7. The method of claim 1, wherein the step of performing the search further comprises:

providing the user with a confirmation request for the determined query term; and

when the determined query term is confirmed by the user, performing the search using the confirmed query term.

8. The method of claim 1, wherein the step of determining whether the at least one query word matches the query term that is full and searchable further comprises:

determining whether the at least one query word corresponds to the full and searchable query term or a partial query term.

9. A display device, comprising:

a media data processing module configured to process media data;

a media data output unit configured to output the processed media data;

an audio input unit configured to receive a speech search command for additional data about the outputted media data from a user, the speech search command including at least one query word; and

a processor configured to

determine whether the at least one query word matches a query term that is full and searchable based on pronunciation patterns of the user,

when the at least one query word matches a query term that is full and searchable and the pronunciation pattern is a first pattern, perform a search for the additional data using the query term, and

when the at least one query word does not match the query term that is full and searchable and the pronunciation pattern is a second pattern, determine the query term from a predetermined amount of the audio data prior to receiving the speech search command and perform the search for the additional data using the query term,

wherein the first pattern means at least one of that the user'"'"'s accent is clear and that there is no mumbling in the user'"'"'s pronunciation, and

wherein the second pattern means at least one of that the user'"'"'s accent is unclear and that there is mumbling in the user'"'"'s pronunciation.

10. The display device of claim 9, wherein the context information comprises at least one of title information, genre information, character information, scene information, schedule information, audio related information, and image related information of the media data.

11. The display device of claim 9, wherein the processor is further configured to

voice-recognize the predetermined amount of the audio data;

extract at least one query term candidate from a corresponding voice recognition result; and

determine the at least one query word that matches the query term that is full and searchable from the at least one query term candidate.

12. The display device of claim 11, wherein the processor is further configured to

output the at least one query term candidate when the at least one query term candidate does not match the at least one query word.

13. The display device of claim 9, wherein the processor is further configured to

voice-recognize the predetermined amount of the audio data;

extract at least one query term candidate from a corresponding voice recognition result;

output the at least one query term candidate; and

receive a command for selecting the query term from the at least one query term candidate.

14. The display device of claim 13, wherein the processor is further configured to

provide the user with the at least one query term candidate in a chronological order;

and

provide the user with an image of the media data while the at least one query term candidate being outputted.

15. The display device of claim 9, wherein the processor is further configured to

provide the user with a confirmation request for the determined query term;

and

when the determined query term is confirmed by the user, perform the search using the confirmed query term.

16. The display device of claim 9, wherein the processor is further configured to determine whether at least one query word corresponds to a full and searchable query term or a partial query term.