INTELLIGENT CLOSED CAPTIONING

• US 20160014476A1
• Filed: 07/11/2014
• Published: 01/14/2016
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: Caliendo, Neal Robert Jr., VanBlon, Russell Speight, Weksler, Arnold S.
• Granted Patent: US 10,321,204 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4396 by muting the audio signal
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

An aspect provides a method, including: playing, on a display device, video content; providing, using at least one speaker, audio content associated with the video content; obtaining, from an external source, data relating to playback context; determining, using a processor, that the data relating to playback context is associated with a reduced audibility context; and providing, on the display device, textual data associated with dialogue of the video content. Other aspects are described and claimed.

Claims

1. A method, comprising:

playing, on a display device, video content;

providing, using at least one speaker, audio content associated with the video content;

obtaining, from an external source, data relating to playback context;

determining, using a processor, that the data relating to playback context is associated with a reduced audibility context; and

providing, on the display device, textual data associated with dialogue of the video content.

2. The method of claim 1, further comprising associating, using a processor, the data associated with a reduced audibility context with dialogue of the audio content.

3. The method of claim 1, wherein the data relating to playback context is selected from the group consisting of gesture data, biometric data, ambient audio data, and display device control data.

4. The method of claim 3, further comprising:

analyzing, using a speech recognition engine, the ambient audio data;

assigning a confidence score relating to recognition of the ambient audio data;

associating dialogue of the audio content with the ambient audio data having a low confidence score; and

in response to the confidence score not exceeding a predetermined threshold, accessing closed caption data;

wherein providing comprises providing closed caption data associated with the dialogue having the low confidence score.

5. The method of claim 4, wherein the closed caption data includes a reference to previously displayed video data.

6. The method of claim 5, wherein the reference indicates a particular speaker within the video data.

7. The method of claim 3, further comprising:

analyzing, using a speech recognition engine, the ambient audio data;

identifying a predetermined trigger expression in recognized ambient audio data; and

in response to identifying the predetermined trigger expression, accessing closed caption data;

wherein the providing comprises providing closed caption data associated with the dialogue and the ambient audio data containing the predetermined trigger expression.

8. The method of claim 7, wherein the predetermined trigger expression is associated using natural language processing to a user provided indication of inadequate audio.

9. The method of claim 3, further comprising:

analyzing the gesture data using a gesture recognition engine;

identifying a predetermined trigger gesture in the gesture data; and

in response to identifying the predetermined trigger gesture, accessing closed caption data;

wherein the providing comprises providing closed caption data associated with the dialogue and the predetermined trigger gesture.

10. The method of claim 3, further comprising:

analyzing the biometric data to identify a specific individual; and

in response to identifying the specific individual, accessing closed caption data;

wherein the providing comprises providing closed caption data associated with the dialogue and the identification of the specific individual.

11. The method of claim 3, further comprising:

analyzing display control data to identify a display control context indicative of a reduced audibility context; and

accessing closed caption data associated in time with the display control context indicative of a reduced audibility context;

wherein the providing comprises providing closed caption data associated with the dialogue and the display control context indicative of a reduced audibility context.

12. An electronic device, comprising:

a display device;

an input component;

at least one speaker;

one or more processors; and

a memory operatively coupled to the one or more processors, the display device, the at least one speaker, and the input component, the memory storing instructions executable by the one or more processors, the instructions being executable by the one or more processors to;

play, on the display device, video content;

provide, using the at least one speaker, audio content associated with the video content;

obtain, from an external source, data relating to playback context;

determine that the data relating to playback context is associated with a reduced audibility context; and

provide, on the display device, textual data associated with the dialogue of the video content.

13. The electronic device of claim 12, wherein the instructions are further executable by the one or more processors to associate the data associated with a reduced audibility context with dialogue of the audio content.

14. The electronic device of claim 12, wherein the data relating to playback context is selected from the group consisting of gesture data, biometric data, ambient audio data, and display device control data.

15. The method of claim 14, wherein the instructions are further executable by the one or more processors to:

analyze, using a speech recognition engine, the ambient audio data;

assign a confidence score relating to recognition of the ambient audio data;

associate dialogue of the audio content with the ambient audio data having the low confidence score; and

in response to the confidence score not exceeding a predetermined threshold, access closed caption data;

wherein to provide comprises providing closed caption data associated with the dialogue having the low confidence score.

16. The electronic device of claim 15, wherein the closed caption data includes a reference to previously displayed video data.

17. The electronic device of claim 16, wherein the reference indicates a particular speaker within the video data.

18. The electronic device of claim 14, wherein the instructions are further executable by the one or more processors to:

analyze, using a speech recognition engine, the ambient audio data;

identify a predetermined trigger expression in recognized ambient audio data; and

in response to identifying the predetermined trigger expression, access closed caption data;

wherein to provide comprises providing closed caption data associated with the dialogue and the ambient audio data containing the predetermined trigger expression.

19. The electronic device of claim 18, wherein the predetermined trigger expression is mapped using natural language processing to a user provided indication of inadequate audio.

20. The electronic device of claim 14, wherein the instructions are further executable by the one or more processors to:

analyze the gesture data using a gesture recognition engine;

identify a predetermined trigger gesture in the gesture data; and

in response to identifying the predetermined trigger gesture, access closed caption data;

wherein to provide comprises providing closed caption data associated with the dialogue and the predetermined trigger gesture.

21. The electronic device of claim 14, wherein the instructions are further executable by the one or more processors to:

analyze the biometric data to identify a specific individual; and

in response to identifying the specific individual, access closed caption data;

wherein to provide comprises providing closed caption data associated with the dialogue and the identification of the specific individual.

22. A computer program product, comprising:

a storage device that stores instructions executable by a processor, the instructions comprising;

instructions that play, on a display device, video content;

instructions that provide, using at least one speaker, audio content associated with the video content;

instructions that obtain, from an external source, data relating to playback context;

instructions that determine, using a processor, that the data relating to playback context is associated with a reduced audibility context; and

instructions that provide, on the display device, textual data associated with dialogue of the video content.