Intelligent closed captioning

• US 10,321,204 B2
• Filed: 07/11/2014
• Issued: 06/11/2019
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: Caliendo, Jr., Neal Robert, VanBlon, Russell Speight, Weksler, Arnold S.
• Primary Examiner(s): Han, Qi
• Application Number:
  US14/329,374
• Publication Number:
  US 20160014476A1
• Time in Patent Office: 1,796 Days
• Field of Search: 704235, 704231, 704246, 704250, 704257, 7042701, 704275
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4396 by muting the audio signal
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

An aspect provides a method, including: playing, on a display device, video content; providing, using at least one speaker, audio content associated with the video content; obtaining, from an external source, data relating to playback context; determining, using a processor, that the data relating to playback context is associated with a reduced audibility context; and providing, on the display device, textual data associated with dialog of the video content. Other aspects are described and claimed.

Claims

1. A method, comprising:

playing, on a display device, video content;

providing, using at least one speaker, audio content associated with the video content;

obtaining, from an external source, playback context data, wherein the playback context data comprises contextual data associated with user comprehension of the audio content and wherein the playback context data comprises ambient audio data;

determining, using a processor, that the playback context data is associated with a reduced audibility context, wherein the determining comprises;

analyzing, using a speech recognition engine, the ambient audio data;

identifying a predetermined trigger expression in the ambient audio data;

accessing, responsive to identifying the predetermined trigger expression, closed caption data; and

providing, on the display device, the closed caption data associated with the dialogue and the ambient audio data comprising the predetermined trigger expression.

2. The method of claim 1, further comprising associating, using a processor, the playback context data associated with a reduced audibility context with the dialogue of the audio content.

3. The method of claim 1, wherein the closed caption data includes a reference to previously displayed video data.

4. The method of claim 3, wherein the reference indicates a particular speaker within the video data.

5. The method of claim 1, wherein the predetermined trigger expression is associated using natural language processing to a user provided indication of inadequate audio.

6. An electronic device, comprising:

a speech recognition engine;

a display device;

an input component;

at least one speaker;

one or more processors; and

a memory operatively coupled to the one or more processors, the display device, the at least one speaker, and the input component, the memory storing instructions executable by the one or more processors, the instructions being executable by the one or more processors to;

play, on the display device, video content;

provide, using the at least one speaker, audio content associated with the video content;

obtain, from an external source, playback context data, wherein the playback context data comprises data associated with user comprehension of the audio content and wherein the playback context data comprises ambient audio data;

determine that the playback context data is associated with a reduced audibility context, wherein the instructions executable by the processor to determine comprise instructions executable by the processor to;

analyze the ambient audio data;

identify a predetermined trigger expression in the ambient audio data;

access, responsive to identifying the predetermined trigger expression, closed caption data; and

provide, on the display device, the closed caption data associated with the dialogue and the ambient audio data comprising the predetermined trigger expression.

7. The electronic device of claim 6, wherein the instructions are further executable by the one or more processors to associate the playback context data associated with a reduced audibility context with the dialogue of the audio content.

8. The electronic device of claim 6, wherein the closed caption data includes a reference to previously displayed video data.

9. The electronic device of claim 8, wherein the reference indicates a particular speaker within the video data.

10. A computer program product, comprising:

a storage device that stores instructions executable by a processor, the instructions comprising;

instructions that play, on a display device, video content;

instructions that provide, using at least one speaker, audio content associated with the video content;

instructions that obtain, from an external source, playback context data, wherein the playback context data comprises contextual data associated with user comprehension of the audio content and wherein the playback context data comprises ambient audio data;

instructions that determine, using a processor, that the playback context data is associated with a reduced audibility context, wherein the instructions that determine comprise instructions that;

analyze the ambient audio data;

identify a predetermined trigger expression in the ambient audio data;

access, responsive to identifying the predetermined trigger expression, closed caption data; and

provide, on the display device, the closed caption data associated with the dialogue and the ambient audio data comprising the predetermined trigger expression.