Automated caption positioning systems and methods

• US 9,456,170 B1
• Filed: 10/07/2014
• Issued: 09/27/2016
• Est. Priority Date: 10/08/2013
• Status: Active Grant

• Current Assignee: 3Play Media, Inc.
• Original Assignee: 3Play Media, Inc.
• Inventors: Johnson, Christopher E., Antunes, Christopher S., Barron, Jeremy E., Schwartz, Andrew H., Miller, Joshua, Khesin, Anatole, Zimmerman, Roger S.
• Primary Examiner(s): Lee, Michael
• Application Number:
  US14/508,866
• Time in Patent Office: 721 Days
• Field of Search: 348/468
• US Class Current: 1/1
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

In at least one embodiment, a system for positioning captions upon video images of a media file is provided. The system includes a memory storing at least one caption frame associated with at least one video frame, at least one processor in data communication with the memory, and a caption engine component executable by the at least one processor. The caption engine component is configured to determine that at least one region of the at least one video frame is free of identified content and position the at least one caption frame within the at least one region.

Claims

1. A system for positioning captions upon video images of a media file, the system comprising:

a memory storing at least one caption frame associated with at least one video frame;

at least one processor in data communication with the memory; and

a caption engine component executable by the at least one processor and configured to;

calculate dimensions of at least one region of the at least one video frame based on caption text included in the at least one caption frame;

determine that the at least one region of the at least one video frame is free of identified content at least in part by executing a text detector component configured to limit text searching to the at least one region; and

position the at least one caption frame within the at least one region.

2. The system of claim 1, wherein the caption engine component is further configured to generate caption information including a caption based on the at least one caption frame.

3. The system of claim 1, wherein the identified content includes text present in the video frame.

4. The system of claim 1, wherein the caption engine component is further configured to determine that a default region of the at least one video frame includes identified content.

5. The system of claim 4, wherein the caption engine component is further configured to identify a series of consecutive video frames as including:

the at least one video frame; and

one or more video frames including identified content within one or more default regions.

6. The system of claim 5, wherein the caption engine component is further configured to:

calculate a duration of the series;

determine that the duration exceeds a threshold value; and

position, in response to determining that the duration exceeds the threshold value, one or more caption frames within one or more candidate regions of one or more video frames included in the series, the one or more candidate regions including the at least one region, the one or more caption frames including the at least one caption frame.

7. The system of claim 6, wherein the caption engine component is further configured to smooth positional transitions between caption frames within a series of consecutive caption frames at least in part by positioning additional caption frames within a region common to the series.

8. The system of claim 6, wherein the caption engine component is further configured to smooth positional transitions between caption frames within a series of consecutive caption frames at least in part by positioning a plurality of caption frames belonging to a textual unit within a region common to the series.

9. The system of claim 6, wherein the caption engine component is further configured to:

calculate a percentage of all caption frames associated with the media file that are within the one or more caption frames;

compare the percentage to a threshold value; and

store an indication that the media file may be a foreign language media file.

10. The system of claim 1, wherein the caption engine component is configured to determine that the at least one region of the at least one video frame is free of identified content by determining that a confidence value that a default region contains text is greater than a confidence value that the at least one region contains text.

11. The system of claim 1, further comprising a user interface component executable by the at least one processor and configured to:

receive input to place the at least one caption frame at a new location; and

position, responsive to receiving the input, the at least one caption at the new location.

12. The system of claim 1, wherein the caption engine component is further configured to:

analyze audio information associated with the media file to generate an analysis result;

determine at least one originating field relative to the video frame based on the analysis result; and

move the at least one region to the at least one originating field.

13. The system of claim 12, wherein the analysis result is at least one of an inter-channel level difference, an inter-channel time difference, an automatic speech recognition right-channel text output, an automatic speech recognition left-channel text output, and means and variance of audio clusters.

14. The system of claim 12, wherein the caption engine component is further configured to:

identify a plurality of source identifiers within the caption text of the at least one caption frame; and

split the at least one caption frame into a plurality of caption frames respectively corresponding to the plurality of source identifiers.

15. A method for positioning captions upon video images of a media file, the method comprising:

calculating dimensions of at least one region of at least one video frame based on caption text included in at least one caption frame;

determining that the at least one region of the at least one video frame is free of identified content at least in part by executing a text detector component configured to limit text searching to the at least one region; and

positioning the at least one caption frame within the at least one region.

16. The method of claim 15, further comprising generating caption information including a caption based on the at least one caption frame.

20. The method of claim 15, further comprising determining that a default region of the at least one video frame includes identified content.

17. A non-transitory computer readable medium storing sequences of computer executable instructions for positioning captions upon video images of a media file, the sequences of computer executable instructions including instructions configured to instruct at least one processor to:

calculate dimensions of at least one region of at least one video frame based on caption text included in at least one caption frame;

determine that at the least one region of the at least one video frame is free of identified content at least in part by executing a text detector component configured to limit text searching to the at least one region; and

position the at least one caption frame within the at least one region.

18. The non-transitory computer readable medium of claim 17, wherein the instructions are further configured to instruct the at least processor to generate caption information including a caption based on the at least one caption frame.

19. The non-transitory computer readable medium of claim 17, wherein the instructions are further configured to instruct the at least processor to determine that a default region of the at least one video frame includes identified content.