Video pictures pattern detection

• US 9,661,261 B2
• Filed: 03/26/2014
• Issued: 05/23/2017
• Est. Priority Date: 05/04/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lin, Ken Kengkuan
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Hasan, Syed
• Application Number:
  US14/226,753
• Publication Number:
  US 20140205268A1
• Time in Patent Office: 1,154 Days
• Field of Search: 348441, 3482221, 348443, 348445, 348448, 348456, 348458, 348459, 348 97, 348478, 348701, 382194, 382128, 386284, 386241, 37524026
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 5/91 Television signal processing therefor
• CPC Class Codes: H04N 7/0112 one of the standards corresponding to a cinematograph film standard
• CPC Class Codes: H04N 7/0115 with details on the detection of a particular field or frame pattern in the incoming video signal, e.g. 3:2 pull-down pattern
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard

Abstract

Some embodiments of the invention provide a media-editing application that performs frame rate conversion detection on a video. For a video that has been converted from one frame rate and format to another frame rate and format, the application detects the conversion method that has been used in the conversion of the video. Some embodiments perform this frame rate conversion detection by detecting patterns of repeating fields and/or frames in a video sequence created by the different conversion processes.

Some embodiments compute (i) a frame difference value for each consecutive frames, (ii) a correlation score for the first and second fields of each frame, and (iii) a correlation score for the second field of each frame and the first field of a succeeding frame. Frame difference values are compared with each other to detect repeated frames and correlation scores are compared with each other to detect repeated fields.

Claims

1. A method for a media editing application that is stored in a non-transitory machine readable medium of an electronic device and executed by at least one processing unit of the electronic device, the method for converting a frame rate of a video file to a native frame rate, the method comprising:

identifying a current frame rate of the video file;

based on the current frame rate, generating a list of encoding cadences, each encoding cadence serving as a candidate encoding cadence that may have been used to generate the video file;

at a format recognition module of the media editing application, for each encoding cadence of the list, applying a set of comparisons to a group of consecutive frames in a sequence of frames of the video file in order to try to identify a pattern of frames that would exist in the video file had the encoding cadence been used to generate the video file; and

at a video editing module of the media editing application, after identifying the pattern of frames by applying the set of comparisons for the particular encoding cadence, using the particular encoding cadence to convert the current frame rate to the native frame rate of the video file.

2. The method of claim 1, wherein identifying the current frame rate comprises extracting the current frame rate from metadata stored in the video file.

3. The method of claim 1, wherein the native frame rate of the video file comprises a frame rate at which the video file was originally created.

4. The method of claim 3, wherein the video file is imported to the media editing application after it was originally created, wherein the video file is converted to the native frame rate in order to edit the video file with the frame rate the video file was originally created.

5. The method of claim 1, wherein applying the set of comparisons to identify the pattern comprises:

identifying first, second, and third pairs of consecutive frames within the sequence of frames, the first pair of consecutive frames having a smallest frame difference in the sequence of frames, the second pair of consecutive frames having a second smallest frame difference in the sequence of frames, the third pair of consecutive frames having a third smallest frame difference in the sequence of frames;

determining that the first and second pairs of consecutive frames are apart from each other by a predetermined number of different frames;

computing a difference between the third smallest difference and the second smallest difference; and

based on the computed difference, identifying the pattern of frames in the video file.

6. The method of claim 5, wherein a difference between two frames of a pair of consecutive frames is a sum of absolute differences between each pixel in each row of one frame and a corresponding pixel in a corresponding row of the other frame.

7. The method of claim 5, wherein identifying the pattern based on the computed difference comprises determining that the computed difference exceeds a predetermined threshold.

8. The method of claim 5 further comprising determining that the first pair of consecutive frames do not overlap with the second pair of consecutive frames.

9. The method of claim 5 further comprising computing a difference between each pair of consecutive frames in the sequence, each pair of consecutive frames having one overlapping frame with at least one adjacent pair of consecutive frames in the sequence.

10. The method of claim 5, wherein the particular encoding cadence is further used to convert the current frame format of the video file to a native frame format.

11. The method of claim 10, wherein the predetermined number of frames is 3 when the particular encoding cadence is a 2:

2;

2;

4 pull-down for converting the video file from a progressive format having 24 frames per second to an interlaced format having 60 fields per second.

12. The method of claim 10, wherein the predetermined number of frames is 23 when the particular encoding cadence is a 24@25 repeat for converting the video file from 24 frames per second in progressive format to 25 frames per second progressive format.

13. A non-transitory machine readable medium storing a media editing application for identifying a rate conversion method applied to a video file, the application executable by at least one processing unit of an electronic device, the application comprising sets of instructions for:

identifying a first difference between a first pair of consecutive frames in a sequence of frames in a display order;

identifying a second difference between a second pair of consecutive frames that immediately precedes the first pair of consecutive frames in the sequence;

identifying a third difference between a third pair of consecutive frames that immediately follows the first pair of consecutive frames in the sequence;

identifying a fourth difference between a fourth pair of consecutive frames that immediately follows the third pair of consecutive frames in the sequence;

when the first difference is larger than each of the second, third, and fourth differences, identifying a pattern of repeated frames in the video file; and

converting a current frame rate and format of the video file back to a native frame rate and format of the video file using the identified pattern.

14. The non-transitory machine readable medium of claim 13, wherein the application further comprises sets of instructions for, before identifying the first difference:

identifying the current frame rate and format of the video file; and

determining that a particular one of a set of conversion methods that are capable of generating the current frame rate and format of the video file has been applied to the video file.

15. The non-transitory machine readable medium of claim 14, wherein the set of instructions for identifying the current frame rate and format of the video file comprises a set of instructions for extracting the current frame rate and format of the video file from meta data stored in the video file.

16. The non-transitory machine readable medium of claim 13, wherein the set of instructions for identifying the pattern comprises sets of instructions for:

identifying a fifth difference between a fifth pair of consecutive frames that immediately follows the fourth pair of consecutive frames in the sequence;

identifying a sixth difference between a sixth pair of consecutive frames that immediately follows the fifth pair of consecutive frames in the sequence; and

determining whether the fifth difference is larger than each of the third, fourth, and sixth differences.

17. The non-transitory machine readable medium of claim 13, wherein the pattern is generated by a 3:

2 pulldown process for converting the video file from 24p to 60p.

18. The non-transitory machine readable medium of claim 13, wherein a difference between two frames of a pair of consecutive frames is a sum of absolute differences between each pixel in each row of one frame and a corresponding pixel in a corresponding row of the other frame.

19. The non-transitory machine readable medium of claim 13, wherein the native frame rate and format of the video file comprises a frame rate and format at which the video file was originally created or captured.