Systems and methods for determining a sample frame order for analyzing a video

• US 10,282,632 B1
• Filed: 09/21/2016
• Issued: 05/07/2019
• Est. Priority Date: 09/21/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Wills, Jonathan, Tse, Daniel, Chik, Desmond
• Primary Examiner(s): Youssef, Menatoallah
• Application Number:
  US15/272,070
• Time in Patent Office: 958 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes

Abstract

A system and/or method configured to determine a sample frame order for analyzing a video. The video may have multiple frames ordered in a sequence from a beginning to an end. A first sample frame order for analyzing the video may be determined. Determining the first sample frame order may include determining an initial frame for a first iteration, and determining secondary frames for a second iteration. Determining the initial frame and the secondary frames may be based on a function of frame position in the sequence of frames. The initial frame may be associated with a first sample position, and the secondary frames may be associated with secondary sample positions in the sample frame order. A first feature of the video may be determined based on an analysis of the frames in the video performed on the frames in the first sample frame order.

Claims

1. A system configured to determine a sample frame order for analyzing a video, the system comprising:

one or more processors configured by machine-readable instructions to;

obtain one or more electronic files that define video content included in the video, the video having multiple frames ordered in a sequence of frames from a beginning to an end;

determine a first sample frame order for analyzing the video based on multiple iterations, wherein determining the first sample frame order includes;

determining, for a first iteration, an initial frame based on a function of frame position in the sequence of frames between the beginning of the video and the end of the video, the initial frame having an initial frame position in the sequence of frames;

associating the initial frame with a first sample position in a first sample frame order that is different from the sequence of frames;

determining, for a second iteration, secondary frames based on a function of frame position in the sequence of frames between the beginning of the video and the initial frame position, and between the initial frame position and the end of the video, wherein the secondary frames include a first secondary frame and a second secondary frame; and

associating the first secondary frame and the second secondary frame with secondary sample positions in the first sample frame order;

analyze the video according to the first sample frame order to determine one or more features of the video;

determine a first feature of the video based on an analysis of frames in the video performed on the frames in the first sample frame order;

identify one or more frames from the multiple frames based on the first feature; and

output the one or more frames identified.

2. The system of claim 1, wherein the initial frame position is a midpoint, or adjacent to the midpoint, of the video such that determining the initial frame based on the function of frame position in the sequence of frames between the beginning of the video and the end of the video includes determining a frame in the sequence of frames that is at the midpoint of the video or adjacent to the midpoint of the video.

3. The system of claim 1, wherein the first secondary frame has a first secondary frame position in the sequence of frames and the second secondary frame has a second secondary frame position in the sequence of frames, wherein the first secondary frame position is located at a secondary midpoint between the beginning of the of the video and the initial frame position or adjacent to the secondary midpoint between the beginning of the of the video and the initial frame position, and the second secondary frame position is located at a secondary midpoint between the initial frame position and the end of the video or adjacent to the secondary midpoint between the initial frame position and the end of the video.

4. The system of claim 1, wherein the first feature includes one or more of a goodness score, a motion metric, a shape metric, a face metric, and/or a persistence metric.

5. The system of claim 1, wherein the multiple iterations include a quantity of iterations beyond the first iteration and the second iteration, and wherein determining the first sample frame order further includes determining, for individual ones of the quantity of iterations, other sets of frames to be associated with other sample positions in the first sample frame order based on a function of frame position in the sequence of frames between a previously determined frame determined during an immediately preceding iteration within the multiple iterations, and one of the beginning of the video, the end of the video, or another previously determined frame determined during another preceding iteration.

6. The system of claim 5, wherein the quantity of iterations indicates a depth of the sample of the video such that less than five iterations indicates a less in-depth sample of the video than five or more iterations.

7. The system of claim 1, wherein the first position in the first sample frame order represents a first depth pass of the video and the secondary positions in the first sample frame order represent a second depth pass of the video.

8. The system of claim 1, wherein the first secondary frame has a first secondary frame position in the sequence of frames and the second secondary frame has a second secondary frame position in the sequence of frames, such that determining the first sample frame order further includes:

determining, for a third iteration, tertiary frames based on a function of frame position in the sequence of frames between the beginning of the video and the first secondary frame position, the first secondary frame position and the initial frame position, the initial frame position and the second secondary frame position, and the second secondary frame position and the end of the video; and

associating the tertiary frames with tertiary sample positions in the first sample frame order.

9. The system of claim 1, wherein the one or more processors are further configured by machine-readable instructions to:

determine an interval between a frame position in the sequence of frames determined for a given iteration and another frame position in the sequence of frames determined for an iteration immediately preceding the given iteration;

determine whether the interval is at least equal to or less than a threshold interval such that the video is analyzed and the first feature is determined responsive to the interval being at least equal to or less than the threshold interval.

10. The system of claim 1, wherein the secondary sample positions come after the first sample position and before any other sample positions in the first sample frame order such that the secondary fames associated with the secondary sample positions are analyzed after the initial frame associated with the first sample position and before any other frames associated with the other sample positions.

11. A method for determining a sample frame order for analyzing a video, the method being implemented in a computer system that includes one or more processors configured by machine-readable instructions, the method comprising:

obtaining one or more electronic files that define video content included in the video, the video having multiple frames ordered in a sequence of frames from a beginning to an end;

determining a first sample frame order for analyzing the video based on multiple iterations, wherein determining the first sample frame order includes;

determining, for a first iteration, an initial frame based on a function of frame position in the sequence of frames between the beginning of the video and the end of the video, the initial frame having an initial frame position in the sequence of frames;

associating the initial frame with a first sample position in a first sample frame order that is different from the sequence of frames;

determining, for a second iteration, secondary frames based on a function of frame position in the sequence of frames between the beginning of the video and the initial frame position, and between the initial frame position and the end of the video, wherein the secondary frames include a first secondary frame and a second secondary frame; and

associating the first secondary frame and the second secondary frame with secondary sample positions in the first sample frame order;

analyzing the video according to the first sample frame order to determine one or more features of the video;

determining a first feature of the video based on an analysis of frames in the video performed on the frames in the first sample frame order;

identifying one or more frames from the multiple frames based on the first feature; and

outputting the one or more frames identified.

12. The method of claim 11, wherein the initial frame position is a midpoint, or adjacent to the midpoint, of the video such that determining the initial frame based on the function of frame position in the sequence of frames between the beginning of the video and the end of the video includes determining a frame in the sequence of frames that is at the midpoint of the video or immediately adjacent to the midpoint of the video.

13. The method of claim 11, wherein the first secondary frame has a first secondary frame position in the sequence of frames and the second secondary frame has a second secondary frame position in the sequence of frames, wherein the first secondary frame position is located at a secondary midpoint between the beginning of the of the video and the initial frame position or adjacent to the secondary midpoint between the beginning of the of the video and the initial frame position, and the second secondary frame position is located at a secondary midpoint between the initial frame position and the end of the video or adjacent to the secondary midpoint between the initial frame position and the end of the video.

14. The method of claim 11, wherein the first feature includes one or more of a goodness score, a motion metric, a shape metric, a face metric, and/or a persistence metric.

15. The method of claim 11, wherein the multiple iterations include a quantity of iterations beyond the first iteration and the second iteration, and wherein determining the first sample frame order further includes determining, for individual ones of the quantity of iterations, other sets of frames to be associated with other sample positions in the first sample frame order based on a function of frame position in the sequence of frames between a previously determined frame determined during an immediately preceding iteration within the multiple iterations, and one of the beginning of the video, the end of the video, or another previously determined frame determined during another preceding iteration.

16. The system of claim 15, wherein the quantity of iterations indicates a depth of the sample of the video such that less than five iterations indicates a less in-depth sample of the video than five or more iterations.

17. The method of claim 11, wherein the first position in the first sample frame order represents a first depth pass of the video and the secondary positions in the first sample frame order represent a second depth pass of the video.

18. The method of claim 11, wherein the first secondary frame has a first secondary frame position in the sequence of frames and the second secondary frame has a second secondary frame position in the sequence of frames, such that determining the first sample frame order further includes:

determining, for a third iteration, tertiary frames based on a function of frame position in the sequence of frames between the beginning of the video and the first secondary frame position, the first secondary frame position and the initial frame position, the initial frame position and the second secondary frame position, and the second secondary frame position and the end of the video; and

associating the tertiary frames with tertiary sample positions in the first sample frame order.

19. The method of claim 11, further comprising:

determining an interval between a frame position in the sequence of frames determined for a given iteration and another frame position in the sequence of frames determined for an iteration immediately preceding the given iteration;

determining whether the interval is at least equal to or less than a threshold interval such that the video is analyzed and the first feature is determined responsive to the interval being at least equal to or less than the threshold interval.

20. The method of claim 11, wherein the secondary sample positions come after the first sample position and before any other sample positions in the first sample frame order such that the secondary fames associated with the secondary sample positions are analyzed after the initial frame associated with the first sample position and before any other frames associated with the other sample positions.