Systems and methods for identifying interesting moments within videos

• US 10,402,698 B1
• Filed: 07/10/2017
• Issued: 09/03/2019
• Est. Priority Date: 07/10/2017
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Tse, Daniel, Chik, Desmond, Wills, Jonathan
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US15/645,451
• Time in Patent Office: 785 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes

Abstract

Video information defining video content may be accessed. The video content may have a progress length and include a capture of an activity and a sub-activity at one or more moments in the progress length. Portions of the video content may be associated individually with values of an interest metric such that the values of the interest metric as a function of progress through the video content form an interest curve for the video content. The activity and the sub-activity captured within the video content may be identified. An activity metric modifier for the interest curve at the one or more moments in the progress length may be determined based on the identification of the activity and the identification of the sub-activity. The interest curve may be modified at the one or more moments based on the activity metric modifier.

Claims

1. A system for identifying interesting moments within videos, the system comprising:

one or more physical processors configured by machine readable instructions to;

access video information defining video content, the video content having a progress length and including a capture of an activity at one or more moments in the progress length, the capture of the activity including a capture of a sub-activity associated with the activity, wherein portions of the video content are associated individually with values of an interest metric such that the values of the interest metric of the individual portions of the video content as a function of progress through the video content form an interest curve for the video content;

identify the activity captured within the video content;

identify the sub-activity captured within the video content based on the identification of the activity;

determine an activity metric modifier for the interest curve at the one or more moments in the progress length based on the identification of the activity and the identification of the sub-activity; and

modify the interest curve at the one or more moments based on the activity metric modifier.

2. The system of claim 1, wherein:

the activity is identified with an activity accuracy;

the sub-activity is identified with a sub-activity accuracy; and

the activity metric modifier is determined further based on the activity accuracy and the sub-activity accuracy.

3. The system of claim 2, wherein the identification of the sub-activity is characterized by a probability distribution, and the activity metric modifier is determined further based on the probability distribution.

4. The system of claim 3, wherein the probability distribution includes identification of a set of sub-activities associated with the activity and accuracies associated with the identification of individual sub-activities.

5. The system of claim 4, wherein the individual sub-activities are associated with individual modifier values.

6. The system of claim 3, wherein the probability distribution is characterized by a degree of unimodality, and the activity metric modifier is determined further based on the degree of unimodality.

7. The system of claim 2, wherein the one or more physical processors are further configured to:

identify an emotion captured within the video content at the one or more moments in the progress length;

determine an emotion metric modifier for the interest curve at the one or more moments in the progress length based on the emotion; and

modify the interest curve at the one or more moments further based on the emotion metric modifier.

8. The system of claim 7, wherein:

the emotion is identified with an emotion accuracy;

the identification of the emotion is characterized by a probability distribution; and

the emotion metric modifier is determined further based on the emotion accuracy and the probability distribution.

9. The system of claim 7, wherein the video content is captured by an image sensor and the one or more physical processors are further configured to:

determine an orientation of the image sensor when the image sensor captured the activity at the one or more moments in the progress length;

determine an orientation metric modifier for the interest curve at the one or more moments in the progress length based on the orientation; and

modify the interest curve at the one or more moments further based on the orientation metric modifier.

10. A method for identifying interesting moments within videos, the method comprising:

accessing video information defining video content, the video content having a progress length and including a capture of an activity at one or more moments in the progress length, the capture of the activity including a capture of a sub-activity associated with the activity, wherein portions of the video content are associated individually with values of an interest metric such that the values of the interest metric of the individual portions of the video content as a function of progress through the video content form an interest curve for the video content;

identifying the activity captured within the video content;

identifying the sub-activity captured within the video content based on the identification of the activity;

determining an activity metric modifier for the interest curve at the one or more moments in the progress length based on the identification of the activity and the identification of the sub-activity; and

modifying the interest curve at the one or more moments based on the activity metric modifier.

11. The method of claim 10, wherein:

the activity is identified with an activity accuracy;

the sub-activity is identified with a sub-activity accuracy; and

the activity metric modifier is determined further based on the activity accuracy and the sub-activity accuracy.

12. The method of claim 11, wherein the identification of the sub-activity is characterized by a probability distribution, and the activity metric modifier is determined further based on the probability distribution.

13. The method of claim 12, wherein the probability distribution includes identification of a set of sub-activities associated with the activity and accuracies associated with the identification of individual sub-activities.

14. The method of claim 13, wherein the individual sub-activities are associated with individual modifier values.

15. The method of claim 12, wherein the probability distribution is characterized by a degree of unimodality, and the activity metric modifier is determined further based on the degree of unimodality.

16. The method of claim 11, further comprising:

identifying an emotion captured within the video content at the one or more moments in the progress length;

determining an emotion metric modifier for the interest curve at the one or more moments in the progress length based on the emotion; and

modifying the interest curve at the one or more moments further based on the emotion metric modifier.

17. The method of claim 16, wherein:

the emotion is identified with an emotion accuracy;

the identification of the emotion is characterized by a probability distribution; and

the emotion metric modifier is determined further based on the emotion accuracy and the probability distribution.

18. The method of claim 16, wherein the video content is captured by an image sensor, the method further comprising:

determining an orientation of the image sensor when the image sensor captured the activity at the one or more moments in the progress length;

determining an orientation metric modifier for the interest curve at the one or more moments in the progress length based on the orientation; and

modifying the interest curve at the one or more moments further based on the orientation metric modifier.

19. A system for identifying interesting moments within videos, the system comprising:

one or more physical processors configured by machine readable instructions to;

access video information defining video content, the video content having a progress length and including a capture of an activity at one or more moments in the progress length, the capture of the activity including a capture of a sub-activity associated with the activity, wherein portions of the video content are associated individually with values of an interest metric such that the values of the interest metric of the individual portions of the video content as a function of progress through the video content form an interest curve for the video content;

identify the activity captured within the video content, wherein the activity is identified with an activity accuracy;

identify the sub-activity captured within the video content based on the identification of the activity, wherein the sub-activity is identified with a sub-activity accuracy and the identification of the sub-activity is characterized by a first probability distribution having a degree of unimodality;

determine an activity metric modifier for the interest curve at the one or more moments in the progress length based on the identification of the activity and the identification of the sub-activity, wherein the determination of the activity metric modifier based on the identification of the activity and the identification of the sub-activity includes a determination based on the activity accuracy, the sub-activity accuracy, the first probability distribution, and the degree of unimodality; and

modify the interest curve at the one or more moments based on the activity metric modifier.

20. The system of claim 19, wherein the one or more physical processors are further configured to:

identify an emotion captured within the video content at the one or more moments in the progress length;

determine an emotion metric modifier for the interest curve at the one or more moments in the progress length based on the emotion; and

modify the interest curve at the one or more moments further based on the emotion metric modifier.