Intelligent selection of scene transitions

• US 10,068,613 B2
• Filed: 08/26/2016
• Issued: 09/04/2018
• Est. Priority Date: 08/28/2015
• Status: Active Grant

• Current Assignee: Redbrick Technologies, Inc.
• Original Assignee: Animoto, Inc. (Redbrick Technologies, Inc.)
• Inventors: Wharton, Tim, Sharp, Peter, Clifton, Stephen J.
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Smith, Stephen
• Application Number:
  US15/249,276
• Publication Number:
  US 20170062008A1
• Time in Patent Office: 739 Days
• Field of Search: 382284, 386278, 386282, 386280, 715723, 715726
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/109 Font handling; Temporal or kinetic typography
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 5/76 Television signal recording

Abstract

Techniques related to intelligent selection of scene transitions are disclosed. In some example embodiments, a sequence of video segments may be received as input. The video segments may include a first segment that is adjacent to a second segment. Properties of the first segment and the second segment may be detected. The properties may be used to determine one or more categories for one or more transitions for connecting the first segment to the second segment. The one or more categories may be used to determine one or more placements for the one or more transitions. A digital video that includes the one or more transitions may be generated as output.

Claims

1. A method comprising:

receiving, as input, a sequence of storyboard blocks comprising a first storyboard block that is adjacent to a second storyboard block, each storyboard block of the sequence of storyboard blocks being generated from a respective template of a plurality of templates, each template of the plurality of templates specifying at least one type of media element;

detecting one or more types of media elements included in the first storyboard block and the second storyboard block;

based on the one or more types of media elements included in the first storyboard block and the second storyboard block, determining one or more categories for one or more transitions for connecting the first storyboard block to the second storyboard block;

based on the one or more categories for the one or more transitions, determining one or more placements for the one or more transitions;

causing generation of output comprising a digital video that includes the one or more transitions connecting the first storyboard block to the second storyboard block; and

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the one or more types of media elements comprise a text media element, an image media element, or a video media element.

3. The method of claim 1, wherein based on the one or more types of media elements included in the first storyboard block and the second storyboard block, determining the one or more categories for the one or more transitions for connecting the first storyboard block to the second storyboard block comprises:

traversing a set of prioritized rules to identify a highest rule that applies to the one or more types of media elements included in the first storyboard block and the second storyboard block; and

identifying the one or more categories specified by the highest rule.

4. The method of claim 1, wherein the one or more categories indicate that the one or more transitions include a transition controlled by the second storyboard block.

5. The method of claim 1, wherein the one or more categories indicate that the one or more transitions include a transition controlled by the first storyboard block.

6. The method of claim 1, wherein the one or more categories indicate that none of the one or more transitions are controlled by the first storyboard block.

7. The method of claim 1, wherein the one or more categories indicate that none of the one or more transitions are controlled by the second storyboard block.

8. The method of claim 1, wherein based on the one or more categories for the one or more transitions, determining the one or more placements for the one or more transitions comprises:

causing a trailing portion of the first storyboard block to overlap a leading portion of the second storyboard block.

9. The method of claim 1, wherein based on the one or more categories for the one or more transitions, determining the one or more placements for the one or more transitions comprises:

causing a leading portion of the second storyboard block to overlap a trailing portion of the first storyboard block.

10. The method of claim 1, wherein based on the one or more categories for the one or more transitions, determining the one or more placements for the one or more transitions comprises:

causing a trailing portion of the first storyboard block and a leading portion of the second storyboard block to overlap a transition of the one or more transitions.

11. A system comprising:

storage media;

one or more processors; and

one or more programs stored in the storage media and configured for execution by the one or more processors, the one or more programs comprising a sequence of instructions for;

receiving, as input, a sequence of storyboard blocks comprising a first storyboard block that is adjacent to a second storyboard block, each storyboard block of the sequence of storyboard blocks being generated from a respective template of a plurality of templates, each template of the plurality of templates specifying at least one type of media element;

detecting one or more types of media elements included in the first storyboard block and the second storyboard block;

based on the one or more types of media elements included in the first storyboard block and the second storyboard block, determining one or more categories for one or more transitions for connecting the first storyboard block to the second storyboard block;

based on the one or more categories for the one or more transitions, determining one or more placements for the one or more transitions; and

causing generation of output comprising a digital video that includes the one or more transitions connecting the first storyboard block to the second storyboard block.

12. The system of claim 11, wherein the one or more types of media elements comprise a text media element, an image media element, or a video media element.

13. The system of claim 11, wherein based on the one or more types of media elements included in the first storyboard block and the second storyboard block, determining the one or more categories for the one or more transitions for connecting the first storyboard block to the second storyboard block comprises:

traversing a set of prioritized rules to identify a highest rule that applies to the one or more types of media elements included in the first storyboard block and the second storyboard block; and

identifying the one or more categories specified by the highest rule.

14. The system of claim 11, wherein the one or more categories indicate that the one or more transitions include a transition controlled by the second storyboard block.

15. The system of claim 11, wherein the one or more categories indicate that the one or more transitions include a transition controlled by the first storyboard block.

16. The system of claim 11, wherein the one or more categories indicate that none of the one or more transitions are controlled by the first storyboard block.

17. The system of claim 11, wherein the one or more categories indicate that none of the one or more transitions are controlled by the second storyboard block.

18. The system of claim 11, wherein based on the one or more categories for the one or more transitions, determining the one or more placements for the one or more transitions comprises:

causing a trailing portion of the first storyboard block to overlap a leading portion of the second storyboard block.

19. The system of claim 11, wherein based on the one or more categories for the one or more transitions, determining the one or more placements for the one or more transitions comprises:

causing a leading portion of the second storyboard block to overlap a trailing portion of the first storyboard block.

20. The system of claim 11, wherein based on the one or more categories for the one or more transitions, determining the one or more placements for the one or more transitions comprises:

causing a trailing portion of the first storyboard block and a leading portion of the second storyboard block to overlap a transition of the one or more transitions.