EXAMPLE BASED VIDEO EDITING

• US 20130132844A1
• Filed: 01/17/2013
• Published: 05/23/2013
• Est. Priority Date: 11/17/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Oakley, Randolph Bruce, Patten, Michael J., Kutruff, Andrew, Randall, Russell Scott
• Granted Patent: US 9,880,693 B2
• Time in Patent Office: Days
• US Class Current: 715/723
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

Enabling creation of a user-specific storyboard style based on a storyboard edited by a user. Metadata defined by analyzing the editing decisions made by the user and the media content associated with the storyboard represents the user-specific style. The user-specific style may be applied to other media content to produce a fully edited storyboard that matches the style of the example storyboard from which the user-specific style was created. The produced storyboard is combined with the other media content to produce audio/video data for consumption by the user.

Claims

1. :

A method for use with storyboards, each of which represents a plurality of cells arranged according to a timeline, said method comprising;

presenting a first storyboard to a user, wherein the first storyboard represents a plurality of cells arranged according to a timeline;

receiving first media content items from the user for association with one or more of a first plurality of cells in the first storyboard, at least a first cell of the first plurality of cells receiving a nested plurality of storyboards different from the first storyboard;

receiving editing decisions from the user for editing the first media content items in the first storyboard;

analyzing at least one of color histogram information and motion vector information of the first storyboard based on the received editing decisions;

based on said analysis, determining one or more changes to each of the first media content items;

based on the received editing decisions and the determined changes, defining a user-specific style associated with each of the first plurality of cells of the first storyboard;

said user specific style further defining a rendering order of the nested plurality of storyboards, said user specific style further comprising nested user-specific styles, each of the nested user-specific styles corresponding to one of the nested plurality of storyboards;

analyzing the edited first media content items and the first storyboard to define, without any further user input, metadata representing the user-specific style, said metadata defining one or more cell properties associated with each of the first plurality of cells in the first storyboard, said metadata further defining storyboard properties describing characteristics of the first storyboard, wherein said storyboard properties are different from said one or more cell properties;

applying the user-specific style to second media content items provided by the user to create a second storyboard having a second plurality of cells, said second plurality of cells conforming to the cell properties defined by the metadata associated with the user-specific style associated with each of the first plurality of cells of the first storyboard, said applying further comprising selecting, based on the cell properties, a subset of less than all of the second media content items for association with the second plurality of cells;

combining the created second storyboard with the second media content items to produce output data including at least one of output audio data and output video data;

displaying the output data to the user.

2. :

The method of claim 1, further comprising storing the output data on a computer-readable medium.

3. :

The method of claim 1, wherein the second storyboard identifies a sequence of the second media content items within each of the second plurality of cells.

4. :

The method of claim 1, wherein receiving the first media content items comprises receiving, via a drag and drop model in a user interface, the first media content items into the first plurality of cells in the first storyboard.

5. :

The method of claim 1, wherein receiving the first media content items comprises receiving, via a cut and paste model in a user interface, the first media content items into the first plurality of cells in the first storyboard.

6. :

The method of claim 1, further comprising receiving from the user a user-defined cell property for association with one of the first plurality of cells in the first storyboard.

7. :

The method of claim 1 wherein said user specific style specifies a cell property for each cell and further comprising overriding a defined value for the cell property with a defined value for at least one of the storyboard properties when said defined value for at least one of the storyboard properties conflicts with said defined value for the cell property.

8. :

The method of claim 7 wherein, said user specific style specifies at least the following cell properties for each cell;

a start transition effect;

an end transition effect;

a priority property for determining, when said end transition effect of said each cell overlaps with a start transition effect of a next cell of the first storyboard, which transition is used; and

whether the user can override said end transition effect; and

when the user is permitted to override the end transition effect, receiving from the user changes to the end transition effect and applying the changes to the end transition effect.

9. :

The method of claim 1, wherein one or more computer-readable media have computer-executable instructions for performing the method of claim 1.

10. :

A system comprising;

a memory area for storing a plurality of storyboard styles, said storyboard styles each having a plurality of cells for association with one or more media content items, said cells each having one or more cell properties describing media content for association therewith, said storyboard styles further each having one or more storyboard properties describing characteristics of the storyboard style; and

a processor configured to execute computer-executable instructions for;

receiving from a user a selection of one of the plurality of storyboard styles stored in the memory area;

presenting the selected storyboard style to the user;

receiving first media content from the user for association with the plurality of cells in the presented storyboard style to create a first storyboard;

receiving editing decisions from the user for application to the first storyboard;

modifying the presented storyboard style with the received editing decisions;

analyzing at least one of color histogram information and motion vector information of the modified storyboard;

based on said analysis, comparing media content of each cell in the first storyboard to media content in a corresponding cell in the modified storyboard;

determining, based on results of said comparing, rules implementing predefined techniques for analyzing audio and video corresponding to said editing decisions of the user;

receiving second media content from the user for association with a second storyboard;

applying the determined rules to the received second media content to create the second storyboard, said applying further comprising selecting, based on the determined rules, a subset of less than all of the second media content for creating the second storyboard;

presenting the second storyboard to the user as output data including at least one of audio data and video data.

11. :

The system of claim 10, wherein each of the plurality of storyboard styles stored in the memory area correspond to one or more of the following;

an event type, a style, an occasion, and a theme.

12. :

The system of claim 10, wherein the processor is configured to execute computer-executable instructions for receiving from the user a user-defined storyboard property for association with at least one of the storyboard styles.

13. :

The system of claim 10, wherein the processor is configured to execute computer-executable instructions for analyzing the second media content based on the cell properties.

14. :

The system of claim 10, further comprising said processor configured to execute computer-executable instructions for defining a storyboard style having a plurality of cells each for association with one or more media content items and for creating a storyboard style based on the received editing decisions.

15. :

The system of claim 10, wherein said user specific style specifies a cell property for each cell and further comprising overriding a defined value for the cell property with a defined value for at least one of the storyboard properties when said defined value for at least one of the storyboard properties conflicts with said defined value for the cell property.

16. :

The system of claim 15, further comprising, said determining comprising determining at least the following cell properties for each cell;

a start transition effect;

an end transition effect;

a priority property for determining, when said end transition effect of said each cell overlaps with a start transition effect of a next cell of the first storyboard, which transition effect is used; and

whether the user can override said end transition effect; and

when the cell properties permit the user to override the end transition effect, receiving from the user changes to the end transition effect and applying the changes to the end transition effect

17. :

One or more computer-readable storage media having computer-executable components, said components comprising;

a storyboard module for defining a first storyboard having a first plurality of cells each for association with one or more first media content items, at least a first cell of the first plurality of cells associated with a nested tree of elements, said elements comprising a plurality of storyboards different from the first storyboard, wherein the first storyboard and each of the plurality of storyboards comprises a plurality of cells in a linear timeline;

a user interface module for receiving the first media content items from a user for association with the first plurality of cells in the first storyboard, said user interface module further receiving editing decisions from the user for application to the first storyboard;

a style module for;

analyzing at least one of color histogram information and motion vector information of the first storyboard based on the received editing decisions;

based on said analysis, determining one or more changes to each of the first media content items;

analyzing the editing decisions, the determined changes, and the first storyboard to define metadata representing a user-specific style associated with the first storyboard, said metadata defining one or more cell properties describing the first plurality of cells in the first storyboard, said one or more cell properties comprising, for each cell;

a start transition;

an end transition;

a priority property for determining, when said end transition of said each cell overlaps with a start transition of a next cell of the first storyboard, which transition is used;

a population module for applying the user-specific style to second media content items provided by the user to populate a second plurality of cells in a second storyboard with the second media content items, said populated second plurality of cells conforming to the cell properties defined by the metadata associated with the user-specific style, said applying further comprising selecting, based on the cell properties, a subset of the second media content items for association with the second plurality of cells; and

a rendering module for combining the second storyboard from the population module with the second media content items to produce output data including at least one of output audio data and output video data;

wherein the user interface module presents the output data produced by the rendering module to the user, wherein the user interface module allows the user to edit the output data produced by the rendering module to the user prior to rendering the produced output data, and wherein the one or more cell properties defined by the style module further comprises a cell property specifying whether the user can edit the end transition effect.

18. :

The computer-readable storage media of claim 17, wherein the user interface module displays, to the user, the output data produced by the rendering module.

19. :

The computer-readable storage media of claim 17, wherein said metadata further defines storyboard properties describing characteristics of the first storyboard, said storyboard properties different from said one or more cell properties, said style module overriding a defined value for a corresponding one of the cell properties with a defined value for at least one of the storyboard properties when said defined value for at least one of the storyboard properties conflicts with said defined value for the corresponding one of the cell properties, said analyzing further comprising analyzing at least one of color histogram information and motion vector information of each cell of the first storyboard by comparing media content items in said each cell before and after the editing decisions have been applied to said each cell to determine rules implementing predefined techniques for analyzing audio and video corresponding to said editing decisions, said metadata including said determined rules, said storyboard properties defined by said metadata including a rendering order for all elements of the nested tree and including nested user-specific styles, each nested user-specific style corresponding to one of the elements of the nested tree

20. :

The computer-readable storage media of claim 17, wherein the rendering module combines the first storyboard with the first media content items to produce the output data.