Automatically generating audiovisual works

• US 8,717,367 B2
• Filed: 05/25/2012
• Issued: 05/06/2014
• Est. Priority Date: 03/02/2007
• Status: Active Grant

• Current Assignee: Redbrick Technologies, Inc.
• Original Assignee: Animoto, Inc. (Redbrick Technologies, Inc.)
• Inventors: Clifton, Stephen J., Hsiao, Jason R., Jefferson, Bradley C., Clifton, Thomas M.
• Primary Examiner(s): Chauhan, Ulka
• Assistant Examiner(s): Valdez, Patrick F
• Application Number:
  US13/481,214
• Publication Number:
  US 20120236005A1
• Time in Patent Office: 711 Days
• Field of Search: None
• US Class Current: 345/473
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04N 2201/3215 of a time or duration
• CPC Class Codes: H04N 2201/3264 of sound signals
• CPC Class Codes: H04N 2201/3266 of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3267 of motion picture signals, e.g. video clip

Abstract

In one embodiment, a method comprises inferentially selecting one or more design animation modules based upon analysis of information obtained from digital visual media items and digital audio media items; and automatically creating an audiovisual work using the selected design animation modules. Audiovisual works can be automatically created based upon inferred and implicit metadata including music genre, image captions, song structure, image focal points, as well as user-supplied data such as text tags, emphasis flags, groupings, and preferred video style.

Claims

1. In a system comprising one or more computing devices, a method for automatically generating an audiovisual work, the method comprising:

inferentially selecting one or more design animation modules from among a plurality of design animation modules to use to generate an audiovisual work based at least upon all of;

(a) one or more first metadata values reflecting one or more detected visual characteristics of at least one of one or more digital visual media items,

(b) one or more second metadata values reflecting one or more detected audio characteristics of at least one of one or more digital audio media items and comprising beat timing information pertaining to the at least one digital audio media item, and

(c) one or more third metadata values associated with at least one of the plurality of design animation modules and comprising beat timing information pertaining to the at least one design animation module;

wherein each design animation module of the plurality of design animation modules is an independent interchangeable unit that can be combined with other design animation modules of the plurality of design animation modules to form different audiovisual works;

wherein the one or more selected design animation modules comprises the at least one design animation module;

wherein inferentially selecting the at least one design animation module comprises comparing the beat timing information of the one or more second metadata values pertaining to the at least one digital audio media item to the beat timing information of the one or more third metadata values pertaining to the at least one design animation module;

wherein the at least one design animation modules comprises a specification of an animation scene;

assigning the at least one digital visual media item to the at least one design animation module including incorporating the at least one digital visual media item into the specification of the animation scene; and

generating the audiovisual work using the one or more selected design animation modules, the one or more digital visual media items, and the one or more digital audio media items;

wherein the method is performed by the one or more computing devices.

2. The method of claim 1:

wherein at least one design animation module of the one or more selected design animation modules comprises at least one visual media item placeholder for eventual replacement by a digital visual media item; and

wherein generating the audiovisual work comprises replacing the at least one digital visual media item placeholder with a digital visual media item of the one or more digital visual media items.

3. The method of claim 1, further comprising:

responsive to successfully generating the audiovisual work, sending a notification to a user that the audiovisual work has been successfully generated; and

wherein the user receives the notification at a computing device of the user.

4. The method of claim 1, further comprising:

responsive to successfully generating the audiovisual work, sending a notification to a user that the audiovisual work has been successfully generated; and

wherein the user receives the notification at a mobile phone of the user.

5. The method of claim 1, further comprising:

responsive to successfully generating the audiovisual work, sending an e-mail notification to a user that the audiovisual work has been successfully generated.

6. The method of claim 1, further comprising:

responsive to successfully generating the audiovisual work, sending a notification to a user that the audiovisual work has been successfully generated; and

wherein the notification comprises a link for downloading the generated audiovisual work.

7. The method of claim 1, further comprising:

prior to inferentially selecting the one or more design animation modules, receiving a specification of an online service from which the one or more digital visual media items can be downloaded; and

downloading the one or more digital visual media items from the online service responsive to receiving the specification.

8. The method of claim 1, further comprising:

prior to inferentially selecting the one or more design animation modules, receiving a specification of an online service from which the one or more digital audio media items can be downloaded; and

downloading the one or more digital audio media items from the online service responsive to receiving the specification.

9. The method of claim 1, further comprising:

prior to inferentially selecting the one or more design animation modules, receiving from a user'"'"'s computing device a request to generate the audiovisual work; and

while generating the audiovisual work, causing a video advertisement to be displayed at the user'"'"'s computing device.

10. The method of claim 1, prior to inferentially selecting the one or more design animation modules, receiving from a user'"'"'s computing device the one or more digital visual media items.

11. The method of claim 1, prior to inferentially selecting the one or more design animation modules, receiving from a user'"'"'s computing device the one or more digital audio media items.

12. The method of claim 1, wherein the one or more detected visual characteristics of the at least one digital visual media item comprises a digital image orientation of the at least one digital visual media item;

and wherein the one or more third metadata values associated with the at least one design animation module indicate a digital image orientation that a digital visual media item placeholder of the at least one design animation module is capable of accepting.

13. The method of claim 1, further comprising re-timing one or more of the one or more selected design animation modules to one or more of the one or more digital audio media items.

14. The method of claim 1, wherein at least one of the one or more digital visual media items is a digital video.

15. The method of claim 1, wherein the one or more detected audio characteristics of the at least one digital audio media item include beat points and song structure attributes of the at least one digital audio media item.

16. The method of claim 1, wherein the one or more detected audio characteristics of the at least one digital audio media item include a ramp-up period, climactic point, verse, chorus, bridge, genre, or style of the at least one digital audio media item.

17. The method of claim 1, wherein the one or more detected visual characteristics of the at least one digital visual media item include a focal point, hue, saturation, orientation, brightness, or contrast of the at least one digital visual media item.

18. The method of claim 1, further comprising:

obtaining one or more text values associated with a collection of the one or more digital visual media items; and

inferentially selecting the one or more design animation modules of the plurality of design animation modules to use to generate the audiovisual work based at least upon all of (a), (b), (c), and the one or more text values.

19. The method of claim 1, further comprising:

obtaining a preferred style for the audiovisual work; and

inferentially selecting the one or more design animation modules of the plurality of design animation modules to use to generate the audiovisual work based at least upon all of (a), (b), (c), and the preferred style.

20. The method of claim 1, further comprising:

obtaining a preferred style for the audiovisual work;

obtaining a genre of the at least one digital audio media item; and

inferentially selecting the one or more design animation modules of the plurality of design animation modules to use to generate the audiovisual work based at least upon all of (a), (b), (c), the preferred style, and the genre.

21. The method of claim 1, wherein the one or more selected design animation modules comprises a plurality of selected design animation modules;

and wherein generating the audiovisual work comprises layering at least two of the plurality of selected design animation modules.

22. The method of claim 1, wherein each of the plurality of design animation modules is generated using design animation software.

23. The method of claim 1,

wherein one or more of the one or more selected design animation modules comprise one or more digital visual media item placeholders;

wherein each placeholder of the one or more digital visual media item placeholders accepts a range of digital visual media items;

wherein the one or more selected design animation modules that comprise the digital visual media item placeholders are associated with one or more particular metadata values, each particular metadata value of the one or more particular metadata values corresponding to one digital visual media item placeholder of the one or more digital visual media item placeholders;

wherein each particular metadata value of the one or more particular metadata values indicates a range of digital visual media items that the corresponding digital visual media item placeholder accepts.

24. The method of claim 23:

wherein each of the one or more digital visual media items is a digital image;

wherein each particular metadata value of the one or more particular metadata values indicates an image orientation of digital visual media items that the corresponding digital visual media item placeholder accepts.

25. The method of claim 1, wherein at least one of the one or more selected design animation modules comprises one or more digital visual media item placeholders that, according to the at least one selected design animation module, are animated in the animation scene.

26. The method of claim 25, wherein generating the audiovisual work comprises replacing the one or more digital visual media item placeholders with one or more of the one or more digital visual media items.

27. The method of claim 1, wherein:

the beat timing information pertaining to the at least one digital audio item comprises one or more beat times and one or more associated weights; and

the beat timing information pertaining to the at least one design animation module specifies a number of beats.

28. The method of claim 1, wherein:

(c) the one or more third metadata values associated with the at least one design animation module comprises acceptable digital visual media item information pertaining to the at least one design animation module indicating one or more visual characteristics of digital visual media items acceptable for the at least one design animation module; and

inferentially selecting the at least one design animation module comprises comparing the acceptable digital visual media item information to one or more of the one or more first metadata values.

29. One or more non-transitory computer-readable media storing instructions which, when executed by one or more computing devices, cause performance of a method for automatically generating an audiovisual work, the method comprising:

inferentially selecting one or more design animation modules from among a plurality of design animation modules to use to generate an audiovisual work based at least upon all of;

(a) one or more first metadata values reflecting one or more detected visual characteristics of at least one of one or more digital visual media items,

(b) one or more second metadata values reflecting one or more detected audio characteristics of at least one of one or more digital audio media items and comprising beat timing information pertaining to the at least one digital audio media item, and

(c) one or more third metadata values associated with at least one of the plurality of design animation modules and comprising beat timing information pertaining to the at least one design animation module;

wherein each design animation module of the plurality of design animation modules is an independent interchangeable unit that can be combined with other design animation modules of the plurality of design animation modules to form different audiovisual works;

wherein the one or more selected design animation modules comprises the at least one design animation module;

wherein the at least one design animation modules comprises a specification of an animation scene;

wherein inferentially selecting the at least one design animation module comprises comparing the beat timing information of the one or more second metadata values pertaining to the at least one digital audio media item to the beat timing information of the one or more third metadata values pertaining to the at least one design animation module;

assigning the at least one digital visual media item to the at least one design animation module including incorporating the at least one digital visual media item into the specification of the animation scene; and

generating the audiovisual work using the one or more selected design animation modules, the one or more digital visual media items, and the one or more digital audio media items.

30. A system for interfacing with an online service for automatically generating audiovisual works, the system comprising:

one or more non-transitory computer-readable media storing downloadable instructions which, when downloaded and executed by a computing device, allow a user of the computing device to provide the online service an indication of one or more digital visual media items, provide the online service an indication of one or more digital audio media items, and request the online service to generate an audiovisual work based on the one or more digital visual items and the one or more digital audio media items;

wherein the online service, responsive to receiving a request from the computing device to generate the audiovisual work, is configured to inferentially select one or more design animation modules from among a plurality of design animation modules to use to generate the audiovisual work based at least upon all of (a) one or more first metadata values reflecting one or more detected visual characteristics of at least one of one or more digital visual media items, (b) one or more second metadata values reflecting one or more detected audio characteristics of at least one of one or more digital audio media items and comprising beat timing information pertaining to the at least one digital audio media item, and (c) one or more third metadata values associated with at least one of the plurality of design animation modules and comprising beat timing information pertaining to the at least one design animation module;

wherein each design animation module of the plurality of design animation modules is an independent interchangeable unit that can be combined with other design animation modules of the plurality of design animation modules to form different audiovisual works;

wherein the one or more selected design animation modules comprises the at least one design animation module;

wherein the at least one design animation modules comprises a specification of an animation scene; and

wherein the online service is further configured to;

inferentially select the at least one design animation module including compare the beat timing information of the one or more second metadata values pertaining to the at least one digital audio media item to the beat timing information of the one or more third metadata values pertaining to the at least one design animation module

assign the at least one digital visual media item to the at least one design animation module including incorporating the at least one digital visual media item into the specification of the animation scene, and

generate the audiovisual work using the one or more selected design animation modules, the one or more digital visual media items, and the one or more digital audio media items.