AUTOMATICALLY GENERATING AUDIOVISUAL WORKS

• US 20080215979A1
• Filed: 03/02/2007
• Published: 09/04/2008
• Est. Priority Date: 03/02/2007
• Status: Active Grant

• Current Assignee: Redbrick Technologies, Inc.
• Original Assignee: Animoto, Inc. (Redbrick Technologies, Inc.)
• Inventors: Jefferson, Bradley C., Clifton, Stephen J., Hsiao, Jason R., Clifton, Thomas M.
• Granted Patent: US 8,347,213 B2
• Time in Patent Office: Days
• US Class Current: 715/716
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04N 2201/3215 of a time or duration
• CPC Class Codes: H04N 2201/3264 of sound signals
• CPC Class Codes: H04N 2201/3266 of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3267 of motion picture signals, e.g. video clip

Abstract

A method comprises receiving one or more digital visual media items; receiving one or more digital audio media items; automatically selecting and arranging one or more design modules based upon analysis of information obtained from the digital visual media items and the digital audio media items; and automatically creating an audiovisual work by arranging the selected and arranged design modules. Audiovisual works can be automatically created based upon inferred and implicit metadata including music genre, image captions, song structure, image focal points, as well as user-supplied data such as text tags, emphasis flags, groupings, and preferred video style.

Claims

1. A computer system, comprising:

logic encoded in one or more storage media for execution and when executed operable to;

receive one or more digital visual media items;

receive one or more digital audio media items;

obtain one or more first metadata values relating to the digital visual media items and the digital audio media items;

inferentially select and arrange one or more design modules, based on the first metadata values;

create an audiovisual work using the selected and arranged design modules.

2. The computer system of claim 1, further comprising a plurality of design modules for portions of an audiovisual work, wherein each design module comprises timing information, and one or more second metadata values of attributes of that design module.

3. The computer system of claim 1, wherein the logic operable to select and arrange comprises logic operable to select and arrange based on one or more second metadata values of attributes of the design modules that are obtained from the design modules.

4. The computer system of claim 1, wherein the logic operable to obtain the first metadata values does not obtain the first metadata values based upon user input, and wherein the logic operable to select one or more of the design modules does not select the one or more design modules based upon user input.

5. The computer system of claim 1, wherein the logic when executed is operable to automatically re-time one or more of the selected design modules to the digital audio media items.

6. The computer system of claim 1, wherein one of the digital visual media items is a digital video file.

7. The computer system of claim 1, wherein the logic when executed is operable to analyze the digital audio media items to identify one or more beat points and song structure attributes of the digital audio media items, and to obtain the first metadata values at least in part based upon any of the beat points or the song structure attributes.

8. The computer system of claim 1, wherein the logic when executed is operable to analyze the digital audio media items to identify any of a ramp-up period, climactic point, verse, chorus, bridge, genre, or style attributes of the digital audio media items, and to obtain the first metadata values at least in part based upon any of the attributes.

9. The computer system of claim 1, wherein the logic when executed is operable to analyze the digital visual media items to identify one or more focal points, image characteristics, or image metadata values, and to obtain the first metadata values at least in part based upon any of the focal points, image characteristics or image metadata values.

10. The computer system of claim 1, wherein the logic when executed is operable to analyze the digital visual media items to identify one or more of a face, subject, hue, saturation, orientation, brightness, contrast, title, or captions and to obtain the first metadata values at least in part based upon any of the face, subject, hue, saturation, orientation brightness, contrast, title, or captions.

11. The computer system of claim 1, wherein the logic when executed is operable to obtain information indicating that one or more specified digital visual media items should be grouped in an audiovisual work, and to create the audiovisual work comprising modules having groupings of the digital visual media items or digital visual media items aligned with structure attributes in the digital audio media items.

12. The computer system of claim 2, wherein the logic when executed is operable to obtain one or more text values associated with a collection of the digital visual media items, and to select the one or more design modules based in part upon a similarity of the one or more text values to the second metadata values.

13. The computer system of claim 2, wherein the logic when executed is operable to obtain a preferred style of the audiovisual work, and to select the one or more design modules based in part upon a similarity of the preferred style to the second metadata values.

14. The computer system of claim 2, wherein the logic when executed is operable to determine a genre of the digital audio media items, to obtain a preferred style of the audiovisual work, and to select the one or more design modules based in part upon a similarity of the second metadata values to all of the genre, preferred style and the first metadata values.

15. The computer system of claim 1, wherein the logic when executed is operable to select one or more of the design modules and to create the audiovisual work wherein the selected design modules are layered.

16. The computer system of claim 1, wherein the logic when executed is operable to determine a starting point in the digital audio media items, and to arrange the selected design modules to the digital audio media items beginning at the specified starting point.

17. A method, comprising:

receiving one or more digital visual media items;

receiving one or more digital audio media items;

obtaining one or more first metadata values relating to the digital visual media items and the digital audio media items;

inferentially selecting and arranging one or more design modules, based on the first metadata values;

creating an audiovisual work using the selected and arranged design modules.

18. The method of claim 17, further comprising accessing a plurality of design modules for portions of an audiovisual work, wherein each design module comprises timing information and one or more second metadata values of attributes of that design module.

19. The method of claim 17, wherein the selecting and arranging comprises selecting and arranging based on one or more second metadata values of attributes of the design modules that are obtained from the design modules.

20. The method of claim 17, wherein obtaining the first metadata values is not based upon user input, and wherein the selecting and arranging the design modules is not based upon user input.

21. The method of claim 17, comprising automatically re-timing one or more of the selected design modules to the digital audio media items.

22. The method of claim 17, wherein one of the digital visual media items is a digital video file.

23. The method of claim 17, comprising analyzing the digital audio media items to identify one or more beat points and song structure attributes of the digital audio media items, and obtaining the first metadata values at least in part based upon any of the beat points or the song structure attributes.

24. The method of claim 17, comprising analyzing the digital audio media items to identify any of a ramp-up period, climactic point, verse, chorus, bridge, genre, or style attributes of the digital audio media items, and obtaining the first metadata values at least in part based upon any of the attributes.

25. The method of claim 17, comprising analyzing the digital visual media items to identify one or more focal points, image characteristics, or image metadata values, and obtaining the first metadata values at least in part based upon any of the focal points, image characteristics or image metadata values.

26. The method of claim 17, comprising analyzing the digital visual media items to identify one or more of a face, subject, hue, saturation, orientation, brightness, contrast, title, or captions and obtaining the first metadata values at least in part based upon any of the face, subject, hue, saturation, orientation brightness, contrast, title, or captions.

27. The method of claim 17, comprising obtaining information indicating that one or more specified digital visual media items should be emphasized or grouped in an audiovisual work, and creating the audiovisual work comprising modules that would allow the digital visual media items or digital visual media item groupings to align with structure attributes in the digital audio media items.

28. The method of claim 18, comprising obtaining one or more text values associated with a collection of the digital visual media items, and selecting the one or more design modules based in part upon a similarity of the one or more text values to the second metadata values.

29. The method of claim 17, comprising obtaining a preferred style of the audiovisual work, and selecting the one or more design modules based in part upon the preferred style.

30. The method of claim 18, comprising determining a genre of the digital audio media items, to obtain a preferred style of the audiovisual work, and selecting the one or more design modules based in part upon a similarity of the second metadata values to all of the genre, preferred style and the first metadata values.

31. The method of claim 17, comprising selecting at least a plurality of the design modules and generating a plurality of layers in the audiovisual work based upon the selected design modules.

32. The method of claim 17, comprising determining a starting point in the digital audio media items, and arranging the selected design modules to the digital audio media items beginning at the specified starting point.

33. A computer-readable storage medium carrying one or more sequences of instructions, which instructions, when executed by one or more processors, cause the one or more processors to perform:

receiving one or more digital visual media items;

receiving one or more digital audio media items;

obtaining one or more first metadata values relating to the digital visual media items and the digital audio media items;

inferentially selecting and arranging one or more design modules, based on the first metadata values;

creating an audiovisual work comprised of the design modules selected and arranged to the digital audio media items.

39. The method of claim 33, comprising providing a preferred style of the audiovisual work, and wherein one or more of the design modules are selected based in part upon the preferred style.

40. The method of claim 33, comprising providing the one or more digital visual media items to the online server from an Internet location.

41. The method of claim 33, comprising providing the one or more digital visual media items by uploading the digital visual media items to the online server from any of a client computer, a mobile phone, or a handheld computer.

34. A method, comprising:

providing one or more digital visual media items to an online server;

providing one or more digital audio media items to the online server;

requesting the online server to generate an audiovisual work based on the digital visual media items and the digital audio media items;

receiving the audiovisual work;

wherein the audiovisual work has been generated based on obtaining one or more first metadata values relating to the digital visual media items and the digital audio media items;

inferentially selecting and arranging one or more design modules, based on the first metadata values; and

creating the audiovisual work using the selected and arranged design modules.

35. The method of claim 34, comprising arranging the digital visual media items in a desired order before the requesting.

36. The method of claim 34, comprising specifying a start point in the digital audio media items at which the audiovisual work should begin.

37. The method of claim 34, comprising obtaining information indicating that one or more specified digital visual media items should be emphasized or grouped in an audiovisual work, and creating the audiovisual work comprising modules that would allow the digital visual media items or digital visual media item groupings to align with structure attributes in the digital audio media items.

38. The method of claim 34, comprising providing one or more text values relating to the digital visual media items, and wherein the one or more design modules are selected based in part upon a similarity of the one or more text values to the second metadata values.

42. A method, comprising:

receiving one or more digital visual media items;

receiving one or more digital audio media items;

obtaining one or more metadata values from any of one or more design modules, the digital visual media items, the digital audio media items, user input, algorithmic extraction, statistical extraction, or an auxiliary data source;

selecting and arranging one or more of the design modules, based in part on the metadata values;

creating an audiovisual work comprised of the design modules selected and arranged to the digital audio media items.

43. The method of claim 42, wherein one of the digital visual media items is a digital video file.

44. The method of claim 42, comprising analyzing the digital audio media items to identify one or more beat points and song structure attributes of the digital audio media items, and obtaining the metadata values at least in part based upon any of the beat points or the song structure attributes.

45. The method of claim 42, comprising analyzing the digital audio media items to identify any of a ramp-up period, climactic point, verse, chorus, bridge, genre, or style attributes of the digital audio media items, and obtaining the metadata values at least in part based upon any of the attributes.

46. The method of claim 42, comprising analyzing the digital visual media items to identify one or more focal points, image characteristics, or image metadata values, and obtaining the metadata values at least in part based upon any of the focal points, image characteristics or image metadata values.

47. The method of claim 42, comprising analyzing the digital visual media items to identify one or more of a face, subject, hue, saturation, orientation, brightness, contrast, title, or captions and obtaining the metadata values at least in part based upon any of the face, subject, hue, saturation, orientation brightness, contrast, title, or captions.

48. The method of claim 42, comprising obtaining information indicating that one or more specified digital visual media items should be emphasized or grouped in an audiovisual work, and creating an audiovisual work comprising modules that would allow the digital visual media items or digital visual media item groupings to align with structure attributes in the digital audio media items.

49. The method of claim 42, comprising obtaining a preferred style of the audiovisual work, and selecting the one or more design modules based in part upon the preferred style.

50. The method of claim 42, comprising determining a genre of the digital audio media items, to obtain a preferred style of the audiovisual work, and selecting the one or more design modules based in part upon a similarity of the metadata values to the genre and preferred style.

51. The method of claim 42, further comprising accessing a plurality of design modules for portions of an audiovisual work, wherein each design module comprises timing information and one or more second metadata values of attributes of that design module.

52. A computer system, comprising:

a storage device comprising a plurality of design modules for portions of an audiovisual work, wherein each design module comprises timing information, and one or more second metadata values of attributes of that design module;

an audiovisual creation system coupled to the storage device and comprising logic encoded in one or more storage media for execution and when executed operable to;

receive and store one or more digital visual media items;

receive and store one or more digital audio media items;

generate one or more first metadata values relating to the digital visual media items and the digital audio media items, and store the first metadata values;

analyze the digital audio media items to identify one or more beat points and song structure attributes of the digital audio media items, and to determine the first metadata values at least in part based upon any of the beat points or the song structure attributes;

analyze the digital visual media items to identify one or more focal points, image characteristics, or image metadata values, and to determine the first metadata values at least in part based upon any of the focal points, image characteristics or image metadata values;

determine an image orientation of the received digital visual media items, to determine a genre of the digital audio media items, obtain a preferred style of the audiovisual work, and select the one or more design modules based upon a similarity of the second metadata values to all of the image orientation, genre, and the first metadata values;

inferentially select and arrange one or more of the design modules based on the first metadata values and the second metadata values;

assign the digital visual media items to the design modules;

create and store an audiovisual work by arranging the selected design modules.

53. A method, comprising:

receiving one or more digital visual media items;

receiving one or more digital audio media items;

automatically selecting and arranging one or more design modules based upon analysis of information relating to the digital visual media items and the digital audio media items;

automatically creating an audiovisual work by arranging the selected and arranged design modules.