Method of and system for automatic compilation of crowdsourced digital media productions

• US 9,418,703 B2
• Filed: 10/08/2014
• Issued: 08/16/2016
• Est. Priority Date: 10/09/2013
• Status: Active Grant

• Current Assignee: Mindset Systems Incorporated
• Original Assignee: Mindset Systems Incorporated
• Inventors: Abbate, Mark Peter, Conway, Michael Francis, Lamm, Robert Earl, Nickel, Steven Philip
• Primary Examiner(s): Shibru, Helen
• Application Number:
  US14/509,120
• Publication Number:
  US 20150098690A1
• Time in Patent Office: 678 Days
• Field of Search: 386/242, 386/248
• US Class Current: 1/1
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 21/27 Server based end-user applications
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

A method of and system for automatic compilation of crowdsourced digital media productions is presented. A system for automated compilation of a digital media program includes interfaces that receive audio/video information, associated temporal information, associated digital device metadata, and media characteristic preference information. The system includes a repository for the digital media input and associated digital device metadata. The system further includes a module that analyzes digital device metadata associated with individual digital media inputs from at least two different recording devices and selects between overlapping portions of the media inputs based on comparing digital device metadata associated with the individual digital media inputs with the media characteristic preference information. The system also includes a digital media editor module that compiles the selected portions of the individual digital media inputs into a digital media program and an output interface that provides access to the digital media program.

Claims

1. A system for analysis of individual digital media inputs and automated compilation of a digital media program, the system comprising:

a digital media interface that receives an individual digital media input, the individual digital media input including at least one of audio and video information and associated temporal information captured by a recording device;

a metadata interface that receives digital device metadata associated with the individual digital media input, the digital metadata including at least (i) information derived from at least one sensor of the recording device that characterizes a physical attribute of the recording device at the time said recording device captured the individual digital media input, and (ii) information temporally associating the sensor data with the individual digital media input;

a source data repository that stores the information of the individual digital media input and associated digital device metadata;

a preference interface that receives media characteristic preference information;

a metadata analyzer module that analyzes digital device metadata associated with individual digital media inputs from at least two different recording devices, the digital media inputs having at least one temporally overlapping portion, and the metadata analyzer module selecting between the overlapping portions based on comparing digital device metadata associated with the individual digital media inputs with the media characteristic preference information;

a digital media editor module that compiles the selected portions of the individual digital media inputs into a digital media program;

an output interface that provides access to the digital media program.

2. The system of claim 1, wherein the information derived from the at least one sensor of the recording device includes at least one of geospatial coordinates of the recording device, an elevation of the recording device, a compass direction of a camera of the recording device, an inclination of a camera of the recording device, a rotation of the recording device about a line of sight, a direction and magnitude of acceleration of the recording device, a focal length of a camera of the recording device, a zoom factor of a camera of the recording device, and lighting levels detected by the recording device.

3. The system of claim 1, the metadata analyzer module further for deriving an estimated location of a subject of the captured video information, based on the information derived from the sensors of the recording device including a geospatial location of the recording device, a line of sight of a camera of the recording device, and an estimated distance to said subject.

4. The system of claim 1, the metadata analyzer module further for deriving digital content metadata based on the individual media input, the digital content metadata including at least one of indications of motion, a scene change, a count of faces present, a count of voices present, a textual conversion of speech, a location of a face, an identification of a face, and an identification of a voice.

5. The system of claim 1, the metadata analyzer module further for deriving digital content metadata based on the individual media input, the digital content metadata including at least one of a starting date/time of the individual media input, a starting timecode value of the individual media input, a duration of the individual media input, a number of frames of the individual media input, a bit rate of the individual media input, a frame rate of the individual media input, a height and width of the individual media input, an encoding profile of the individual media input, and digital format of the individual media input.

6. The system of claim 1, the metadata interface further receiving further metadata from the recording device including at least one of a sensor resolution of a camera of the recording device, an optical focal length of a camera of the recording device, a brand of the recording device, a model identifier of the recording device, a serial number of the recording device, a type or category of the recording device, an operating system identifier of the recording device, and a user identifier of the recording device.

7. The system of claim 1, the media characteristic preference information including information that describes preferences of a particular user of the system.

8. The system of claim 7, the preferences of the particular user of the system being learned based on monitoring interactions between the particular user and the system.

9. The system of claim 1, the media characteristic preference information including information that describes aggregate preferences of a plurality of users of the system.

10. The system of claim 1, further comprising a profile repository that stores relationship information about a plurality of users, the metadata analyzer module selecting portions of the individual digital media inputs from the different recording devices for inclusion in the digital media program based on the relationship information and an identification of which users are associated with the individual digital media inputs.

11. The system of claim 1, further comprising a profile repository that stores information about a plurality of users and an advertisement repository that stores advertisement segments and associated metadata describing the advertisement content, the metadata analyzer module selecting advertisement segments for inclusion in the digital media program based on a designation of a particular user of the plurality and a comparison of the information about the particular user from the profile repository and the metadata associated with the advertisement segments.

12. A computer system-implemented method of analyzing individual digital media inputs and automating compilation of a digital media program, the method comprising:

receiving, at a computer system, an individual digital media input, the individual digital media input including at least one of audio and video information and associated temporal information captured by a recording device;

receiving, at the computer system, digital device metadata associated with the individual digital media input, the digital metadata including at least (i) information derived from at least one sensor of the recording device that characterizes a physical attribute of the recording device at the time said recording device captured the individual digital media input, and (ii) information temporally associating the sensor data with the individual digital media input;

storing in a computer-readable format the information of the individual digital media input and associated digital device metadata;

receiving, at the computer system, media characteristic preference information;

analyzing, at the computer system, digital device metadata associated with individual digital media inputs from at least two different recording devices, the digital media inputs having at least one temporally overlapping portion, and selecting, at the computer system, between the overlapping portions based on comparing digital device metadata associated with the individual digital media inputs with the media characteristic preference information;

compiling, at the computer system, the selected portions of the individual digital media inputs into a digital media program;

providing access, at the computer system, to the digital media program.

13. The method of claim 12, wherein the information derived from the at least one sensor of the recording device includes at least one of geospatial coordinates of the recording device, an elevation of the recording device, a compass direction of a camera of the recording device, an inclination of a camera of the recording device, a rotation of the recording device about a line of sight, a direction and magnitude of acceleration of the recording device, a focal length of a camera of the recording device, a zoom factor of a camera of the recording device, and lighting levels detected by the recording device.

14. The method of claim 12, further comprising deriving, at the computer system, an estimated location of a subject of the captured video information, based on the information derived from the sensors of the recording device including a geospatial location of the recording device, a line of sight of a camera of the recording device, and an estimated distance to said subject.

15. The method of claim 12, further comprising deriving, at the computer system, digital content metadata based on the individual media input, the digital content metadata including at least one of indications of motion, a scene change, a count of faces present, a count of voices present, a textual conversion of speech, a location of a face, an identification of a face, and an identification of a voice.

16. The method of claim 12, further comprising deriving, at the computer system, digital content metadata based on the individual media input, the digital content metadata including at least one of a starting date/time of the individual media input, a starting timecode value of the individual media input, a duration of the individual media input, a number of frames of the individual media input, a bit rate of the individual media input, a frame rate of the individual media input, a height and width of the individual media input, an encoding profile of the individual media input, and digital format of the individual media input.

17. The method of claim 12, further comprising receiving, at the computer system, further metadata from the recording device including at least one of a sensor resolution of a camera of the recording device, an optical focal length of a camera of the recording device, a brand of the recording device, a model identifier of the recording device, a serial number of the recording device, a type or category of the recording device, an operating system identifier of the recording device, and a user identifier of the recording device.

18. The method of claim 12, the media characteristic preference information including information that describes preferences of a particular user of the system.

19. The method of claim 18, further comprising learning, at the computer system, the preferences of the particular user based on monitoring interactions between the particular user and a digital media compilation system.

20. The method of claim 12, the media characteristic preference information including information that describes aggregate preferences of a plurality of users of the system.

21. The method of claim 12, further comprising:

storing in a computer-readable format relationship information about a plurality of users; and

selecting, at the computer system, portions of the individual digital media inputs from the different recording devices for inclusion in the digital media program based on the relationship information and an identification of which users are associated with the individual digital media inputs.

22. The method of claim 12, further comprising:

storing in a computer-readable format information about a plurality of users and an advertisement repository that stores advertisement segments and associated metadata describing the advertisement content; and

selecting, at the computer system, advertisement segments for inclusion in the digital media program based on a designation of a particular user of the plurality and a comparison of the information about the particular user from the profile repository and the metadata associated with the advertisement segments.