Systems and methods for creating and enhancing videos

• US 10,008,237 B2
• Filed: 12/24/2014
• Issued: 06/26/2018
• Est. Priority Date: 09/12/2012
• Status: Active Grant

• Current Assignee: AlpineReplay, Inc.
• Original Assignee: AlpineReplay, Inc.
• Inventors: Lokshin, Anatole M., Lokshin, David J.
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Tekle, Daniel
• Application Number:
  US14/583,012
• Publication Number:
  US 20150340066A1
• Time in Patent Office: 1,280 Days
• Field of Search: 348157
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals

Abstract

Embodiments of the present disclosure help to automatically generate video selected from multiple video sources using intelligent sensor processing, thereby providing viewers with a unique and rich viewing experience quickly and inexpensively.

Claims

1. A computer-implemented method comprising:

receiving, by a computer system, first video of an actor from a first source, wherein the first video is captured during a first time period;

receiving, by the computer system, second video of the actor from a second source, wherein the second video is captured during a second time period, and wherein the first time period and second time period at least partially overlap;

receiving sensor data related to the first source during the first time period and sensor data related to the second source during the second time period;

determining, based on the sensor data related to the first source, one or more of;

a trajectory of the first source during the first time period, an orientation of the first source during the first time period, and a zoom factor for the first source during the first time period;

determining, based on the trajectory and the orientation of the first source during the first time period, a viewing direction between the first source and the actor during the first time period;

determining, based on the sensor data related to the second source, one or more of;

a trajectory of the second source during the second time period, an orientation of the second source during the second time period, and a zoom factor for the second source during the first time period;

determining, based on the trajectory and the orientation of the second source during the second time period, a viewing direction between the second source and the actor during the second time period;

receiving, by a computer system, sensor data related to motion by the actor over one or more of the first time period and the second time period;

determining, by the computer system and based on the sensor data, a plurality of motion characteristics, and a position of the actor during the first time period and the second time period;

identifying, based on the plurality of motion characteristics, an athletic maneuver associated with the motion;

creating, by the computer system, a combined video that displays the actor performing the athletic maneuver and includes a frame of video from the first video and a frame of video from the second video based on the viewing direction between the first source and the actor, and the viewing direction between the second source and the actor.

2. The method of claim 1, wherein the common time frame is based on time information received from a network in communication with the first source and the second source.

3. The method of claim 1, wherein the common time frame is based on time information received from a global positioning satellite in communication with the first source and the second source.

4. The method of claim 1, wherein the combined video includes a first frame from the first video source and a second frame from the second video source, wherein the first frame and the second frame are taken at the same synchronized time, and wherein combined video displays the first frame and the second frame simultaneously.

5. The method of claim 1, wherein receiving the sensor data related to the motion by the actor includes:

receiving a first set of sensor data from a first sensor that includes a first set of time tags;

receiving a second set of sensor data from a second sensor that includes a second set of time tags; and

synchronizing the first set of time tags and the second set of time tags based on a common time frame based on time information from one or more of a network in communication with the first sensor and the second sensor, and a global positioning satellite in communication with the first sensor and the second sensor.

6. The method of claim 1, wherein selecting the frame of video includes assigning a first frame quality score to the frame of video from the first video based on the viewing direction between the first source and the actor, and assigning a second frame quality score to the frame of video from the second video on the viewing direction between the second source and the actor.

7. The method of claim 1, wherein creating the combined video includes:

assigning a first frame quality score to a frame of video from the first video based on one or more of;

a distance between the first source and the actor, a viewing direction between the first source and the actor, a type of athletic maneuver being performed by the actor, and a zoom factor for the first source;

assigning a second frame quality score to a frame of video from the second video based on one or more of;

a distance between the second source and the actor, a viewing direction between the second source and the actor, a type of athletic maneuver being performed by the actor, and a zoom factor for the second source; and

selecting the frame from the first video or the frame from the second video based on the first frame quality score and the second frame quality score.

8. The method of claim 7, wherein selecting the frame is further based on whether the first source or the second source is selected to provide one or more previously-selected frames.

9. The method of claim 1, wherein the plurality of motion characteristics are selected from the group consisting of:

position;

velocity;

acceleration;

orientation;

rotation;

translation;

deformation; and

combinations thereof.

10. The method of claim 1, wherein the sensor data includes data from a sensor selected from the group consisting of:

a gyroscope;

an accelerometer;

a magnetometer;

a Hall effect sensor;

a global positioning system;

an ultrasonic sensor;

an optical sensor;

a barometric sensor; and

combinations thereof.

11. The method of claim 1, wherein identifying the athletic maneuver includes comparing the plurality of determined motion characteristics to one or more motion characteristics associated with a known athletic maneuver.

12. The method of claim 11, wherein identifying the athletic maneuver further includes determining a level of similarity between the plurality of determined motion characteristics and the motion characteristics associated with the known athletic maneuver.

13. The method of claim 11, wherein the known athletic maneuver is stored in a database with one or more associated motion characteristics.

14. The method of claim 1, further comprising:

overlaying information related to the athletic maneuver on the combined video of the actor performing the athletic maneuver, wherein the information related to the athletic maneuver is based on the plurality of motion characteristics.

15. The method of claim 14, wherein overlaying the information related to the athletic maneuver includes synchronizing the overlaid information and the video based on a time measured from a common time reference.

16. The method of claim 1, further comprising presenting the combined video to a user via a display screen in communication with the computer system.

17. The method of claim 1, further comprising:

tagging the combined video with information related to the athletic maneuver; and

storing the combined video in a data store, wherein the combined video is searchable via the tag.

18. The method of claim 1, wherein creating the combined video includes modifying, in response to identifying the athletic maneuver, at least a portion of the video displaying the actor performing the athletic maneuver.

19. The method of claim 18, wherein modifying the video includes one or more of:

speeding up the video, or slowing down the video.

20. A non-transitory, computer-readable medium storing instructions that, when executed, cause a computer system to:

receive first video of an actor from a first source, wherein the first video is captured during a first time period;

receive second video of the actor from a second source, wherein the second video is captured during a second time period, the first time period and second time period at least partially overlap, wherein the first video includes a first set of time tags and the second video includes a second set of time tags;

synchronize the first set of time tags and the second set of time tags based on a common time frame;

receive sensor data related to the first source during the first time period and sensor data related to the second source during the second time period;

determine, based on the sensor data related to the first source, one or more of;

a trajectory of the first source during the first time period, an orientation of the first source during the first time period, and a zoom factor for the first source during the first time period;

determine, based on the trajectory and the orientation of the first source during the first time period, a viewing direction between the first source and the actor during the first time period;

determine, based on the sensor data related to the second source, one or more of;

a trajectory of the second source during the second time period, an orientation of the second source during the second time period, and a zoom factor for the second source during the first time period;

determine, based on the trajectory and the orientation of the second source during the second time period, a viewing direction between the second source and the actor during the second time period;

receive sensor data related to motion by the actor over one or more of the first time period and the second time period;

determine, based on the sensor data, a plurality of motion characteristics;

identify, based on the plurality of motion characteristics, an athletic maneuver associated with the motion;

create a combined video that displays the actor performing the athletic maneuver and includes a frame of video from the first video and a frame of video from the second video based on the viewing direction between the first source and the actor, and the viewing direction between the second source and the actor.

21. A computer system comprising:

a processor; and

memory in communication with the processor and storing instructions that, when executed by the processor, cause the computer system to;

receive first video of an actor from a first source, wherein the first video is captured during a first time period;

receive second video of the actor from a second source, wherein the second video is captured during a second time period, the first time period and second time period at least partially overlap, wherein the first video includes a first set of time tags and the second video includes a second set of time tags;

synchronize the first set of time tags and the second set of time tags based on a common time frame;

receive sensor data related to the first source during the first time period and sensor data related to the second source during the second time period;

determine, based on the sensor data related to the first source, one or more of;

a trajectory of the first source during the first time period, an orientation of the first source during the first time period, and a zoom factor for the first source during the first time period;

determine, based on the trajectory and the orientation of the first source during the first time period, a viewing direction between the first source and the actor during the first time period;

determine, based on the sensor data related to the second source, one or more of;

a trajectory of the second source during the second time period, an orientation of the second source during the second time period, and a zoom factor for the second source during the first time period;

determine, based on the trajectory and the orientation of the second source during the second time period, a viewing direction between the second source and the actor during the second time period;

receive sensor data related to motion by the actor over one or more of the first time period and the second time period;

determine, based on the sensor data, a plurality of motion characteristics;

identify, based on the plurality of motion characteristics, an athletic maneuver associated with the motion;

create a combined video that displays the actor performing the athletic maneuver and includes at least one frame of the first video from the first source and at least one frame of the second video from the second source.

22. A computer system comprising:

a processor; and

memory in communication with the processor and storing instructions that, when executed by the processor, cause the computer system to;

retrieve first video of an actor captured by a first source from a database in communication with the computer system, wherein the first video is captured during a first time period;

retrieve second video of the actor captured by a second source from the database, wherein the second video is captured during a second time period, the first time period and second time period at least partially overlap, wherein the first video includes a first set of time tags and the second video includes a second set of time tags;

synchronize the first set of time tags and the second set of time tags based on a common time frame;

retrieve, from the database, sensor data related to motion by the actor over one or more of the first time period and the second time period;

determine, based on the sensor data, a plurality of motion characteristics;

identify, based on the plurality of motion characteristics, an athletic maneuver associated with the motion;

create a combined video that displays the actor performing the athletic maneuver and includes a frame of video from the first video and a frame of video from the second video based on the viewing direction between the first source and the actor, and the viewing direction between the second source and the actor.