SYSTEMS AND METHODS FOR CREATING TARGET MOTION, CAPTURING MOTION, ANALYZING MOTION, AND IMPROVING MOTION

• US 20180357472A1
• Filed: 06/04/2018
• Published: 12/13/2018
• Est. Priority Date: 06/07/2017
• Status: Active Grant

• Current Assignee: David Scott Dreessen
• Original Assignee: David Scott Dreessen
• Inventors: Dreessen, David Scott
• Granted Patent: US 11,132,533 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel

Abstract

The present technology enables viewing, analyzing, and improve a variety of motions for a subject by facilitating comparison to a target video. Motions can include activities such as walking, athletic activities, and educational activities. The comparison can be accomplished by adjusting spatial aspects of a target and subject video to match. The comparison can also include identifying synchronization points, between the videos, that identify a temporal correspondence between of motions or objects in the videos and adjusting playback of one or more segments in the videos so corresponding synchronization points are displayed at the same time during playback. A version of one of the videos can be provided as an outline or as partially transparent, which can be overlaid with the other video to create a video combination for playback.

Claims

1. A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations for creating a video combination, the operations comprising:

obtaining a target video by one or more of;

manual selection of the target video;

or

automatic selection of the target video by;

receiving a target category;

identifying one or more first target videos showing an expert with target physical characteristics that correspond to subject physical characteristics of a subject, the subject physical characteristics either specified by a user or determined through automatic analysis of a subject video;

identifying one or more second target videos showing an expert with target motions that correspond to subject motions determined through automatic analysis of the subject video; and

selecting a target video that A) matches the target category, B) is one of the first target videos, and C) is one of the second target videos;

wherein the selected target video has been modified to have at least a background portion made transparent; and

wherein the selected target video is associated with camera positioning data indicating one or more camera locations;

obtaining a subject video by;

providing guidance to position one or more cameras based on the camera positioning data associated with the selected target video; and

receiving the subject video captured using the one or more cameras positioned according to the guidance;

adjusting spatial aspects of the target video or of the subject video such that particular spatial aspects of the target video corresponding to spatial aspects of the subject video, wherein the adjusting the spatial aspects comprises;

identifying a first size of a first identified object, in the subject video, comprising a body part or a piece of equipment; and

adjusting a second size of at least a portion of the target video, including a second identified object corresponding to the first identified object;

to align with the first size;

identifying synchronization points between the target video and the subject video, wherein each synchronization point identifies a temporal correspondence of motions or objects between the target video and the subject video, based on one or more of;

manual selections of one or more of the synchronization points;

automatic selection of points where a location of an identified object in the target video corresponds to a location of a corresponding identified object in the subject video;

automatic selection of points where a motion identified in the target video correspond to a motion identified object in the subject video;

automatic selection of points where a timing of sounds from the target video correspond to timing of sounds from the subject video;

automatic selection of points by a machine learning model that was trained using a set of training videos corresponding to the target category with marked synchronization points;

or

any combination thereof;

adjusting playback of one or more segments of the target video or of the subject video such that at least a first synchronization point in the target video is displayed at the same time as a corresponding synchronization point in the subject video during playback;

modifying the target video to create a version of the target video with at least a portion of the expert displayed as an outline or partially transparent;

creating the video combination to facilitate comparison of the target video with the subject video by overlaying the version of the target video with the subject video; and

facilitating comparison, in the video combination, between the version of the target video and the subject video by performing one or more of;

in response to a viewpoint selection, changing a viewpoint of the target video and changing a viewpoint of the subject view;

automatically identifying that a deviation between the expert in the target video and the subject in the subject video is above a threshold deviation amount, and in response, displaying a deviation indication or playing a sound;

extrapolating motions of the subject to correspond to motions of the expert and, based on the extrapolating, providing an expected performance difference;

identifying aspects of motions of the subject which would benefit from an equipment change and providing a suggestion of the equipment change;

or

any combination thereof.

2. A method for creating a video combination, the method comprising:

obtaining a target video and a subject video;

adjusting spatial aspects of the target video or of the subject video such that particular spatial aspects of the target video correspond to spatial aspects of the subject video;

identifying synchronization points between the target video and the subject video, wherein an individual synchronization point identifies a temporal correspondence of motions or objects between the target video and the subject video;

adjusting playback of one or more segments of the target video or of the subject video such that at least a first synchronization point in the target video is displayed at the same time as a corresponding synchronization point in the subject video during playback; and

creating the video combination to facilitate comparison of the target video with the subject video by either;

overlaying a version of the target video on a version of the subject video;

or

overlaying the version of the subject video on the version of the target video.

3. The method of claim 2 wherein obtaining the subject video comprises:

providing guidance to position one or more cameras based on camera positioning data associated with the target video; and

receiving the subject video captured using the one or more cameras positioned according to the guidance.

4. The method of claim 2 wherein adjusting spatial aspects of the target video or of the subject video comprises:

identifying a first size of a first identified object, in the subject video, comprising a body part or a piece of equipment; and

adjusting a second size of at least a portion of the target video, including a second identified object corresponding to the first identified object, to align with the first size.

5. The method of claim 2 wherein adjusting spatial aspects of the target video or of the subject video comprises adjusting a rotation of the target video to align with the subject video or adjusting a rotation of the subject video to align with the target video.

6. The method of claim 2 wherein identifying synchronization points between the target video and the subject video comprises at least one manual selection of one or more of the synchronization points.

7. The method of claim 2 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point where a location of an identified object in the target video corresponds to a location of a corresponding identified object in the subject video.

8. The method of claim 2 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point where a motion identified in the target video correspond to a motion identified object in the subject video.

9. The method of claim 2 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point where a timing of sounds from the target video correspond to timing of sounds from the subject video.

10. The method of claim 2 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point by a machine learning model that was trained using a set of training videos corresponding to the target category with marked synchronization points.

11. The method of claim 2 wherein the version of the target video is the target video modified to display at least a portion of an expert in the target video as an outline or as partially transparent.

12. The method of claim 2 wherein facilitating the comparison of the target video with the subject video further comprises one or more of:

automatically identifying that a deviation between an expert shown in the target video and a subject shown in the subject video is above a threshold deviation amount, and in response, displaying a deviation indication or playing a sound;

extrapolating motions of the subject to correspond to motions of the expert and, based on the extrapolating, providing an expected performance difference;

or

any combination thereof.

13. The method of claim 2 wherein obtaining the subject video comprises automatic target video selection including:

identifying one or more first target videos showing an expert with target physical characteristics that correspond to subject physical characteristics of a subject, the subject physical characteristics either specified by a user or determined through automatic analysis of the subject video;

wherein the selected target video has been modified to have at least a background portion made transparent.

14. A system for creating a video combination, the system comprising:

one or more processors; and

a memory storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising;

obtaining a target video and a subject video;

identifying synchronization points between the target video and the subject video, wherein an individual synchronization point identifies a temporal correspondence of motions or objects between the target video and the subject video;

adjusting playback of one or more segments of the target video or of the subject video such that at least a first synchronization point in the target video is displayed at the same time as a corresponding synchronization point in the subject video during playback; and

creating the video combination to facilitate comparison of the target video with the subject video by causing concurrent display of a version of the target video with a version of the subject video.

15. The system of claim 14 wherein either or both of the concurrently displayed version of the target video or version of the subject video can be freely repositioned within a display area in response to user input.

16. The system of claim 14 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point where a location of an identified object in the target video corresponds to a location of a corresponding identified object in the subject video.

17. The system of claim 14 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point where a motion identified in the target video correspond to a motion identified object in the subject video.

18. The system of claim 14 wherein identifying synchronization points between the target video and the subject video comprises automatic selection of at least one point where a timing of sounds from the target video correspond to timing of sounds from the subject video.

19. The system of claim 14 wherein the version of the subject video is the subject video modified to display at least a portion of a subject in the subject video as an outline or as partially transparent.

20. The system of claim 14 wherein facilitating the comparison of the target video with the subject video further comprises automatically identifying that a deviation between an expert shown in the target video and a subject shown in the subject video is above a threshold deviation amount.