Systems and methods for synchronized display of athletic maneuvers

• US 10,213,137 B2
• Filed: 12/30/2016
• Issued: 02/26/2019
• Est. Priority Date: 03/07/2013
• Status: Active Grant

• Current Assignee: AlpineReplay, Inc.
• Original Assignee: AlpineReplay, Inc.
• Inventors: Lokshin, Anatole M., Lokshin, David J.
• Primary Examiner(s): Barbee, Manuel L
• Application Number:
  US15/395,411
• Publication Number:
  US 20170106238A1
• Time in Patent Office: 788 Days
• CPC Class Codes: A61B 2503/10 Athletes
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/1122 of movement trajectories
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/6804 Garments; Clothes
• CPC Class Codes: A61B 5/6813 Specially adapted to be attached to a specific body part A61B5/6802 takes precedence
• CPC Class Codes: A61B 5/6895 Sport equipment
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: A61B 5/744 Displaying an avatar, e.g. an animated cartoon character
• CPC Class Codes: A63B 2024/0015 Comparing movements or motion sequences with computerised simulations of movements or motion sequences, e.g. for generating an ideal template as reference to be achieved by the user
• CPC Class Codes: A63B 2071/0647 Visualisation of executed movements
• CPC Class Codes: A63B 2220/12 Absolute positions, e.g. by using GPS
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/803 Motion sensors
• CPC Class Codes: A63B 24/0006 Computerised comparison for qualitative assessment of motion sequences or the course of a movement
• CPC Class Codes: A63B 69/00 Training appliances or apparatus for special sports training of parachutists B64D23/00
• CPC Class Codes: A63B 71/0622 Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
• CPC Class Codes: G01C 19/5776 Signal processing not specific to any of the devices covered by groups G01C19/5607 - G01C19/5719
• CPC Class Codes: G01P 15/18 in two or more dimensions
• CPC Class Codes: G01S 19/19 : Sporting applications
• CPC Class Codes: G06Q 10/0639 : Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 13/40 : of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G09B 19/0038 : Sports
• CPC Class Codes: H04W 4/80 : Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Embodiments of the present disclosure help to synchronize multiple video segments of an actor or actors performing athletic maneuvers. Among other things, various embodiments help provide a side-by-side visual comparison of athletic maneuvers.

Claims

1. A computer-implemented method comprising:

receiving, by a computer system, first sensor data related to motion by an actor over a first time period and second sensor data related to motion by an actor over a second time period;

determining, by the computer system and based on the first sensor data and the second sensor data, a plurality of motion characteristics;

identifying, based on the plurality of motion characteristics, an athletic maneuver associated with both the motion by the actor during the first time period and the motion by the actor during the second time period;

synchronizing the first sensor data with a visual display of the athletic maneuver performed during the first time period;

synchronizing the second sensor data with a visual display of the athletic maneuver performed during the second time period;

simultaneously presenting, by the computer system, the visual display of the athletic maneuver performed during the first time period and the visual display of the athletic maneuver performed during the second time period;

wherein identifying the athletic maneuver includes comparing the plurality of determined motion characteristics to one or more motion characteristics associated with a known athletic maneuver; and

wherein the one or more motion characteristics associated with the known athletic maneuver includes one or more of;

an orientation of the actor at a start of the time period;

an angle of rotation around a local axis of the actor;

a direction of rotation of the actor around a local axis of the actor;

an angle of rotation around an absolute axis perpendicular to a plane of motion of the actor;

a direction of rotation round an absolute axis perpendicular to a plane of motion of the actor; and

a leading edge of a piece of sports equipment at a start of the time period.

2. The method of claim 1, wherein the actor performing the athletic maneuver during the first time period and the actor performing the athletic maneuver during the second time period are the same actor.

3. The method of claim 1, wherein the actor performing the athletic maneuver during the first time period is a first human actor and the actor performing the athletic maneuver during the second time period is a second human actor.

4. The method of claim 1, wherein the visual display of the athletic maneuvers performed during the first time period and the second time period include one or more of video and computer animation.

5. The method of claim 4, wherein the visual display of the athletic maneuver performed during the first time period is a video of a human actor performing the athletic maneuver, and the visual display of the athletic maneuver performed during the second time period is a computer-generated animation of an avatar performing the athletic maneuver.

6. The method of claim 1, wherein synchronizing the first sensor data with the visual display of the athletic maneuver performed during the first time period is based on a time measured from one or more of:

a global positioning system, a computer network, and a communication network.

7. The method of claim 1, wherein synchronizing the first sensor data with the visual display of the athletic maneuver performed during the first time period is based on a time determined based on a signal from at least one of a multi-axis gyroscope, a multi-axis accelerometer, and a multi-axis magnetic sensor.

8. The method of claim 1, wherein the plurality of motion characteristics include one or more of:

position;

velocity;

acceleration;

orientation;

rotation; and

translation.

9. The method of claim 1, wherein identifying the athletic maneuver includes determining a level of similarity between the plurality of determined motion characteristics and the motion characteristics associated with the known athletic maneuver.

10. The method of claim 1, wherein simultaneously presenting the visual display of the athletic maneuver performed during the first time period and the visual display of the athletic maneuver performed during the second time period includes presenting a phase of the athletic maneuver performed during the first time period at a slower speed than the phase of the athletic maneuver performed during the second time period.

11. The method of claim 1, wherein the phase of the athletic maneuver performed during the first time period is presented at a speed different from its nominal speed.

12. The method of claim 1, further comprising:

receiving, via a user interface in communication with the computer system, input related to an athletic maneuver definition;

in response to the input, associating the athletic maneuver definition with one or more of the plurality of motion characteristics; and

storing, in a database in communication with the computer system, the athletic maneuver definition and the one or more associated motion characteristics.

13. The method of claim 12, further comprising storing, in the database, one or more of:

the first sensor data;

the second sensor data;

one or more of the plurality of motion characteristics;

the visual display of the athletic maneuver performed during the first time period; and

the visual display of the athletic maneuver performed during the second time period.

14. The method of claim 13, further comprising:

receiving, via the computer system, a request for visual displays of an athletic maneuver, wherein the request defines a characteristic of the athletic maneuver;

retrieving, from the database, a plurality of visual displays of the athletic maneuver that include the characteristic; and

simultaneously displaying, by the computer system, the plurality of retrieved visual displays.

15. The method of claim 1, further comprising overlaying information related to the athletic maneuver on one or more of the visual display of the athletic maneuver performed during the first time period and the athletic maneuver performed during the second time period.

16. The method of claim 15, wherein the overlaid information includes a visual annotation indicating one or more differences between the athletic maneuver performed during the first time period and the athletic maneuver performed during the second time period.

17. A non-transitory, computer-readable medium storing instructions that, when executed, cause a computing device to:

receive first sensor data related to motion by an actor over a first time period and second sensor data related to motion by an actor over a second time period;

determine, based on the first sensor data and the second sensor data, a plurality of motion characteristics;

identify, based on the plurality of motion characteristics, an athletic maneuver associated with both the motion by the actor during the first time period and the motion by the actor during the second time period;

synchronize the first sensor data with a visual display of the athletic maneuver performed during the first time period;

synchronize the second sensor data with a visual display of the athletic maneuver performed during the second time period;

simultaneously present the visual display of the athletic maneuver performed during the first time period and the visual display of the athletic maneuver performed during the second time period;

wherein to identify the athletic maneuver includes comparing the plurality of determined motion characteristics to one or more motion characteristics associated with a known athletic maneuver; and

wherein the one or more motion characteristics associated with the known athletic maneuver includes one or more of;

an orientation of the actor at a start of the time period;

an angle of rotation around a local axis of the actor;

a direction of rotation of the actor around a local axis of the actor;

an angle of rotation around an absolute axis perpendicular to a plane of motion of the actor;

a direction of rotation round an absolute axis perpendicular to a plane of motion of the actor; and

a leading edge of a piece of sports equipment at a start of the time period.

18. A system comprising:

at least one processor; and

memory in communication with the at least one processor and storing instructions that, when executed by the processor, cause the system to;

receive first sensor data related to motion by an actor over a first time period and second sensor data related to motion by an actor over a second time period;

determine, based on the first sensor data and the second sensor data, a plurality of motion characteristics;

identify, based on the plurality of motion characteristics, an athletic maneuver associated with both the motion by the actor during the first time period and the motion by the actor during the second time period;

synchronize the first sensor data with a visual display of the athletic maneuver performed during the first time period;

synchronize the second sensor data with a visual display of the athletic maneuver performed during the second time period;

simultaneously present the visual display of the athletic maneuver performed during the first time period and the visual display of the athletic maneuver performed during the second time period;

wherein to identify the athletic maneuver includes comparing the plurality of determined motion characteristics to one or more motion characteristics associated with a known athletic maneuver; and

wherein the one or more motion characteristics associated with the known athletic maneuver includes one or more of;

an orientation of the actor at a start of the time period;

an angle of rotation around a local axis of the actor;

a direction of rotation of the actor around a local axis of the actor;

an angle of rotation around an absolute axis perpendicular to a plane of motion of the actor;

a direction of rotation round an absolute axis perpendicular to a plane of motion of the actor; and

a leading edge of a piece of sports equipment at a start of the time period.