Method and System for Automated Personal Training

• US 20160101321A1
• Filed: 12/17/2015
• Published: 04/14/2016
• Est. Priority Date: 11/05/2010
• Status: Active Grant

• Current Assignee: Nike, Inc.
• Original Assignee: Nike, Inc.
• Inventors: Aragones, Teresa, Walling, Willoughby H., Self, Christina S., Jacob, Xavier, Northcott, Geoff
• Granted Patent: US 9,919,186 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63B 2024/0009 Computerised real time comparison with previous movements or motion sequences of the user
• CPC Class Codes: A63B 2024/0012 Comparing movements or motion sequences with a registered reference
• CPC Class Codes: A63B 2024/0015 Comparing movements or motion sequences with computerised simulations of movements or motion sequences, e.g. for generating an ideal template as reference to be achieved by the user
• CPC Class Codes: A63B 2024/0065 Evaluating the fitness, e.g. fitness level or fitness index
• CPC Class Codes: A63B 2024/0068 Comparison to target or threshold, previous performance or not real time comparison to other individuals
• CPC Class Codes: A63B 2024/0096 using performance related parameters for controlling electronic or video games or avatars
• CPC Class Codes: A63B 22/00 Exercising apparatus specially adapted for conditioning the cardio-vascular system, for training agility or co-ordination of movements force-resisting aspects A63B21/00; for particular parts of the body, e.g. to strengthen particular limbs or muscles A63B23/00; electric or electronic controls therefor A63B24/00
• CPC Class Codes: A63B 22/001 by simultaneously exercising arms and legs, e.g. diagonally in anti-phase A63B22/0076 takes precedence
• CPC Class Codes: A63B 2220/51 Force
• CPC Class Codes: A63B 2220/80 Special sensors, transducers or devices therefor
• CPC Class Codes: A63B 24/00 Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
• CPC Class Codes: A63B 24/0003 Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0006 Computerised comparison for qualitative assessment of motion sequences or the course of a movement
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: A63B 24/0075 Means for generating exercise programs or schemes, e.g. computerized virtual trainer, e.g. using expert databases
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising

Abstract

Example embodiments may relate to a system, method, apparatus, and computer readable media configured for monitoring a user performing an exercise and generating a avatar of the user and a virtual shadow, wherein the virtual shadow illustrates proper form of the exercise. The example embodiments may further be configured for determining an amount of overlap between the virtual avatar and the virtual shadow, and generating a feedback score based on the amount of overlap.

Claims

1. A computer-implemented method comprising:

providing instructions to a user to perform an athletic movement;

monitoring with at least a first sensor the user performing the athletic movement;

generating a virtual avatar of the user during the user'"'"'s performance of the athletic movement, the avatar having a first contiguous area shaped like a portion of a first human subject, wherein the first contiguous area comprises at least one sub-area representing a body part of the first human subject;

displaying on a display device the virtual avatar overlaid relative to a virtual shadow, wherein the virtual shadow comprises a second contiguous area that is larger than the first contiguous area, and shaped like the portion of the first human subject, and the second contiguous area comprises at least one sub-area representing the body part of the first human subject, wherein the at least one sub-area of the virtual shadow is larger than but proportional to the at least one sub-area of the virtual avatar, wherein the virtual shadow is configured to illustrate a proper form of the athletic movement, such that if the first user is performing the proper form of the athletic movement, then the first contiguous area is entirely within the second contiguous area;

determining, by a processor, an amount of overlap between the virtual avatar and the virtual shadow; and

generating a feedback score based on, at least in part, the amount of overlap.

2. The method of claim 1, further comprising:

generating the virtual shadow, wherein a size of the virtual shadow is based, at least in part, according to a skill level determined for the user.

3. The method of claim 1, further comprising:

determining a skill level for the user based on an amount of overlap between the virtual avatar and each of a plurality of shadow avatars, wherein each of the shadow avatars is associated with a threshold.

4. The method of claim 1, wherein the monitoring of the user comprises receiving a plurality of images from an image capture device, and wherein the method further comprises:

processing subsections of at least one image from the plurality of images to determine an amount of overlap between the virtual avatar and the virtual shadow within each of the subsections.

5. The method of claim 4, further comprising:

identifying a subsection in which the amount of overlap is less than a predetermined threshold.

6. The method of claim 5, further comprising:

displaying a magnification of the identified subsection.

7. The method of claim 1, further comprising:

communicating the feedback score via a network; and

ranking the user relative to at least one other user who performed the athletic movement.

8. The method of claim 1, further comprising:

determining a virtual target score based on a measuring whether the user moved a body part to coincide with a virtual target, wherein the feedback score is based, at least in part, on the virtual target score.

9. The method of claim 1, further comprising:

determining an angle score based on a comparing an angle between two body parts of the user during performance of the athletic movement and a desired angle between the two body parts during the athletic movement, wherein the feedback score is based on the angle score.

10. The method of claim 1, further comprising:

determining an overlap score based on the amount of overlap between the virtual avatar and the shadow, wherein the feedback score is based on the overlap score.

11. The method of claim 1, wherein the feedback score is based on a weighted sum of an angle score, a virtual target score, and an overlap score.

12. A non-transitory computer readable medium comprising computer-executable instructions that, when executed, perform a method comprising:

providing instructions to a user to perform an athletic movement;

monitoring with at least a first sensor the user performing the athletic movement;

generating a virtual avatar of the user during the user'"'"'s performance of the athletic movement, the avatar having a first contiguous area shaped like a portion of a first human subject, wherein the contiguous area comprises at least one sub-area representing a body part of the first human subject;

generating a virtual shadow comprising a second contiguous area that is larger than the first contiguous area and is sized based, at least in part, according to a skill level of the user for the instructed athletic movement, and shaped like the portion of the first human subject, the second contiguous area comprising at least one sub-area representing the body part of the first human subject, wherein the at least one sub-area of the virtual shadow is larger than but proportional to the at least one sub-area of the virtual avatar, and configured to illustrate a proper form of the user'"'"'s athletic movement such that if the first user performs the proper form of the athletic movement, then the first contiguous area is entirely within the second contiguous area;

displaying on a display device the virtual avatar overlaid relative to the virtual shadow;

determining, by a processor, an amount of overlap between the virtual avatar and the virtual shadow; and

generating a feedback score based on the amount of overlap.

13. The computer readable medium of claim 12, wherein the computer-readable medium further comprises instructions that when executed, perform the method comprising:

determining a skill level for the user based on an amount of overlap between the virtual avatar and each of a plurality of shadow avatars, wherein each of the shadow avatars is associated with a threshold.

14. The computer readable medium of claim 12, wherein the monitoring of the user comprises receiving a plurality of images from an image capture device, and wherein the method further comprises:

processing subsections of at least one image from the plurality of images to determine an amount of overlap between the virtual avatar and the virtual shadow within each of the subsections.

15. The computer readable medium of claim 14, wherein the computer-readable medium further comprises instructions that when executed, perform the method comprising:

identifying a subsection in which the amount of overlap is less than a predetermined threshold; and

displaying a magnification of the identified subsection.

16. The computer readable medium of claim 12, wherein the computer-readable medium further comprises instructions that when executed, perform the method comprising:

determining an angle score based on a comparing an angle between two body parts of the user during performance of the athletic movement and a desired angle between the two body parts during the athletic movement, wherein the feedback score is based on the angle score.

17. An apparatus comprising:

at least one processor; and

at least one non-transitory memory storing instructions that, when executed, cause the apparatus at least to perform;

providing instructions to a user to perform an athletic movement;

monitoring with at least a first sensor the user performing the athletic movement;

generating a virtual avatar of the user during the user'"'"'s performance of the athletic movement, the avatar having a first contiguous area shaped like a portion of a first human subject, wherein the contiguous area comprises at least one sub-area representing a body part of the first human subject;

displaying on a display device the virtual avatar overlaid relative to a virtual shadow, comprising a second contiguous area that is larger than the first contiguous area, and shaped like the portion of the first human subject, and the second contiguous area comprising at least one sub-area representing the body part of the first human subject, wherein the at least one sub-area of the virtual shadow is larger than the at least one sub-area of the virtual avatar, wherein the virtual shadow illustrates a proper form of the athletic movement, such that if the first user performing the proper form of the athletic movement, then the first contiguous area is entirely within the second contiguous area;

determining, by a processor, an amount of overlap between the virtual avatar and the virtual shadow; and

generating a feedback score based on the amount of overlap.

18. The apparatus of claim 17, wherein the instructions, when executed, cause the apparatus to determine a skill level based on an amount of overlap between the virtual avatar and each of a plurality of shadow avatars, wherein each of the shadow avatars is associated with a threshold.

19. The apparatus of claim 17, wherein the monitoring of the user comprises recording of video of the user, and wherein the instructions, when executed, cause the apparatus to:

processing subsections of at least one image from the plurality of images to determine an amount of overlap between the virtual avatar and the virtual shadow within each of the subsections;

identifying a subsection in which the amount of overlap is less than a predetermined threshold; and

displaying a magnification of the identified subsection.

20. The apparatus of claim 17, wherein the instructions, when executed, cause the apparatus to:

determine a virtual;

target score based on a measuring whether the user moved a body part to coincide with a virtual target;

determine an angle score based on a comparing an angle between two body parts of the user and a desired angle between the two body parts; and

determine an overlap score based on determining an amount of overlap between the virtual avatar and the shadow, wherein the feedback score is based on a weighted sum of the angle score, the virtual target score, and the overlap score.