Method and system for interactive fitness training program

• US 8,113,991 B2
• Filed: 02/25/2009
• Issued: 02/14/2012
• Est. Priority Date: 06/02/2008
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Omek Interactive Ltd. (Intel Corporation)
• Inventors: Kutliroff, Gershom
• Primary Examiner(s): Thanh, Loan
• Assistant Examiner(s): GANESAN, SUNDHARA M
• Application Number:
  US12/392,879
• Publication Number:
  US 20090298650A1
• Time in Patent Office: 1,084 Days
• Field of Search: 482/1, 482/8, 482/9, 715/863, 348/46, 348/51, 348/77, 273/444, 600/595, 434/257
• US Class Current: 482/8
• CPC Class Codes: A63B 2024/0012 Comparing movements or motion sequences with a registered reference
• CPC Class Codes: A63B 2024/0071 Distinction between different activities, movements, or kind of sports performed
• CPC Class Codes: A63B 2024/0078 Exercise efforts programmed as a function of time
• CPC Class Codes: A63B 2024/0096 using performance related parameters for controlling electronic or video games or avatars
• CPC Class Codes: A63B 2071/0677 Input by image recognition, e.g. video signals
• CPC Class Codes: A63B 2214/00 Training methods
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 24/0006 Computerised comparison for qualitative assessment of motion sequences or the course of a movement
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: A63B 24/0087 Electric or electronic controls for exercising apparatus of groups A63B21/00 - A63B23/00, e.g. controlling load
• CPC Class Codes: A63B 71/0622 Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
• CPC Class Codes: H04W 68/00 User notification, e.g. alerting and paging, for incoming communication, change of service or the like

Abstract

A system and method of providing an interactive training program by monitoring a user'"'"'s actions is disclosed. A user is prompted through the interactive training program to perform certain movements, the user'"'"'s movements are monitored relative to the prompted movements, and feedback is provided to the user through the program.

Claims

1. A system for providing an interactive training program, comprising:

a sensor configured to capture depth images of user movements without using markers on the user;

a processor configured to;

generate a virtual environment,

generate a prompt for a user to perform a movement,

determine from the depth images three-dimensional coordinates for each of a plurality of feature points on the user'"'"'s body;

interpret user movements, wherein interpreting user movements comprises;

calculating a similarity measure between user movements and catalogued movements in a database using the determined three-dimensional coordinates of the plurality of feature points to identify the user movements, and monitoring user movements using collision boxes in the virtual environment,

analyze the user movements and physiological responses of the user to generate quantitative data describing the user movements, wherein physiological responses include heart rate or respiration rate,

generate a report of user'"'"'s performance based on the analysis and monitoring, and

create a customized program for the user based on the analysis and monitoring or based on a user-specified objective;

a display device configured to show the virtual environment, and the prompt to perform the movement;

a training apparatus configured to measure the physiological responses of the user;

a transmitter configured to transmit the report to a personal trainer;

a first memory configured to store the images of the user movements; and

a second memory configured to store the analysis and reports.

2. The system of claim 1 wherein the display device is configured to show an advertisement from a sponsor while the interactive training program is running.

3. The system of claim 1 wherein the sensor is a depth camera.

4. The system of claim 1 wherein the report includes a comparison of the user'"'"'s performance in a current training session with the user'"'"'s performance during previous sessions or with the user'"'"'s target goals.

5. The system of claim 1, wherein the movement is a physical therapy movement.

6. The system of claim 1, wherein the movement is a fitness movement.

7. The system of claim 1, wherein the processor is further configured to represent the user as an avatar in the virtual environment, wherein performance by the user of the movement progresses the avatar in the virtual environment, and further wherein user movements are substantially mimicked by the avatar.

8. The system of claim 7 wherein interpreting the user movements further comprises tracking the user movements and substantially imposing the user movements on the avatar.

9. The system of claim 7 wherein the processor is further configured to provide feedback to the user by advancing the avatar through the virtual environment when the user successfully performs the movement and moving the avatar backwards through the virtual environment when the user fails to perform the movement.

10. A system for providing an interactive training program, comprising:

a display device for displaying a virtual environment, and a prompt to perform a movement;

means for capturing depth images of user movements, wherein the means for capturing depth images does not use markers on the user;

means for determining from the depth images three-dimensional coordinates for each of a plurality of feature points on the user'"'"'s body;

means for storing movements for detecting and performing gesture recognition on the user movements; and

means for interpreting the user movements based at least upon the stored movements and the determined three-dimensional coordinates of the plurality of feature points.

11. The system of claim 10, wherein the movement is a physical therapy movement.

12. The system of claim 10, wherein the display device further displays an avatar representing the user, wherein performance by the user of the movement progresses the avatar in the virtual environment.

13. A system for providing an interactive training program, comprising:

a sensor configured to capture depth images of user movements without using markers on the user; and

a processor configured to;

generate a virtual environment;

generate a prompt for a user to perform a movement;

determine from the depth images three-dimensional coordinates for each of a plurality of feature points on the user'"'"'s body;

interpret user movements using the determined three-dimensional coordinates of the plurality of feature points;

analyze the user movements to generate quantitative data describing the user movements;

generate a report of user'"'"'s performance based on the analysis of the user'"'"'s movements; and

cause to display the virtual environment, and the prompt to perform the movement.

14. The system of claim 13, wherein the movement is a fitness movement.

15. The system of claim 13, wherein the movement is a sports movement.

16. The system of claim 13, further comprising a database including at least catalogued movements, wherein interpreting the user movements comprises calculating a similarity measure between the user movements and each of the catalogued movements in the database to identify the user movements.

17. The system of claim 13, further comprising a transmitter configured to transmit the report to a personal trainer.

18. The system of claim 13, wherein the processor creates a customized training program for the user based on the analysis of the user movements.

19. The system of claim 13, further comprising a training apparatus configured to measure physiological responses of the user during the interactive training program, wherein the processor is further configured to analyze the physiological responses with the user movements, generate a report of the user'"'"'s performance based on the analysis, and create a customized training program for the user based on the analysis of the physiological responses with the user movements.

20. The system of claim 13, wherein the display is configured to show an advertisement from a sponsor while the interactive training program is running.

21. The system of claim 13 wherein the sensor is a depth camera.

22. The system of claim 13 wherein the report includes a comparison of the user'"'"'s performance in a current session training session with the user'"'"'s performance during previous sessions or with the user'"'"'s target goals.

23. The system of claim 13, wherein the movement is a physical therapy movement.

24. The system of claim 13, wherein the processor is further configured to represent the user as an avatar in the virtual environment, wherein performance by the user of the movement progresses the avatar in the virtual environment, and further wherein user movements are substantially mimicked by the avatar.

25. The system of claim 24, wherein interpreting the user movements comprises tracking the user movements, substantially imposing the user movements on the avatar, and using one or more collision boxes in the virtual environment to monitor the user movements.

26. The system of claim 24 wherein the processor is further configured to provide feedback to the user by advancing the avatar through the virtual environment when the user successfully performs the fitness movement and moving the avatar backwards through the virtual environment when the user fails to perform the fitness movement.

27. A non-transitory computer-readable medium encoded with processing instructions for implementing a method performed by a computer, the method comprising:

generating a virtual environment;

generating a prompt for a user to perform a movement;

determining from a plurality of depth images of the user three-dimensional coordinates for each of a plurality of feature points on the user'"'"'s body;

interpreting user movements using the determined three-dimensional coordinates of the plurality of feature points;

analyzing the user movements to generate quantitative data describing the user movements; and

causing to display the virtual environment and the prompt to perform the movement.