Mobile video-based therapy

• US 8,094,873 B2
• Filed: 04/30/2008
• Issued: 01/10/2012
• Est. Priority Date: 04/30/2007
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kelusky, Ronald L., Robinson, Scott
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/112,573
• Publication Number:
  US 20080267447A1
• Time in Patent Office: 1,350 Days
• Field of Search: 382100-305
• US Class Current: 382/100
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61H 2201/0142 Beds
• CPC Class Codes: A61H 2201/501 connected to external computer devices or networks
• CPC Class Codes: A61H 2201/5043 Displays
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2230/62 Posture
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

Mobile video-based therapy, using a portable therapy device that includes a camera, a therapy application database, a processor, and a display. The camera is configured to generate images of a user, and the therapy application database is configured to store therapy applications. The processor is configured to select, from the therapy application database, a therapy application appropriate for assisting in physical or cognitive rehabilitation or therapy of the user, to invoke the therapy application, to recognize a gesture of the user from the generated images, and to control the invoked therapy application based on the recognized gesture. The display is configured to display an output of the controlled therapy application.

Claims

1. A portable therapy device comprising:

a camera configured to generate images of a user;

a therapy application database configured to store therapy applications;

a processor configured to;

select, from the therapy application database, a therapy application appropriate for assisting in physical or cognitive rehabilitation or therapy of the user,

invoke the therapy application,

recognize a gesture of the user from the generated images, and

control the invoked therapy application based on the recognized gesture; and

a display configured to display an output of the controlled therapy application;

wherein the processor is configured to select the therapy application by;

recognizing, from one of the images, whether the user possesses a controller; and

selecting a therapy application that does not require possession of the controller, if the user is recognized to not possess the controller.

2. The portable therapy device of claim 1, wherein the therapy application is selected and controlled without requiring the user to touch the portable therapy device.

3. A computer-implemented method comprising:

capturing images of a user via a camera;

accessing the images of the user;

selecting a therapy application appropriate for assisting in physical or cognitive rehabilitation or therapy of the user;

invoking the therapy application by executing the therapy application at a processor;

recognizing a gesture of the user from the accessed images; and

controlling the invoked therapy application based on the recognized gesture;

wherein selecting the therapy application comprises;

recognizing, from one of the images, whether the user possesses a controller; and

selecting a therapy application that does not require possession of the controller, if the user is recognized to not possess the controller.

4. The method of claim 3, further comprising:

determining a medical condition of the user,

wherein the therapy application is selected based on the determined medical condition.

5. The method of claim 4, wherein selecting the therapy application further comprises:

accessing a matrix correlating therapy applications with medical conditions and therapeutic effects;

inputting the determined medical condition or a desired therapeutic effect remedying the determined medical condition into the matrix;

accessing, as an output of the matrix, a set of therapy applications correlating to the determined medical condition or the desired therapeutic effect; and

selecting the therapy application from the accessed set of therapy applications.

6. The method of claim 4, wherein determining a medical condition of the user further comprises:

determining an identity of the user based on performing facial detection and recognition on one of the accessed images; and

querying a database that correlates users with medical conditions, using the determined identity.

7. The method of claim 4, wherein:

determining the medical condition of the user further comprises detecting that the user has suffered a stroke; and

selecting the therapy application further comprises selecting a hand-eye coordination therapy application.

8. The method of claim 4, wherein determining the medical condition of the user further comprises:

recognizing a medical appliance in one of the images; and

correlating the medical appliance to the medical condition.

9. The method of claim 4, wherein determining the medical condition of the user further comprises:

determining, from one of the images, a position or orientation of the user; and

correlating the position or orientation to the medical condition.

10. The method of claim 4, wherein determining the medical condition of the user further comprises:

receiving, from a medical professional user other than the user, a selection of the medical condition of the user.

11. The method of claim 4, wherein determining the medical condition of the user further comprises:

performing an optical character recognition on a portion of one of the images;

recognizing, from the portion, a word based on performing the optical character recognition; and

matching the recognized word with the medical condition.

12. The method of claim 3, wherein:

invoking the therapy application further comprises;

generating a virtual environment, and

displaying a representation of the user within the virtual environment;

recognizing the gesture further comprises recognizing a magnitude and direction of the gesture; and

controlling the selected therapy application further comprises;

moving the representation within the virtual environment in a magnitude and direction correlating to the determined magnitude and direction of the gesture, and

providing visual or audible feedback to the user based on moving the representation.

13. The method of claim 3, wherein the therapy application comprises a music therapy application, a sports therapy application, or an immersive gaming therapy application.

14. The method of claim 3, wherein:

recognizing the gesture further comprises recognizing wave gesture above a head of the user; and

controlling the invoked therapy application further comprises starting or stopping the invoked therapy application.

15. The method of claim 3, wherein controlling the invoked therapy application further comprises interacting with a virtual object in a virtual world defined by the therapy application.

16. The method of claim 3, wherein controlling the invoked therapy application further comprises enabling a user to play a song or portion of a song using the recognized gesture.

17. The method of claim 3, wherein controlling the invoked therapy application further comprises mapping the recognized gesture to a mouse event.

18. The method of claim 3, wherein selecting the therapy application further comprises:

selecting a therapy application that requires possession of the controller, if the user is recognized to possess the controller.

19. The method of claim 3, wherein selecting the therapy application further comprises:

displaying icons each representing a candidate therapy application, adjacent to an avatar;

recognizing, from the accessed images, a gesture of the user;

controlling the avatar based on the recognized gesture; and

outputting, as the selected therapy application, the candidate therapy application whose icon is affected by the controlled avatar.

20. A tangible computer-readable medium encoded with a computer program, the computer program comprising instructions that, when executed, cause a computer to perform operations comprising:

accessing images of a user;

selecting a therapy application appropriate for assisting in physical or cognitive rehabilitation or therapy of the user;

invoking the therapy application;

recognizing a gesture of the user from the accessed images; and

controlling the invoked therapy application based on the recognized gesture;

wherein selecting the therapy application comprises;

recognizing, from one of the images, whether the user possesses a controller; and

selecting a therapy application that does not require possession of the controller, if the user is recognized to not possess the controller.