Projected visual cues for guiding physical movement

• US 9,210,401 B2
• Filed: 05/03/2012
• Issued: 12/08/2015
• Est. Priority Date: 05/03/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Wilson, Andrew, Benko, Hrvoje, Sodhi, Rajinder
• Primary Examiner(s): Egloff, Peter
• Assistant Examiner(s): Bullington, Robert P
• Application Number:
  US13/463,585
• Publication Number:
  US 20130295539A1
• Time in Patent Office: 1,314 Days
• Field of Search: 434/247, 434/248, 434/249, 434/250, 434/251, 434/252
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

Physical movement of a human subject may be guided by a visual cue. A physical environment may be observed to identify a current position of a body portion of the human subject. A model path of travel may be obtained for the body portion of the human subject. The visual cue may be projected onto the human subject and/or into a field of view of the human subject. The visual cue may indicate the model path of travel for the body portion of the human subject.

Claims

1. A method of guiding physical movement of a human subject, the method comprising:

observing, via one or more cameras, a physical environment;

identifying, with a computing device configured to process image data from the camera, a current three-dimensional position of a body portion of a human subject within the physical environment;

obtaining, with the computing device, a model path of future travel for the body portion of the human subject; and

projecting, via an image projector, a visual cue onto the human subject at the current three-dimensional position of the body portion, the visual cue indicating the model path of future travel for the body portion of the human subject.

2. The method of claim 1, wherein the visual cue further indicates how the human subject'"'"'s previous physical movements tracked a model path of travel.

3. The method of claim 1, wherein the visual cue indicates the model path of future travel by showing how the human subject'"'"'s future physical movements can mimic the model path of travel.

4. The method of claim 1, wherein the visual cue indicates the model path of future travel by showing a deviation between the current position of the body portion and a model position along the model path of future travel.

5. The method of claim 4, further comprising:

determining one or both of a magnitude and a direction of the deviation between the current position of the body portion and the model position along the model path of future travel;

wherein the visual cue indicates one or both of the magnitude and the direction of the deviation.

6. The method of claim 1, wherein projecting the visual cue onto the human subject includes projecting the visual cue onto the body portion of the human subject.

7. The method of claim 1, wherein the model path of future travel is a multi-dimensional model path of travel.

8. The method of claim 1, wherein the visual cue includes a visual representation of a multi-dimensional directional indicator.

9. The method of claim 1, wherein the visual cue includes a directional indicator that guides the body portion of the human subject toward or along the model path of future travel.

10. The method of claim 1, wherein the visual cue includes one or more colors that indicate the current position of the body portion relative to the model path of future travel.

11. The method of claim 1, wherein the visual cue directs the body portion of the human subject toward the model path of future travel by:

directing the body portion toward a future model position of the model path of travel;

or

directing the body portion along a parallel model path of travel offset from the model path of future travel.

12. The method of claim 1, wherein the visual cue indicates the model path of future travel by following the model path of future travel.

13. The method of claim 1 further comprising:

updating the current position of the body portion of the human subject over time as the body portion moves within the physical environment; and

varying a spatial parameter of projection of the visual cue as the body portion moves within the physical environment to maintain projection of the visual cue within a boundary of the body portion.

14. A computer comprising:

a depth-camera input to receive depth-camera information from a depth camera;

an image-projector output to send projector information to an image projector;

a processor operatively connected to the depth-camera input and to the image-projector output;

a storage device holding instructions executable by the processor to;

identify, from the depth-camera information, a current three-dimensional position of a body portion of a human subject imaged by the depth camera;

obtain a model path of future travel for the body portion of the human subject; and

project a visual cue via the image projector onto the human subject at the current three-dimensional position, the visual cue indicating the model path of future travel for the body portion of the human subject.

15. The storage device of claim 14, wherein the visual cue further indicates how the human subject'"'"'s previous physical movements tracked the model path of travel.

16. The storage device of claim 14, wherein the visual cue indicates the model path of future travel by showing how the human subject'"'"'s future physical movements can mimic the model path of future travel.

17. The storage device of claim 14, wherein the visual cue includes a visual representation of the model path of future travel.

18. The storage device of claim 14, wherein the visual cue includes one or more of a directional indicator indicating a deviation between the current position of the body portion and a model position along the model path of future travel.

19. The storage device of claim 14, wherein the visual cue includes one or more colors indicating a deviation between the current position of the body portion and a model position along the model path of future travel.

20. A method of guiding physical movement of a human subject, the method comprising:

identifying, with a computing device configured to process image data from a camera, a current three-dimensional position of a body portion of a human subject within a physical environment;

obtaining a multi-dimensional model path of future travel for the body portion of the human subject; and

projecting a visual cue onto the body portion at the current three-dimensional position via one or more image projectors, the visual cue including;

a visible light portion that indicates a first direction of movement for the body portion toward or along the model path of future travel in a first plane, and

a directional indicator that indicates a second direction of movement for the body portion toward or along the model path of future travel perpendicular to the first plane.