SYSTEM AND METHOD FOR CLOSE-RANGE MOVEMENT TRACKING

• US 20120327125A1
• Filed: 06/25/2012
• Published: 12/27/2012
• Est. Priority Date: 06/23/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Omek Interactive Ltd. (Intel Corporation)
• Inventors: Kutliroff, Gershom, Yanai, Yaron, Bleiweiss, Amit, Fleishman, Shahar, Livny, Yotam, Epstein, Jonathan
• Granted Patent: US 9,910,498 B2
• Time in Patent Office: Days
• US Class Current: 345/660
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system and method for close range object tracking are described. Close range depth images of a user'"'"'s hands and fingers or other objects are acquired using a depth sensor. Using depth image data obtained from the depth sensor, movements of the user'"'"'s hands and fingers or other objects are identified and tracked, thus permitting the user to interact with an object displayed on a screen, by using the positions and movements of his hands and fingers or other objects.

Claims

1. A method for operating a user interface, the method comprising:

acquiring close range depth images of a part of a user'"'"'s body with a depth sensor;

identifying from the depth images movement within a designated zone of the part of the user'"'"'s body;

tracking the movement within the zone of the part of the user'"'"'s body;

displaying the part of the user'"'"'s body as a first object on a screen, wherein the first object on the screen is shown performing a gesture corresponding to the identified movement of the part of the user'"'"'s body.

2. The method of claim 1, wherein the first object interacts with other objects on the screen.

3. The method of claim 2, wherein the movement generates a force that interacts with the other objects on the screen.

4. The method of claim 1, wherein the part of the user'"'"'s body includes one or more fingers and each of the one or more fingers are represented as one or more separate first objects on the screen, and each of the separate first objects interact with other objects on the screen.

5. The method of claim 1, wherein the part of the user'"'"'s body includes a hand, and further wherein the identified movement of the hand corresponds to a selection gesture, wherein the first object on the screen selects a second object on the screen.

6. The method of claim 1, wherein the part of the user'"'"'s body includes a hand, and further wherein the identified movement of the hand corresponds to a manipulation gesture, wherein the second object on the screen is manipulated according to a predefined action associated with the manipulation gesture.

7. The method of claim 1, further comprising

determining a distance of the part of the user'"'"'s body from the screen;

zooming the screen in and out based on changes in the distance.

8. The method of claim 1, further comprising

determining a distance of the user'"'"'s hand from the screen, wherein the part of the user'"'"'s body includes the hand;

moving a cursor on the screen to indicate a selected object from a stack of objects on the screen, wherein the cursor is moved based upon the distance of the user'"'"'s hand from the screen,

wherein moving the cursor comprises moving the cursor towards a bottom of the stack of objects as the user'"'"'s hand moves closer to the screen and moving the cursor towards a top of the stack of objects as the user'"'"'s hand moves farther from the screen.

9. The method of claim 1, further comprising scaling a size of the first object based on the distance of the part of the user'"'"'s body from the screen.

10. A method comprising:

acquiring depth images of a user'"'"'s hand or hands and fingers with a depth sensor;

tracking movements of the user'"'"'s hand or hands and fingers from the depth images,

wherein the movements correspond to a gesture that performs a function.

11. The method of claim 10, wherein the function is operation of a virtual musical instrument.

12. The method of claim 11, further comprising identifying positions of the user'"'"'s hand or hands and fingers from the depth images to determine a selection of the virtual musical instrument.

13. The method of claim 11, further comprising providing sounds corresponding to notes played by the movements of the user'"'"'s hand or hands and fingers on the virtual musical instrument.

14. The method of claim 11, further comprising providing a visual display of operation of the virtual musical instrument on the screen.

15. The method of claim 10, further comprising performing the function on an electronic device.

16. A method of translating gestures made as part of a gesture-based language, comprising:

acquiring depth images of a user'"'"'s hands and fingers with a depth sensor;

identifying a gesture made by the user'"'"'s hands and fingers from the depth images as a pre-defined gesture of the gesture-based language;

providing a translation of the pre-identified gesture as an output.

17. The method of claim 16, wherein the output includes an audio output or a text output on the screen.

18. A system comprising:

a depth sensor configured to acquire at close range depth images of at least a part of a user'"'"'s body;

a tracking module configured to identify the at least the part of the user'"'"'s body from the depth image data and track a movement of the at least the part of the user'"'"'s body from the depth image data;

an output module configured to permit the movement of the at least the part of the user'"'"'s body to interact with one or more user interface elements;

a display configured to show a representation of at least part of the user'"'"'s body and the one or more user interface elements.

19. The system of claim 18, wherein the tracking module executes a background segmentation algorithm, a feature detection algorithm, and an object identification algorithm.

20. The system of claim 18, wherein the tracking module further executes a skeleton model construction algorithm.

21. A system, comprising:

means for acquiring close range depth images of a part of a user'"'"'s body with a depth sensor;

means for identifying from the depth images movement within a designated zone of the part of the user'"'"'s body;

means for tracking the movement within the zone of the part of the user'"'"'s body;

means for displaying the part of the user'"'"'s body as a first object on a screen, wherein the first object on the screen is shown performing a gesture corresponding to the identified movement of the part of the user'"'"'s body.