Three dimensional user interface cursor control

• US 8,872,762 B2
• Filed: 12/08/2011
• Issued: 10/28/2014
• Est. Priority Date: 12/08/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Galor, Micha, Gelbourt, Idan, Or, Ofir, Pokrass, Jonathan, Hoffnung, Amir
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US13/314,207
• Publication Number:
  US 20120223882A1
• Time in Patent Office: 1,055 Days
• Field of Search: 345156-158, 715700-866
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method, including receiving, by a computer executing a non-tactile three dimensional (3D) user interface, a first set of multiple 3D coordinates representing a gesture performed by a user positioned within a field of view of a sensing device coupled to the computer, the first set of 3D coordinates comprising multiple points in a fixed 3D coordinate system local to the sensing device. The first set of multiple 3D coordinates are transformed to a second set of corresponding multiple 3D coordinates in a subjective 3D coordinate system local to the user.

Claims

1. A method, comprising:

receiving, by a computer executing a non-tactile three dimensional (3D) user interface, a set of 3D coordinates representing a gesture performed by a limb of a user, wherein the limb is positioned within a field of view of a sensing device coupled to the computer and includes a joint, the set of 3D coordinates comprising first multiple points measured in a fixed 3D coordinate system local to the sensing device, the fixed 3D coordinate system having a first depth axis;

upon the set of the 3D coordinates indicating that a motion of the joint has exceeded a specified threshold, transforming the first multiple points that indicate the limb to be moving along the first depth axis to corresponding second multiple points along a different, second depth axis local to the user; and

applying the second multiple points in executing the non-tactile 3D user interface.

2. The method according to claim 1, wherein the limb comprises a hand and the joint comprises an elbow associated with the hand.

3. An apparatus, comprising:

a display; and

a computer executing a non-tactile three dimensional (3D) user interface, and configured to receive a set of 3D coordinates representing a gesture performed by a limb of a user, wherein the limb is positioned within a field of view of a sensing device coupled to the computer and includes a joint, the set of 3D coordinates comprising first multiple points measured in a fixed 3D coordinate system local to the sensing device, the fixed 3D coordinate system having a first depth axis, and upon the set of the 3D coordinates indicating that a motion of the joint has exceeded a specified threshold, to transform the first multiple points that indicate the limb to be moving along the first depth axis to corresponding second multiple points along a different, second depth axis local to the user, and to apply the second multiple points in executing the non-tactile 3D user interface.

4. The apparatus according to claim 3, wherein the limb comprises a hand and the joint comprises an elbow associated with the hand.

5. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile three dimensional user interface, cause the computer to receive a set of 3D coordinates representing a gesture performed by a limb of a user, wherein the limb is positioned within a field of view of a sensing device coupled to the computer and includes a joint, the set of 3D coordinates comprising first multiple points measured in a fixed 3D coordinate system local to the sensing device, the fixed 3D coordinate system having a first depth axis, and upon the set of the 3D coordinates indicating that a motion of the joint has exceeded a specified threshold, to transform the first multiple points that indicate the limb to be moving along the first depth axis to corresponding second multiple points along a different, second depth axis local to the user, and to apply the second multiple points in executing the non-tactile 3D user interface.

6. A method, comprising:

presenting, by a computer executing a non-tactile three dimensional (3D) user interface, a cursor, having a given cursor size, in proximity to one or more items on a display;

receiving in the computer, from a sensing device, a set of 3D coordinates representing a gesture performed by a body part of a user positioned within a field of view of the sensing device;

calculating a ratio between the cursor size and a body part size of the body part that performed the gesture; and

moving the cursor responsively to the received set of the coordinates in proportion to the calculated ratio.

7. The method according to claim 6, wherein the body part comprises a hand.

8. An apparatus, comprising:

a display; and

a computer executing a non-tactile three dimensional (3D) user interface, and configured to present a cursor, having a given cursor size, in proximity to one or more items on the display, to receive in the computer, from a sensing device, a set of 3D coordinates representing a gesture performed by a body part of a user positioned within a field of view of the sensing device, to calculate a ratio between the cursor size and a body part size of the body part that performed the gesture, and to move the cursor responsively to the received set of the coordinates in proportion to the calculated ratio.

9. The apparatus according to claim 8, wherein the body part comprises a hand.

10. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile three dimensional user interface, cause the computer to present a cursor, having a given cursor size, in proximity to one or more items on the display, to receive in the computer, from a sensing device, a set of 3D coordinates representing a gesture performed by a body part of a user positioned within a field of view of the sensing device, to calculate a ratio between the cursor size and a body part size of the body part that performed the gesture, and to move the cursor responsively to the received set of the coordinates in proportion to the calculated ratio.

11. A method, comprising:

presenting, by a computer executing a non-tactile three dimensional (3D) user interface, an interactive cursor shaped as a hand in proximity to one or more items on a display;

receiving in the computer, from a sensing device, a set of 3D coordinates representing a gesture performed by a user positioned within a field of view of the sensing device;

positioning the interactive cursor on the display responsively to the received set of the coordinates; and

conveying feedback to the user, responsively to the received set of the coordinates, indicating a proximity of the cursor to the one or more items, wherein a disposition of one or more fingers of the hand changes responsively to a distance of the cursor from the one of the items.

12. The method according to claim 11, wherein the feedback further comprises a sound.

13. An apparatus, comprising:

a display; and

a computer executing a non-tactile three dimensional (3D) user interface, and configured to present an interactive cursor shaped as a hand in proximity to one or more items on the display, to receive in the computer, from a sensing device, a set of 3D coordinates representing a gesture performed by a user positioned within a field of view of the sensing device, to position the interactive cursor on the display responsively to the received set of the coordinates, and to convey feedback to the user, responsively to the received set of the coordinates, indicating a proximity of the cursor to one of the items, wherein a disposition of one or more fingers of the hand changes responsively to a distance of the cursor from the one of the items.

14. The apparatus according to claim 13, wherein the feedback further comprises a sound.

15. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile three dimensional user interface, cause the computer to present an interactive cursor shaped as a hand in proximity to one or more items on a display, to receive in the computer, from a sensing device, a set of 3D coordinates representing a gesture performed by a user positioned within a field of view of the sensing device, to position the interactive cursor on the display responsively to the received set of the coordinates, and to convey feedback to the user, responsively to the received set of the coordinates, indicating a proximity of the cursor to one of the items, wherein a disposition of one or more fingers of the hand changes responsively to a distance of the cursor from the one of the items.