Three dimensional user interface session control using depth sensors

• US 9,829,988 B2
• Filed: 08/11/2016
• Issued: 11/28/2017
• Est. Priority Date: 01/14/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Galor, Micha, Pokrass, Jonathan, Hoffnung, Amir
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US15/233,969
• Publication Number:
  US 20160349853A1
• Time in Patent Office: 474 Days
• Field of Search: 345156-158, 715700-866
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A method, including receiving, by a computer executing a non-tactile three dimensional (3D) user interface, a set of multiple 3D coordinates representing a gesture by a hand positioned within a field of view of a sensing device coupled to the computer, the gesture including a first motion in a first direction along a selected axis in space, followed by a second motion in a second direction, opposite to the first direction, along the selected axis. Upon detecting completion of the gesture, the non-tactile 3D user interface is transitioned from a first state to a second state.

Claims

1. A method for interactive control, comprising:

processing, by a computer executing a non-tactile three dimensional (3D) user interface, 3D coordinates of a hand positioned within a field of view of a sensing device coupled to the computer in order to detect gestures made by the hand;

in response to a first gesture detected by the computer, transitioning the 3D user interface from a first state in which the 3D user interface is disengaged to a second state in which the 3D user interface tracks but does not respond to the detected gestures;

in response to a second gesture detected by the computer, subsequent to the first gesture, transitioning the 3D user interface from the second state to a third state in which user control of the 3D user interface is engaged; and

in response to a third gesture detected by the computer, subsequent to the second gesture, accepting and executing a command indicated by the third gesture.

2. The method according to claim 1, wherein the first gesture comprises a first motion made by the hand in a first direction, while the second gesture comprises a second motion made by the hand in a second direction, different from the first direction.

3. The method according to claim 2, wherein the first motion is made in the first direction along a specified axis in space, and the second direction is opposite to the first direction along the specified axis.

4. The method according to claim 1, wherein each of the first and the second gestures comprises a motion of at least 10 centimeters performed at a speed of at least 10 centimeters per second.

5. The method according to claim 1, wherein in the second state the 3D user interface is locked, and wherein the second gesture unlocks the 3D user interface.

6. The method according to claim 5, wherein the second gesture comprises a specified unlock gesture, and wherein the 3D user interface in the second state ignores all gestures by the user other than the specified unlock gesture.

7. The method according to claim 1, wherein the first gesture comprises a focus gesture, which causes the 3D user interface to track and interpret subsequent gestures made by the hand.

8. The method according to claim 1, and comprising conveying a first visual feedback to the user prior to the first gesture, conveying a second visual feedback subsequent to the first gesture, and conveying a third visual feedback subsequent to the second gesture.

9. The method according to claim 1, wherein the first gesture comprises moving the hand from a first predefined region in space to a second predefined region, and wherein the second gesture comprises moving the hand from the second predefined region to a third predefined region.

10. The method according to claim 1, and comprising disengaging the 3D user interface in response to a fourth gesture detected by the computer, subsequent to the second gesture.

11. Apparatus for interactive control, comprising:

a sensing device, configured to output an indication of three dimensional (3D) coordinates of a hand positioned within a field of view of the sensing device; and

a computer configured to execute a non-tactile 3D user interface and to detect, based on the indication output by the sensing device gestures made by the hand, comprising at least first, second and third gestures,

wherein in response to the first gesture, the computer transitions the 3D user interface from a first state in which the 3D user interface is disengaged to a second state in which the 3D user interface tracks but does not respond to the detected gestures, and

in response to the second gesture detected by the computer, subsequent to the first gesture, the computer transitions the 3D user interface from the second state to a third state in which user control of the 3D user interface is engaged, and

in response to the third gesture detected by the computer, subsequent to the second gesture, the computer accepts and executes a command indicated by the third gesture.

12. The apparatus according to claim 11, wherein the first gesture comprises a first motion made by the hand in a first direction, while the second gesture comprises a second motion made by the hand in a second direction, different from the first direction.

13. The apparatus according to claim 11, wherein each of the first and the second gestures comprises a motion of at least 10 centimeters performed at a speed of at least 10 centimeters per second.

14. The apparatus according to claim 11, wherein in the second state the 3D user interface is locked, and wherein the second gesture unlocks the 3D user interface.

15. The apparatus according to claim 11, wherein the computer is configured to disengage the 3D user interface in response to a fourth gesture detected by the computer, subsequent to the second gesture.