Three-dimensional user interface session control

• US 9,035,876 B2
• Filed: 10/17/2013
• Issued: 05/19/2015
• Est. Priority Date: 01/14/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Galor, Micha, Pokrass, Jonathan, Hoffnung, Amir
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US14/055,997
• Publication Number:
  US 20140043230A1
• Time in Patent Office: 579 Days
• Field of Search: 345156-158, 715700-866, 3408154-81592
• US Class Current: 345/156
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A method, including receiving, by a computer executing a non-tactile three dimensional (3D) user interface, a set of multiple 3D coordinates representing a gesture by a hand positioned within a field of view of a sensing device coupled to the computer, the gesture including a first motion in a first direction along a selected axis in space, followed by a second motion in a second direction, opposite to the first direction, along the selected axis. Upon detecting completion of the gesture, the non-tactile 3D user interface is transitioned from a first state to a second state.

Claims

1. A method, comprising:

receiving, by a computer executing a non-tactile three dimensional (3D) user interface, a set of multiple 3D coordinates representing a gesture by a hand positioned within a field of view of a sensing device coupled to the computer, the gesture comprising a first motion in a first direction along a selected axis in space, followed by a second motion in a second direction, opposite to the first direction, along the selected axis; and

transitioning the non-tactile 3D user interface from a first state to a second state upon detecting completion of the gesture.

2. The method according to claim 1, wherein the selected axis is selected from a list consisting of a depth axis and a horizontal axis.

3. The method according to claim 1, wherein the first state comprises a not tracked state and the second state comprises a tracked state.

4. The method according to claim 1, wherein the first state comprises a locked state and the second state comprises an unlocked state.

5. The method according to claim 1, wherein the first state comprises an inactive state and the second state comprises an active state.

6. The method according to claim 1, and comprising conveying a first visual feedback to the user prior to the first motion, conveying a second visual feedback subsequent to the first motion, and conveying a third visual feedback subsequent to the second motion.

7. The method according to claim 6, wherein the first visual feedback, the second visual feedback and the third visual feedback comprise illuminating or darkening one or more visual feedback devices coupled to the computer.

8. The method according to claim 6, wherein the first visual feedback, the second visual feedback and the third visual feedback comprise altering a feedback item presented on a display coupled to the computer.

9. The method according to claim 1, wherein the hand performs each of the first and the second motions for at least a focus gesture distance at a minimum focus gesture speed.

10. The method according to claim 9, wherein the focus gesture distance comprises 10 centimeters, and the minimum focus gesture speed comprises 10 centimeters per second.

11. A method, comprising:

associating, in a computer executing a non-tactile three dimensional (3D) user interface, multiple regions, comprising at least first and second regions, within a field of view of a sensing device coupled to the computer with respective states of the non-tactile 3D user interface, comprising at least first and second states associated respectively with the first and second regions;

conveying visual feedback to a user of the computer on a display having a vertical orientation;

receiving a set of multiple 3D coordinates representing a vertical hand movement from the first region to the second region; and

responsively to the vertical hand movement, transitioning the non-tactile 3D user interface from the first state to the second state.

12. The method according to claim 11, wherein the states of the non-tactile 3D user interface are selected from a list consisting of a tracked and active state, a tracked and inactive state, and a not tracked and inactive state.

13. The method according to claim 11, wherein the visual feedback to the user indicates a current state of the non-tactile 3D user interface.

14. The method according to claim 11, and comprising adjusting boundaries of the multiple regions responsively to recent movements of the hand.

15. An apparatus, comprising:

a three dimensional (3D) optical sensor having a field of view and coupled to a computer executing a non-tactile three dimensional (3D) user interface; and

an illumination element that when illuminated, is configured to be visible to a user when the user is positioned within the field of view of the 3D optical sensor so as to convey visual feedback to the user indicating the user'"'"'s position relative to the field of view.

16. The apparatus according to claim 15, wherein the field of view comprises multiple regions and illumination element is configured to present a current state of the non-tactile 3D user interface to the user positioned in one of the multiple regions.

17. The apparatus according to claim 16, wherein the computer is configured to select the state of the non-tactile 3D user interface from a list consisting of tracked, not tracked, locked, not locked, active and inactive.

18. The apparatus according to claim 17, wherein each of the states is associated with a specific color and the illumination element is configured to present the current state by illuminating in the specific color associated with the current state of the non-tactile 3D user interface.

19. The apparatus according to claim 15, and comprising a conical shaft, wherein the one or more illumination elements are positioned in proximity to an apex of the conical shaft.

20. An apparatus, comprising:

a sensing device; and

a computer executing a non-tactile three dimensional (3D) user interface and configured to receive, from the sensing device, a set of multiple 3D coordinates representing a gesture by a hand positioned within a field of view of the sensing device, the gesture comprising a first motion in a first direction along a selected axis in space, followed by a second motion in a second direction, opposite to the first direction, along the selected axis, and to transition the non-tactile 3D user interface from a first state to a second state upon detecting completion of the gesture.

21. The apparatus according to claim 20, wherein the computer is configured to select the axis is selected from a list consisting of a depth axis and a horizontal axis.

22. The apparatus according to claim 20, wherein the first state comprises a not tracked state and the second state comprises a tracked state.

23. The apparatus according to claim 20, wherein the first state comprises a locked state and the second state comprises an unlocked state.

24. The apparatus according to claim 20, wherein the first state comprises an inactive state and the second state comprises an active state.

25. The apparatus according to claim 20, wherein the computer is configured to convey a first visual feedback to the user prior to the first motion, to convey a second visual feedback subsequent to the first motion, and to convey a third visual feedback subsequent to the second motion.

26. The apparatus according to claim 25, and comprising one or more visual feedback devices, wherein the computer is configured to convey the first visual feedback, the second visual feedback and the third visual feedback by illuminating or darkening the one or more visual feedback devices.

27. The apparatus according to claim 25, and comprising a display, wherein the computer is configured to convey first visual feedback, the second visual feedback and the third visual feedback by altering a feedback item presented on the display.

28. The apparatus according to claim 20, wherein the hand performs each of the first and the second motions for at least a focus gesture distance at a minimum focus gesture speed.

29. The apparatus according to claim 28, wherein the focus gesture distance comprises 10 centimeters, and the minimum focus gesture speed comprises 10 centimeters per second.

30. An apparatus, comprising:

a sensing device;

a display having a vertical orientation; and

a computer coupled to drive the display to convey visual feedback to a user of the computer while executing a non-tactile three dimensional (3D) user interface and configured to associate multiple regions, comprising at least first and second regions, within a field of view of the sensing device with respective states of the non-tactile 3D user interface, comprising at least first and second states associated respectively with the first and second regions, to receiving a set of multiple 3D coordinates representing a vertical hand movement from the first region to the second region, and responsively to the vertical hand movement, to transition the non-tactile 3D user interface from the first state to the second state.

31. The apparatus according to claim 30, wherein the computer is configured to select the states of the non-tactile 3D user interface from a list consisting of a tracked and active state, a tracked and inactive state, and a not tracked and inactive state.

32. The apparatus according to claim 30, wherein the visual feedback to the user indicates a current state of the non-tactile 3D user interface.

33. The apparatus according to claim 30, wherein the computer is configured to adjust boundaries of the multiple regions responsively to recent movements of the hand.

34. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile user interface, cause the computer to receive, from a sensing device, a set of multiple 3D coordinates representing a gesture by a hand positioned within a field of view of the sensing device, the gesture comprising a first motion in a first direction along a selected axis in space, followed by a second motion in a second direction, opposite to the first direction, along the selected axis, and to transition the non-tactile 3D user interface from a first state to a second state upon detecting completion of the gesture.

35. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer coupled to drive a display having a vertical orientation to convey visual feedback to a user of the computer and executing a non-tactile user interface, cause the computer to associate multiple regions, comprising at least first and second regions, within a field of view of a sensing device with respective states of the non-tactile 3D user interface, comprising at least first and second states associated respectively with the first and second regions, to receive a set of multiple 3D coordinates representing a vertical hand movement from the first region to the second region, and responsively to the vertical hand movement, to transition the non-tactile 3D user interface from the first state to the second state.