Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface

• US 9,030,498 B2
• Filed: 08/14/2012
• Issued: 05/12/2015
• Est. Priority Date: 08/15/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Galor, Micha, Pokrass, Jonathan, Hoffnung, Amir, Or, Ofir
• Primary Examiner(s): Haley, Joseph
• Assistant Examiner(s): ILUYOMADE, IFEDAYO B
• Application Number:
  US13/584,831
• Publication Number:
  US 20130044053A1
• Time in Patent Office: 1,001 Days
• Field of Search: None
• US Class Current: 345/653
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method including presenting, by a computer, multiple interactive items on a display coupled to the computer, and receiving, from a depth sensor, a sequence of three-dimensional (3D) maps containing at least a hand of a user of the computer. An explicit select gesture performed by the user toward one of the interactive items is detected in the maps, and the one of the interactive items is selected responsively to the explicit select gesture. Subsequent to selecting the one of the interactive items, a TimeClick functionality is actuated for subsequent interactive item selections to be made by the user.

Claims

1. A method, comprising:

presenting, by a computer, multiple interactive items on a display coupled to the computer;

receiving, from a depth sensor, a sequence of three-dimensional (3D) maps containing at least a hand of a user of the computer at a variable distance from the depth sensor;

detecting, in the maps, an explicit select gesture performed by the user toward one of the interactive items;

selecting the one of the interactive items responsively to the explicit select gesture;

actuating, subsequent to selecting the one of the interactive items, a TimeClick functionality for subsequent interactive item selections to be made by the user; and

detecting, after actuating the TimeClick functionality, a TimeClick gesture directed at one of the subsequent interactive items, wherein the TimeClick gesture comprises the user keeping the hand relatively steady for a specific period of time, and wherein the specific time period is adjusted in response to the distance between the user and the depth sensor.

2. The method according to claim 1, and comprising presenting a cursor on the display, the cursor having a visual effect, and decreasing the visual effect upon detecting the user keeping the hand relatively steady for the specific period of time.

3. The method according to claim 1, and comprising adjusting the specific time period in response to a proficiency of the user.

4. The method according to claim 1, wherein the multiple interactive items comprise alphanumeric keys for an on-screen keyboard presented on the display, and comprising adjusting the specific time period in response to a predictive text algorithm.

5. The method according to claim 1, wherein one of the multiple interactive items comprises a TimeClick timer icon, and comprising adjusting the specific time period upon detecting, in the subsequent sequence of 3D maps a subsequent select gesture performed by the user toward the TimeClick timer icon.

6. The method according to claim 5, wherein the subsequent select gesture is selected from a list comprising the explicit select gesture and the TimeClick gesture.

7. The method according to claim 1, wherein one of the multiple interactive icons comprises a mode icon, and comprising actuating the TimeClick functionality upon detecting, in the sequence of 3D maps, the explicit select gesture toward the mode icon.

8. The method according to claim 1, wherein the explicit select gesture is selected from a list comprising a Push gesture, a Pull gesture, and a Grab gesture.

9. The method according to claim 1, and comprising prior to detecting the select gesture, defining coordinates for an interactive surface positioned in space in proximity to the user, and wherein completing the explicit select gesture comprises the hand crossing the interactive surface.

10. The method according to claim 9, wherein actuating the TimeClick functionality comprises redefining the coordinates of the interactive surface.

11. An apparatus, comprising:

a depth sensor; and

a computer executing a non-tactile three dimensional (3D) user interface and configured to present, multiple interactive items on a display coupled to the computer, to receive, from the depth sensor, a sequence of three-dimensional (3D) maps containing at least a hand of a user of the computer at a variable distance from the depth sensor, to detect, in the maps, an explicit select gesture performed by the user toward one of the interactive items, to select the one of the interactive items responsively to the explicit select gesture, and to actuate, subsequent to selecting the one of the interactive items, a TimeClick functionality for subsequent interactive item selections to be made by the user,

wherein the computer is configured to detect, after actuating the TimeClick functionality, a TimeClick gesture directed at one of the subsequent interactive items, wherein the TimeClick gesture comprises the user keeping the hand relatively steady for a specific period of time, and wherein the specific time period is adjusted in response to the distance between the user and the depth sensor.

12. The apparatus according to claim 11, wherein the computer is configured to present a cursor on the display, the cursor having a visual effect, and to decrease the visual effect upon detecting the user keeping the hand relatively steady for the specific period of time.

13. The apparatus according to claim 11, wherein the computer is configured to adjust the specific time period in response to a proficiency of the user.

14. The apparatus according to claim 11, wherein the multiple interactive items comprise alphanumeric keys for an on-screen keyboard presented on the display, and wherein the computer is configured to adjust the specific time period in response to a language model.

15. The apparatus according to claim 11, wherein one of the multiple interactive items comprises a TimeClick timer icon, and wherein the computer is configured to adjust the specific time period upon detecting, in the subsequent sequence of 3D maps a subsequent select gesture performed by the user toward the TimeClick timer icon.

16. The apparatus according to claim 15, wherein the computer is configured to select the subsequent select gesture from a list comprising the explicit select gesture and the TimeClick gesture.

17. The apparatus according to claim 11, wherein one of the multiple interactive icons comprises a mode icon, and wherein the computer is configured to actuate the TimeClick functionality upon detecting, in the sequence of 3D maps, the explicit select gesture toward the mode icon.

18. The apparatus according to claim 11, wherein the computer is configured to select the explicit select gesture from a list comprising a Push gesture, a Pull gesture, and a Grab gesture.

19. The apparatus according to claim 11, wherein the computer is configured to the define, prior to detecting the select gesture, coordinates for an interactive surface positioned in space in proximity to the user, and wherein completing the explicit select gesture comprises the hand crossing the interactive surface.

20. The apparatus according to claim 19, wherein the computer is configured to actuate the TimeClick functionality by redefining the coordinates of the interactive surface.

21. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile user interface, cause the computer to present multiple interactive items on a display coupled to the computer, to receive, from a depth sensor, a sequence of three-dimensional (3D) maps containing at least a hand of a user of the computer at a variable distance from the depth sensor, to detect, in the maps, an explicit select gesture performed by the user toward one of the interactive items, to select the one of the interactive items responsively to the explicit select gesture, and to actuate, subsequent to selecting the one of the interactive items, a TimeClick functionality for subsequent interactive item selections to be made by the user,

wherein the computer is configured to detect, after actuating the TimeClick functionality, a TimeClick gesture directed at one of the subsequent interactive items, wherein the TimeClick gesture comprises the user keeping the hand relatively steady for a specific period of time, and wherein the specific time period is adjusted in response to the distance between the user and the depth sensor.