Gesture-Based User Interface

• US 20120204133A1
• Filed: 03/19/2012
• Published: 08/09/2012
• Est. Priority Date: 01/13/2009
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Guendelman, Eran, Maizels, Aviad, Berliner, Tamir, Pokrass, Jonathan
• Application Number:
  US13/423,314
• Publication Number:
  US 20120204133A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A user interface method, including capturing, by a computer, a sequence of images over time of at least a part of a body of a human subject, and processing the images in order to detect a gesture, selected from a group of gestures consisting of a grab gesture, a push gesture, a pull gesture, and a circular hand motion. A software application is controlled responsively to the detected gesture.

Claims

1. A user interface method, comprising:

capturing, by a computer, a sequence of images over time of at least a part of a body of a human subject;

processing the images in order to detect a gesture, selected from a group of gestures consisting of a grab gesture, a push gesture, a pull gesture, and a circular hand motion; and

controlling a software application responsively to the detected gesture.

2. The method according to claim 1, wherein the images comprise depth maps captured by a three-dimensional sensing device.

3. The method according to claim 1, wherein the images comprise two-dimensional images captured by one or more two-dimensional sensing devices.

4. The method according to claim 1, wherein the detected gesture comprises the grab gesture.

5. The method according to claim 4, wherein the part of the body comprises a hand, and wherein the grab gesture comprises the human subject folding one or more fingers of the hand toward a palm of the hand;

6. The method according to claim 4, wherein the part of the body comprises a hand, and wherein the grab gesture comprises the human subject making a first with the hand.

7. The method according to claim 4, wherein the part of the body comprises a hand, and wherein the grab gesture comprises the human subject pinching together two or more fingers of the hand with a thumb of the hand.

8. The method according to claim 1, wherein controlling the software application comprises the computer performing an operation associated with an application object presented on a display coupled to the computer.

9. The method according to claim 7, and comprising the computer ending the operation in response to the images indicating a conclusion of the grab gesture.

10. The method according to claim 1, wherein the detected gesture comprises the push gesture.

11. The method according to claim 10, wherein the part of the body comprises a hand, and wherein the push gesture comprises the human subject moving the hand toward the application object.

12. The method according to claim 1, wherein the detected gesture comprises the pull gesture.

13. The method according to claim 12, wherein the part of the body comprises a hand, and wherein the pull gesture comprises the human subject moving the hand away from the application object.

14. The method according to claim 1, wherein the detected gesture comprises the circular hand motion.

15. The method according to claim 14, wherein controlling the software application comprises rotating, on a display coupled to the computer, an application object in the direction of the detected circular motion, and performing an operation associated with rotating the application object.

16. The method according to claim 14, wherein the circular motion comprises moving a palm in a counterclockwise direction in a vertical plane.

17. The method according to claim 14, wherein the circular motion comprises moving a palm in a clockwise direction in a vertical plane.

18. An apparatus, comprising:

a display; and

a computer coupled to the display and configured to capture a sequence of images over time of at least a part of a body of a human subject, to process the images in order to detect a gesture, selected from a group of gestures consisting of a grab gesture, a push gesture, a pull gesture, and a circular hand motion, and to control a software application responsively to the detected gesture.

19. The apparatus according to claim 18, wherein the images comprise depth maps, and the computer is configured to capture the depth maps conveyed by a three-dimensional sensing device.

20. The apparatus according to claim 18, wherein the images comprise two-dimensional images, and the computer is configured to capture the two-dimensional images conveyed by one or more two-dimensional sensing devices.

21. The apparatus according to claim 18, wherein the detected gesture comprises the grab gesture.

22. The apparatus according to claim 21, wherein the part of the body comprises a hand, and wherein the grab gesture comprises the human subject folding one or more fingers of the hand toward a palm of the hand.

23. The apparatus according to claim 21, wherein the part of the body comprises a hand, and wherein the grab gesture comprises the human subject making a first with the hand;

24. The apparatus according to claim 21, wherein the part of the body comprises a hand, and wherein the grab gesture comprises the human subject pinching together two or more fingers of the hand with a thumb of the hand.

25. The apparatus according to claim 18, wherein the computer is configured to control the software application by performing an operation associated with an application object presented on the display.

26. The apparatus according to claim 25, wherein the computer is configured to end the operation in response to the images indicating a conclusion of the grab gesture.

27. The apparatus according to claim 18, wherein the detected gesture comprises the push gesture.

28. The apparatus according to claim 27, wherein the part of the body comprises a hand, and wherein the push gesture comprises the human subject moving the hand toward the application object.

29. The apparatus according to claim 18, wherein the detected gesture comprises the pull gesture.

30. The apparatus according to claim 29, wherein the part of the body comprises a hand, and wherein the pull gesture comprises the human subject moving the hand away from the application object.

31. The apparatus according to claim 18, wherein the detected gesture comprises the circular hand motion.

32. The apparatus according to claim 31, wherein the computer is configured to control the software application by rotating an application object, presented on the display, in the direction of the detected circular motion, and performing an operation associated with rotating the application object.

33. The apparatus according to claim 31, wherein the circular motion comprises moving a palm in a counterclockwise direction in a vertical plane.

34. The apparatus according to claim 31, wherein the circular motion comprises moving a palm in a clockwise direction in a vertical plane.

35. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer to capture a sequence of depth maps over time of at least a part of a body of a human subject, to process the depth maps in order to detect a gesture, selected from a group of gestures consisting of a grab gesture, a push gesture, a pull gesture, and a circular hand motion, and to control a software application responsively to the detected gesture.