SYSTEMS AND METHODS FOR A VIRTUAL GRASPING USER INTERFACE

• US 20150220149A1
• Filed: 02/04/2013
• Published: 08/06/2015
• Est. Priority Date: 02/14/2012
• Status: Abandoned Application

• Current Assignee: Google Inc. (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Plagemann, Christian, Dahlkamp, Hendrik, Ganapathi, Varun
• Application Number:
  US13/758,746
• Publication Number:
  US 20150220149A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A location of a first portion of a hand and a location of a second portion of the hand are detected within a working volume, the first portion and the second portion being in a horizontal plane. A visual representation is positioned on a display based on the location of the first portion and the second portion. A selection input is initiated when a distance between the first portion and the second portion meets a predetermined threshold, to select an object presented on the display, the object being associated with the location of the visual representation. A movement of the first portion of the hand and the second portion of the hand also may be detected in the working volume while the distance between the first portion and the second portion remains below the predetermined threshold and, in response, the object on the display can be repositioned.

Claims

1. A computer program product, the computer program product being tangibly embodied on a computer-readable storage medium and comprising instructions that, when executed by one or more processors, cause a computing device to:

detect a plurality of parts of a human hand within a working volume of the computing device;

determine that the plurality of parts are in a configuration suitable for a grasping gesture; and

translate a location of the plurality of parts to a visual representation on a display of the computing device,

the visual representation allowing a user to interact with the computing device.

2. The computer program product of claim 1, wherein as part of the detecting, the instructions are further configured to detect at least three portions of the hand and, as part of the determining, determine that the first portion of the hand, the second portion of the hand, and the third portion of the hand are in a configuration for the grasping gesture.

3. The computer program product of claim 1, the instructions further configured to:

make an initial determination that the plurality of parts of the hand are in a horizontal plane; and

in response, initiate a cursor control mode allowing the user to manipulate the cursor by moving the plurality of parts of the hand within the working volume.

4. The computer program product of claim 3, the instructions further configured to cause the computing device to:

receive an input based on a predetermined position of the plurality of parts of the hand, the input configured to trigger termination of the cursor control mode and trigger a keyboard control mode of the computing device.

5. The computer program product of claim 1, wherein the plurality of parts include a portion of a thumb and a portion of a finger.

6. The computer program product of claim 5, the instructions further configured to cause the computing device to initiate a select-and-hold input when a distance between the portion of the thumb and the portion of the finger meets a threshold.

7. The computer program product of claim 6, the instructions further configured to cause the computing device to:

select an object presented on the display in response to initiating the select-and-hold input, the object corresponding with the visual representation on the display.

8. The computer program product of claim 7, further comprising instructions configured to cause the computing device to:

detect a movement of the portion of the thumb and the portion of the finger in the working volume during the select-and-hold input; and

reposition the object on the display based on the detected movement.

9. The computer program product of claim 5, further comprising instructions configured to cause the computing device to:

identify a focus point located between the portion of the thumb and the portion of the finger, wherein the focus point corresponds to the location of the plurality of parts.

10. The computer program product of claim 9, wherein as part of the identifying, the instructions are further configured to cause the computing device to identify the focus point at a location closer to the portion of the thumb than the portion of the finger.

11. A computer-implemented method, comprising:

detecting, by one or more processors, a first location of a first portion of a hand and a second location of a second portion of the hand within a working volume of a computing device;

determining that the first portion of the hand and the second portion of the hand are in a horizontal plane; and

positioning a visual representation on a display of the computing device based on the first location and the second location, wherein the hand is not in contact with the display of the computing device.

12. The method of claim 11, further comprising initiating a cursor control mode in response to the determining, the cursor control mode allowing a user to manipulate the visual representation by moving the hand within the working volume.

13. The method of claim 12, further comprising:

determining that the location of the first portion and the location of the second portion are not in a horizontal plane; and

terminating the cursor control mode and triggering a keyboard control mode of the computing device.

14. The method of claim 11, wherein the determining further includes detecting a location of a third portion of the hand and determining that the location of the first portion, the location of the second portion, and the location of the third portion are in the horizontal plane.

15. The method of claim 11, further comprising identifying a focus point located between the first location and the second location, wherein the visual representation is positioned at the focus point.

16. The method of claim 15, further comprising initiating a selection input when a distance between the first portion and the second portion meets a predetermined threshold.

17. The method of claim 16, further comprising:

selecting an object presented on the display in response to initiating the selection input, based on a location of the focus point on the display.

18. The method of claim 17, further comprising:

detecting a movement of the first portion of the hand and the second portion of the hand in the working volume while the distance between the first portion and the second portion remains below the predetermined threshold; and

repositioning the object on the display based on the detected movement.

19. The method of claim 15, wherein the identifying includes identifying the focus point at a location closer to the location of the first portion of the hand than the location of the second portion of the hand.

20. A computing device including instructions recorded on a non-transitory computer-readable medium and executable by at least one processor, the computing device comprising:

a gesture classification module configured to detect a gesture of a user within a working volume associated with the computing device, the gesture classification module configured to trigger initiation of a gesture cursor control mode of operating the computing device when the gesture matches a predetermined gesture signature stored within the computing device;

an imaging device configured to provide imaging data associated with the working volume to the gesture classification module; and

a gesture tracking module configured to;

position a cursor within a display portion of the computing device at a location based on a position of a first portion of a hand and a position of a second portion of the hand within the working volume, and

move the cursor within the display portion to correspond to movement of the first portion of the hand and the second portion of the hand within the working volume when the computing device is in the gesture cursor control mode.

21. The computing device of claim 20, wherein as part positioning the cursor, the gesture tracking module is configured to:

determine a location of the first portion of the hand of the user and the second portion of the hand of the user within the working volume; and

locate a focus point between the location of the first portion and the location of the second portion.

22. The computing device of claim 21, wherein the first portion of the hand is a tip of a thumb and the second portion of the hand is a tip of a finger and the focus point is located closer to the location of the thumb than the location of the finger.

23. The computing device of claim 20, wherein the predetermined gesture signature includes the first portion of the hand and the second portion of the hand being in a horizontal plane within the working volume.

24. The computing device of claim 20, wherein the imaging device includes a capture device configured to provide 3-dimensional information associated with the working volume to the gesture tracking module.

25. The computing device of claim 20, wherein the imaging device is located within a housing of the computer computing device.

26. The computing device of claim 25, wherein the working volume is above a keyboard portion of the computing device.

27. The computing device of claim 20, wherein the gesture classification module is further configured to:

detect a second gesture of the user within the working volume; and

initiate a selection input when the second gesture matches a second predetermined gesture signature stored within the computing device.

28. The computing device of claim 27, wherein the second gesture signature includes a distance between the first portion of the hand and the second portion of the hand meeting a predetermined threshold.