Gesture Detection Haptics and Virtual Tools

• US 20160349845A1
• Filed: 05/26/2016
• Published: 12/01/2016
• Est. Priority Date: 05/28/2015
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Arnall, Timo, Schwesig, Carsten C., Poupyrev, Ivan, Schulze, Jack
• Application Number:
  US15/166,198
• Publication Number:
  US 20160349845A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Gesture detection haptics and virtual tools are described. In one example, movements are detected that involve contact in three-dimensional space, such as through use of radio waves, camera based techniques, and so forth. The contact provides haptic feedback to the user as part of making the movements. In another example, movements are detected that are used to both identify a virtual tool and a gesture that corresponds to the virtual tool. From these movements, gestures are identified that are used to initiate operations of a computing device.

Claims

1. A method of controlling operation of a computing device based on gesture detection, the method comprising:

detecting, by the computing device, inputs involving movement in three-dimensional space of body parts of a user in relation to each other, the movement imparting haptic feedback to the user through contact of the body parts, one to another;

recognizing, by the computing device, a gesture from the detected inputs; and

controlling, by the computing device, performance of one or more operations of the computing device that correspond to the recognized gesture.

2. The method as described in claim 1, wherein the detecting is performed using a three dimensional object detection system of the computing device that performs the detecting using radar techniques involving radio waves.

3. The method as described in claim 2, wherein the radio waves correspond to a frequency band included as part of a Wi-Fi radio spectrum.

4. The method as described in claim 1, wherein the detecting of the contact of the body parts specifies initiation of the one or more operations of the gesture.

5. The method as described in claim 4, wherein the detecting of release of the contact of the body parts specifies cessation of the one or more operations of the gesture.

6. The method as described in claim 1, wherein the movement of the body parts includes rotational movement of appendages of the user in relation to each other.

7. The method as described in claim 1, wherein the movement of the body parts includes pressing of an appendage of the user against another said body part of the user.

8. A method of controlling operation of a computing device based on gesture detection, the method comprising:

detecting, by the computing device, inputs involving user movement in three-dimensional space as both mimicking existence of a virtual tool and operation of the virtual tool;

identifying, by the computing device, the virtual tool from the detected inputs;

recognizing, by the computing device, a gesture from the detected inputs corresponding to the virtual tool; and

controlling, by the computing device, performance of one or more operations of the computing device that correspond to the identified virtual tool.

9. The method as described in claim 8, wherein the virtual tool mimics a size and shape of physical tool configured for interaction by a hand of the user.

10. The method as described in claim 8, wherein the virtual tool mimics a dial, screwdriver, hammer, tongs, power tool, or wipe.

11. The method as described in claim 8, wherein the existence of a virtual tool and the operation of the virtual tool are identified and recognized from the detected inputs as involving movement in three-dimensional space of body parts of a user in relation to each other.

12. The method as described in claim 8, wherein the detecting is performed using a three-dimensional object detection system configured to generate the inputs using radio waves.

13. The method as described in claim 12, wherein the radio waves correspond to a frequency band included as part of a Wi-Fi radio spectrum.

14. A system comprising:

a three-dimensional object detection system implemented at least partially in hardware of a computing device to detect inputs involving movement in three-dimensional space of body parts of a user in relation to each other, the movement imparting haptic feedback to the user through contact of the body parts, one to another; and

a gesture module implemented at least partially in hardware of the computing device to recognize a gesture from the detected inputs and control performance of one or more operations of the computing device that correspond to the recognized gesture.

15. The system as described in claim 14, wherein the three-dimensional object detection system is configured to detect the inputs using radio waves.

16. The system as described in claim 15, wherein the radio waves correspond to a frequency band included as part of a Wi-Fi radio spectrum.

17. The system as described in claim 14, wherein the three-dimensional object detection system is configured to detect the inputs through an article of clothing worn by the user.

18. The system as described in claim 14, further comprising a housing having the three-dimensional object detection system and the gesture module disposed therein.

19. The system as described in claim 18, wherein the housing is configured to be worn or carried by a user.

20. The system as described in claim 18, wherein the housing is part of an automobile, television, or desktop computer.