Gesture detection and interactions

• US 10,088,908 B1
• Filed: 09/23/2015
• Issued: 10/02/2018
• Est. Priority Date: 05/27/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Poupyrev, Ivan, Schwesig, Carsten, Schulze, Jack, Arnall, Timo, Bishop, Durrell Grant Bevington
• Primary Examiner(s): Earles, Bryan
• Application Number:
  US14/862,409
• Time in Patent Office: 1,105 Days
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Gesture detection and interaction techniques are described. Object detection used to support the gestures may be accomplished in a variety of ways, such as by using radio waves as part of a radar technique. In a first example, the techniques are implemented such that one hand of a user sets a context for a gesture that is defined by another hand of the user. In another example, a gesture recognition mode is utilized. In yet another example, detection of distance is used such that the same motions may be used to different between operations performed. In a further example, split gestures are supported. In another instance, entry into a gesture recognition mode may be implemented through touch and then recognized through three-dimensional orientation and motion of that hand or another.

Claims

1. A method of controlling operation of a computing device based on gesture detection, the method comprising:

detecting a first input by the computing device by recognizing a first gesture performed by a first hand of a user;

detecting a second input by the computing device by recognizing a second gesture performed by a second hand of the user, the second input defining one or more operations to be performed; and

controlling, by the computing device, performance of the one or more operations by dynamically adjusting, based on the first input, an amount of scale used in the performance of the one or more operations.

2. The method of claim 1, wherein detecting the first and second gestures includes detecting movement or orientation of respective first or second hands of the user.

3. The method of claim 1, wherein the second gesture defines a relative amount as part of the performance of the one or more operations and the first gesture describes the amount of scale used to scale the relative amount as part of the performance.

4. The method of claim 1, wherein the one or more operations including navigation, scrolling, or zoom of a user interface of the computing device.

5. The method of claim 1, wherein the detecting of the first input and the detecting of the second input are performed using a three dimensional object detection system of the computing device.

6. The method of claim 5, wherein the three dimensional object detection system performs the detecting of the first input and the detecting of the second input using radar techniques involving radio waves.

7. The method of claim 6, wherein the radio waves correspond to a frequency band included as part of a defined wireless communication radio spectrum.

8. The method of claim 1, wherein the detecting of the first input and the detecting of the second input are performed by the computing device through an article of clothing worn by the user.

9. A computing device comprising:

a housing;

a three dimensional object detection system disposed within the housing and configured to identify and detect orientation and movement of first and second hands of a user in three dimensional space using radio waves; and

a gesture module implemented at least partially in hardware and disposed within the housing, the gesture module configured to control performance of one or more operations by the computing device responsive to detection of first orientation and movement of the first hand of the user to modify a scaling factor used in the performance of the one or more operations defined by a gesture corresponding to second orientation and movement of the second hand of the user.

10. The computing device of claim 9, wherein the radio waves correspond to a frequency band included as part of a wireless communication radio spectrum.

11. The computing device of claim 9, wherein the first orientation and movement of the first hand of the user are usable to disambiguate the second orientation and movement of the second hand of the user as corresponding to different said gestures when the second orientation and movement of the second hand of the user matches the first orientation and movement of the first hand.

12. The computing device of claim 9, wherein the housing is configured to be worn or carried by the user.

13. The computing device of claim 9, wherein the housing is part of an automobile, television, or desktop computer.

20. The computing device of claim 9, wherein the gesture module is further configured to:

increase the scaling factor responsive to the first movement of the first hand being in a first direction; and

decrease the scaling factor responsive to the first movement of the first hand being in a second direction that is different than the first direction.

14. A computing device comprising:

a three dimensional object detection system configured to detect orientation or movement of objects in three dimensional space; and

a gesture module implemented at least partially in hardware, the gesture module configured to;

detect a first input, received by the computing device, by recognizing a first gesture defined by a first movement or orientation of a first hand of a user;

detect a second input, received by the computing device, by recognizing a second gesture defined by a second movement or orientation of a second hand of the user; and

control, by the computing device, performance of one or more operations defined by the second gesture by dynamically adjusting an amount of scale used in performance of the one or more operations based on the first gesture.

15. The computing device of claim 14, wherein the second gesture defines a relative amount as part of the performance of the one or more operations and the first gesture describes the amount of scale used to scale the relative amount as part of the performance.

16. The computing device of claim 14, wherein the one or more operations including navigation, scrolling, or zoom of a user interface of the computing device.

17. The computing device of claim 14, wherein the first input and the second input are detected based on radar techniques involving radio waves.

18. The computing device of claim 17, wherein the radio waves correspond to a frequency band included as part of a defined wireless communication radio spectrum.

19. The computing device of claim 14, wherein the amount of scale is dynamically adjusted based on the first movement of the first hand of the user, and wherein the adjusting includes:

increasing the amount of scale responsive to the first movement of the first hand being in a first direction;

or

decreasing the amount of scale responsive to the first movement of the first hand being in a second direction that is different than the first direction.