Seeing with your hand

• US 8,248,364 B1
• Filed: 07/29/2011
• Issued: 08/21/2012
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ho, Harvey, Chi, Liang-Yu, Ryskamp, Robert Allen, Gomez, Luis Ricardo Prada, Brin, Sergey
• Primary Examiner(s): Liang, Regina
• Application Number:
  US13/194,622
• Time in Patent Office: 389 Days
• Field of Search: 345156-158, 345/169, 715/856, 715/863, 382/103, 348/155, 348/169, 348/208.99, 3482081-2084, 348/208.14, 463 37- 39
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks

Abstract

The disclosure describes methods and systems for gathering and conveying information, for example, such as with a hand of a user. In one embodiment, the method may include using a detector to record a series of images of an environment and detecting a predetermined motion by comparing two or more images in the series. The method may include selecting a function based on the predetermined motion and triggering the function. In another embodiment, the method may include using a first detector to record a first series of images of an environment and using a second detector to record a second series of images of an environment. The method may include detecting a predetermined relative motion by comparing one or more images from the first series with one or more images from the second series, and selecting and/or triggering a function based on the predetermined relative motion.

Claims

1. A glove, comprising:

a first detector configured to record a first series of images of an environment;

a second detector configured to record a second series of images of the environment;

a processor configured to (i) determine at least one predetermined motion of the first detector by comparing at least two images in the first series of images, (ii) based on the at least one predetermined motion, select a first at least one function from a plurality of functions, (iii) determine at least one predetermined relative motion of the first detector and the second detector by comparing at least one image in the first series of images with at least one image in the second series of images, and, (iv) based on the at least one predetermined relative motion, select a second at least one function from the plurality of functions; and

an output interface configured to trigger the first at least one function and the second at least one function.

2. The glove of claim 1, wherein the output interface being configured to trigger the first at least one function comprises the output interface being configured to trigger the first at least one function to be carried out on a display.

3. The glove of claim 2, wherein the display comprises at least one of a wearable display that is wearable by a user and a remote display that is remote from the glove.

4. A wearable device, comprising:

a first detector configured to record a first series of images of an environment;

a second detector configured to record a second series of images of the environment;

a processor configured to (i) determine at least one predetermined motion of the first detector by comparing at least two images in the first series of images, (ii) based on the at least one predetermined motion, select a first at least one function from a plurality of functions, (iii) determine at least one predetermined relative motion of the first detector and the second detector by comparing at least one image in the first series of images with at least one image in the second series of images, and, (iv) based on the at least one predetermined relative motion, select a second at least one function from the plurality of functions; and

an output interface configured to transmit an output based at least in part on the first at least one function and the second at least one function.

5. The wearable device of claim 4, further comprising an orientation sensor configured to sense motion of the first detector.

6. The wearable device of claim 5, wherein the processor is further configured to stabilize the first series of images based on the motion of the first detector.

7. The wearable device of claim 4, wherein the output interface is further configured to transmit the output to a display that comprises at least one of a wearable display that is wearable by a user and a remote display that is remote from the wearable device.

8. The wearable device of claim 4, wherein the processor being configured to, based on the at least one predetermined motion, select the first at least one function from a plurality of functions comprises the processor being configured to look up the at least one predetermined motion in a set of correlations so as to determine the first at least one function.

9. The wearable device of claim 4, wherein the output interface being configured to transmit an output based at least in part on the first at least one function comprises the output interface being configured to trigger the first at least one function to be carried out on a display.

10. The wearable device of claim 4, further comprising at least one additional detector configured to record at least one additional series of images.

11. The wearable device of claim 10, wherein the processor is further configured to detect at least one predetermined relative motion of the at least one additional detector, the first detector, and the second detector by comparing at least one image in the first series of images with at least one image in the second series of images and at least one image in the at least one additional series of images.

12. A method, comprising:

recording a first series of images of an environment with a first wearable detector;

determining at least one predetermined motion by comparing at least two images in the first series of images;

maintaining a first set of correlations between a plurality of predetermined motions and a first plurality of functions; and

identifying the at least one predetermined motion in the first set of correlations so as to determine a first at least one function associated with the at least one predetermined motion;

recording a second series of images of the environment with a second wearable detector;

determining at least one predetermined relative motion of the first wearable detector and the second wearable detector by comparing at least one image in the first series of images with at least one image in the second series of images;

maintaining a second set of correlations between a plurality of predetermined relative motions and a second plurality of functions; and

identifying the at least one predetermined relative motion in the second set of correlations so as to determine a second at least one function associated with the at least one predetermined relative motion.

13. The method of claim 12, further comprising triggering the first at least one function.

14. The method of claim 12, further comprising triggering the second at least one function.

15. The method of claim 12, wherein at least one of the first at least one function and the second at least one function comprises at least one of entering at least one character in an application displayed by a display, zooming in on the display, zooming out on the display, scrolling on the display, panning on the display, moving an indicator on the display, selecting an object displayed by the display, using image recognition to identify at least one object in the environment, searching for information related to the at least one object, displaying information related to the at least one object on the display, identifying a cardinal direction, and displaying the cardinal direction on the display.

16. The method of claim 12, wherein the first at least one function comprises at least a first function and a second function, and the method further comprises:

selecting between the first function and the second function based on one or more of a determined speed of one of the first detector or the second detector, a determined distance moved by one of the first detector or the second detector, an object identified in at least one image in the series of images, and one or more previously-triggered functions; and

triggering the selected function.