Gesture-based small device input

• US 10,139,915 B1
• Filed: 03/18/2016
• Issued: 11/27/2018
• Est. Priority Date: 10/24/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Miller, James B., Gossweiler, III, Richard C.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US15/074,205
• Time in Patent Office: 984 Days
• Field of Search: 345 8, 345102, 345156-179, 345204, 345213, 345418, 345581, 345589, 345629, 73495, 4555661, 361728, 600561, 600461, 600462, 600540
• CPC Class Codes: G04G 21/04 using radio waves radio-controlled time-pieces G04R
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A computer-implemented method is described that includes providing a watch defining a body and a watch face on a surface of the body, the body having one or more sensors arranged to sense user inputs in an area adjacent to the body. The method further comprises sensing a motion of an object in the area adjacent to, but not touching, the body using the one or more sensors and changing a display of a pointing element on a graphical user interface on the watch in coordination with the sensed motion.

Claims

1. A computer-implemented method, comprising:

monitoring for movement, by a computing device using ultrasound augmenting a camera, an area on a surface of a user, the surface comprising a surface on the user'"'"'s body;

determining, by the computing device, that movement of an object has occurred in the area on the surface of the user; and

in response to determining that the movement of the object has occurred in the area on the surface of the user, performing an operation that corresponds to the movement.

2. The computer-implemented method of claim 1, wherein the computing device is affixed to the user while the computing device monitors the area on the surface of the user for movement, wherein the area on the surface of the user is adjacent to a location on the user at which the computing device is affixed.

3. The computer-implemented method of claim 2, wherein the computing device comprises a watch that includes one or more sensors in a body of the watch, the one or more sensors arranged to detect movement of objects within a proximity of the body of the watch.

4. The computer-implemented method of claim 1, comprising:

identifying a path traced by the object within the area on the surface of the user; and

causing a pointing element displayed in a graphical user interface to move in coordination with the path traced by the object within the area on the surface of the user.

5. The computer-implemented method of claim 4, wherein:

the computing device comprises a watch that includes an electronic display on a face of the watch;

the graphical user interface is displayed on the electronic display on the face of the watch; and

causing the pointing element to move in coordination with the path traced by the object within the area on the surface of the user comprises moving the pointing element across the face of the watch.

6. The computer-implemented method of claim 1, wherein determining that movement of the object has occurred in the area on the surface of the user comprises detecting movement of a finger or a stylus in the area on the surface of the user.

7. The computer-implemented method of claim 1, comprising:

projecting an image onto the surface of the user in the area monitored by the computing device for movement, the image comprising a graphical user interface that includes a user-selectable element; and

determining that movement of the object has occurred in the area on the surface of the user comprises identifying user interaction with the user-selectable element in the area on the surface of the user.

8. The computer-implemented method of claim 7, wherein the user-selectable element in the graphical user interface of the image projected onto the surface of the user comprises a virtual key.

9. The computer-implemented method of claim 1, wherein determining that movement of the object has occurred in the area on the surface of the user comprises identifying a change in an intensity of light reflected from the area on the surface of the user.

10. The computer-implemented method of claim 9, wherein the computing device comprises a light source and a light sensor;

and

the method comprises;

illuminating the area on the surface of the user using the light source of the computing device; and

using the light sensor to monitor for changes in the intensity of light reflected from the area on the surface of the user.

11. A computing device, comprising:

one or more processors; and

a computer-readable medium having instructions stored thereon that, when executed by the one or more processors, cause performance of operations comprising;

monitoring for movement, by the computing device using ultrasound augmenting a camera, an area on a surface of a user, the surface comprising a surface on the user'"'"'s body;

determining, by the computing device, that movement of an object has occurred in the area on the surface of the user; and

in response to determining that the movement of the object has occurred in the area on the surface of the user, performing an operation that corresponds to the movement.

12. The computing device of claim 11, wherein the computing device is affixed to the user while the computing device monitors the area on the surface of the user for movement, wherein the area on the surface of the user is adjacent to a location on the user at which the computing device is affixed.

13. The computing device of claim 12, wherein the computing device comprises a watch that includes one or more sensors in a body of the watch, the one or more sensors arranged to detect movement of objects within a proximity of the body of the watch.

14. The computing device of claim 11, wherein the operations comprise:

identifying a path traced by the object within the area on the surface of the user; and

causing a pointing element displayed in a graphical user interface to move in coordination with the path traced by the object within the area on the surface of the user.

15. The computing device of claim 14, wherein:

the computing device comprises a watch that includes an electronic display on a face of the watch;

the graphical user interface is displayed on the electronic display on the face of the watch; and

causing the pointing element to move in coordination with the path traced by the object within the area on the surface of the user comprises moving the pointing element across the face of the watch.

16. The computing device of claim 11, wherein determining that movement of the object has occurred in the area on the surface of the user comprises detecting movement of a finger or a stylus in the area on the surface of the user.

17. The computing device of claim 11, wherein the operations comprise:

projecting an image onto the surface of the user in the area monitored by the computing device for movement, the image comprising a graphical user interface that includes a user-selectable element; and

determining that movement of the object has occurred in the area on the surface of the user comprises identifying user interaction with the user-selectable element in the area on the surface of the user.

18. The computing device of claim 17, wherein the user-selectable element in the graphical user interface of the image projected onto the surface of the user comprises a virtual key.

19. The computing device of claim 11, wherein determining that movement of the object has occurred in the area on the surface of the user comprises identifying a change in an intensity of light reflected from the area on the surface of the user.

20. The computing device of claim 19, wherein the computing device comprises a light source and a light sensor;

and

the operations comprise;

illuminating the area on the surface of the user using the light source of the computing device; and

using the light sensor to monitor for changes in the intensity of light reflected from the area on the surface of the user.