Gesture detection using an array of short-range communication devices

• US 8,638,190 B1
• Filed: 09/12/2012
• Issued: 01/28/2014
• Est. Priority Date: 02/02/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Want, Roy, Li, Yang, Schilit, William Noah
• Primary Examiner(s): Wu, Daniel
• Assistant Examiner(s): OTT, FREDERICK R
• Application Number:
  US13/612,427
• Time in Patent Office: 503 Days
• Field of Search: 715/863
• US Class Current: 340/5.1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/046 by electromagnetic means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

In general, techniques and systems for defining a gesture with a computing device using short-range communication are described. In one example, a method includes obtaining position information from an array of position devices using near-field communication (NFC) during a movement of the computing device with respect to the array, wherein the position information identifies unique positions within the array for each position device from which position information was obtained. The method may also include determining sequence information associated with the position information, wherein the sequence information is representative of an order in which the position information was obtained from each position device, and performing, by the computing device, an action based at least in part on the position information and the sequence information, wherein the position information and the sequence information are representative of a gesture input associated with the movement of the computing device.

Claims

1. A method comprising:

obtaining, by a computing device, position information from an array of position devices using near-field communication during a movement of the computing device with respect to the array, wherein the position information identifies unique positions within the array for each position device from which position information was obtained;

determining, by the computing device, sequence information associated with the position information, wherein the sequence information is representative of an order in which the position information was obtained from each position device;

defining, within at least one of the computing device and a networked device, a gesture input based at least in part on the sequence information and the unique positions of each of the position devices from which the position information was obtained;

comparing, within at least one of the computing device and the networked device, the gesture input to a plurality of predefined gestures;

presenting, by the computing device and for display at a display device, a representation of the gesture input, wherein the representation of the gesture input indicates the movement of the computing device with respect to the array of position devices;

receiving, by the computing device, a confirmation input that confirms the gesture input; and

responsive to receiving the confirmation input, performing, by the computing device, an action based at least in part on the gesture input, wherein the action is associated with one of the predefined gestures substantially matching the gesture input.

2. The method of claim 1, further comprising:

transmitting, by the computing device, the position information and the sequence information for each position device to a networked device via a network; and

receiving, by the computing device, gesture information from the networked device based at least in part on the position information and the sequence information, wherein the gesture information comprises a command to perform the action.

3. The method of claim 2, further comprising:

generating at least one uniform resource locator representative of the position information from at least one of the position devices and at least a portion of the sequence information.

4. The method of claim 1, wherein determining the sequence information comprises adding a time stamp to the position information that indicates when the position information was obtained from each position device.

5. The method of claim 1, further comprising:

capturing, by the computing device, an image of a printed graphic associated with the array; and

selecting, by the computing device, the action based at least in part on the gesture input and the captured image.

6. The method of claim 1, further comprising:

presenting an image on a display associated with the computing device prior to obtaining the position information; and

selecting, by the computing device, the action based at least in part on the gesture input and the presented image.

7. The method of claim 1, further comprising:

responsive to obtaining initial position information from a first position device of the array of position devices, presenting a representation of at least a subset of the plurality of predefined gestures on a display associated with the computing device, wherein each of the plurality of predefined gestures are associated with a respective action performable by the computing device.

8. The method of claim 1, further comprising:

receiving an initiation input that commands the computing device to obtain the position information, wherein obtaining the position information further comprises obtaining the position information in response to receiving the initiation input.

9. The method of claim 8, further comprising:

determining a time period from when the initiation input was received; and

responsive to the time period exceeding a defined threshold, terminating the obtaining of the position information.

10. The method of claim 8, wherein obtaining the position information further comprises obtaining the position information when continually receiving the initiation input.

11. The method of claim 8, wherein:

receiving the initiation input comprises receiving a first initiation input and a second initiation input;

obtaining the position information comprises;

obtaining a first set of position information from a first subset of the array of position devices in response to receiving the first initiation input; and

obtaining a second set of position information from a second subset of the array of position devices in response to receiving the second initiation input; and

the method further comprising;

defining the gesture input based on the first set of position information and the second set of position information.

12. The method of claim 1, further comprising:

determining when the computing device is unable to obtain position information from any position devices of the array; and

responsive to the determination, presenting at least one of a visual cue or an audio cue.

13. The method of claim 1, wherein obtaining the position information from the array of position devices comprises obtaining, by a near-field communication module of the computing device, the position information, and wherein the method further comprises:

detecting, by a sensor different than the near-field communication module, movement of the computing device during a time period that includes a time when the position information was obtained.

14. The method of claim 13, further comprising:

refining at least one of a shape and a length of the gesture input based at least in part on the detected movement, wherein the gesture input is defined based at least in part on the position information and the sequence information.

15. The method of claim 1, wherein the gesture input comprises one of a question mark, a plus sign, a minus sign, a checkmark, a number, or a letter.

16. The method of claim 1, wherein the computing device is a mobile device.

17. A computer-readable storage device encoded with instructions that cause one or more processors of a computing device to perform operations comprising:

obtaining position information from an array of position devices using near-field communication during a movement of the computing device with respect to the array, wherein the position information identifies unique positions within the array for each position device from which position information was obtained;

determining sequence information associated with the position information, wherein the sequence information is representative of an order in which the position information was obtained from each position device;

defining, within at least one of the computing device and a networked device, a gesture input based at least in part on the sequence information and the unique positions of each of the position devices from which position information was obtained;

comparing, within at least one of the computing device and the networked device, the gesture input to a plurality of predefined gestures;

presenting, for display at a display device, a representation of the gesture input, wherein the representation of the gesture input indicates the movement of the computing device with respect to the array of position devices;

receiving a confirmation input that confirms the gesture input; and

responsive to receiving the confirmation input, performing an action based at least in part on the gesture input, wherein the action is associated with one of the predefined gestures substantially matching the gesture input.

18. The computer-readable storage device of claim 17, wherein the instructions that cause the one or more processors of the computing device to perform operations comprising obtaining the position information from the array of position devices further comprises instructions that cause the one or more processors of the computing device to perform operations comprising obtaining, by a near-field communication module of the computing device, the position information, and wherein the computer-readable storage device further comprises instructions that cause the one or more processors of the computing device to perform operations comprising:

detecting, by a sensor different than the near-field communication module, movement of the computing device during a time period that includes a time when the position information was obtained; and

refining at least one of a shape and a length of the gesture input based at least in part on the detected movement, wherein the gesture input is defined based at least in part on the position information and the sequence information.

19. A computing device comprising:

a near-field communication module configured to obtain position information from an array of position devices using near-field communication during a movement of the computing device with respect to the array, wherein the position information identifies unique positions within the array for each position device from which position information was obtained; and

at least one processor configured to;

determine sequence information associated with the position information, wherein the sequence information is representative of an order in which the position information was obtained from each position device,

define, within at least one of the computing device and a networked device, a gesture input based at least in part on the sequence information and the unique positions of each of the position devices from which position information was obtained;

compare, within at least one of the computing device and the networked device, the gesture input to a plurality of predefined gestures;

present, for display at a display device, a representation of the gesture input, wherein the representation of the gesture input indicates the movement of the computing device with respect to the array of position devices;

receive a confirmation input that confirms the gesture input; and

responsive to reception of the confirmation input, perform an action based at least in part on the gesture input, wherein the action is associated with one of the predefined gestures substantially matching the gesture input.

20. The computing device of claim 19, further comprising a sensor, different than the near-field communication module, configured to detect movement of the computing device during a time period that includes a time when the position information was obtained, wherein the at least one processor is configured to refine at least one of a shape and a length of the gesture input based at least in part on the detected movement, and wherein the gesture input is defined based at least in part on the position information and the sequence information.