Device interaction with spatially aware gestures

• US 10,254,847 B2
• Filed: 05/19/2017
• Issued: 04/09/2019
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Kauffmann, Alejandro Jose, Plagemann, Christian
• Primary Examiner(s): Cheng, Joe H
• Application Number:
  US15/599,719
• Publication Number:
  US 20170277273A1
• Time in Patent Office: 690 Days
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Described is a system and technique for providing the ability for a user to interact with one or more devices by performing gestures that mimic real-world physical analogies. More specifically, the techniques described herein provide the ability for a user to interact with a device by limiting the conscious gesturing for a computer component by camouflaging computer-recognizable gestures within manipulations of a physical objects.

Claims

1. A method performed by one or more computing devices, the method comprising:

determining, by the one or more computing devices, that a display interface of a first electronic device is displaying a graphical representation of data content;

determining, by the one or more computing devices, that a user associated with the first electronic device has performed a gesture associated with a request to transfer the data content, the gesture corresponding to a selection of

the graphical representation of data content through a touch input received at the display interface of the first electronic device;

determining, by the one or more computing devices, that a display interface of a second electronic device has received a touch input;

generating, by the one or more computing devices, an instruction to transmit the data content from the first electronic device to the second electronic device based on (i) the touch input received at the display interface of the first electronic device, (ii) the touch input at the display interface of the second electronic device, and (iii) the gesture associated with the request to transfer the data content; and

transmitting, by the one or more computing device to the first electronic device through a wireless computer network, the instruction to transmit the data content to the second electronic device.

2. The method of claim 1, further comprising:

before generating the instruction to transmit the data content to the second electronic device, authorizing, by the one or more computing devices, the second electronic device to access the data content,

wherein the instruction to transmit the data content to the second electronic device is generated in response to authorizing the second electronic device.

3. The method of claim 1, further comprising:

before transmitting the instruction to transmit the data content to the second electronic device;

determining privacy controls associated with the data content;

determining that (I) the data content includes personally identifiable information associated with a user of the first electronic device, and (II) that the privacy controls associated with the data content include a restriction to share the personally identifiable information associated with the user of the first electronic device; and

generating instructions to the first electronic device to remove the personally identifiable information associated with the user of the first electronic device from the data content to be transmitted to the second electronic device.

4. The method of claim 1, further comprising:

detecting the first electronic device and the second electronic device within a field of view of a capture device; and

determining, based on data obtained from the capture device, a distance between the first electronic device and the capture device and a distance between the second electronic device and the capture device.

5. The method of claim 1, further comprising:

determining that the touch input at the display interface of the first electronic device and the touch input at the display interface of the second electronic device were performed by the same user.

6. The method of claim 1, further comprising:

determining that the touch input at the display interface of the first electronic device and the touch input at the display interface of the second electronic device were performed by the different users.

7. The method of claim 1, wherein determining, by the one or more computing devices, that the user associated with the first electronic device has performed the gesture associated with the request to transfer the data content comprises:

detecting, using a capture device, one or more body movements of the user; and

determining that the one or more body movements of the user correspond to a request to transfer data content.

8. A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

determining that a display interface of a first electronic device is displaying a graphical representation of data content;

determining that a user associated with the first electronic device has performed a gesture associated with a request to transfer the data content, the gesture corresponding to a selection of

the graphical representation of data content through a touch input received at the display interface of the first electronic device;

determining that a display interface of a second electronic device has received a touch input;

generating an instruction to transmit the data content from the first electronic device to the second electronic device based on (i) the touch input received at the display interface of the first electronic device, (ii) the touch input at the display interface of the second electronic device, and (iii) the gesture associated with the request to transfer the data content; and

transmitting, to the first electronic device through a wireless computer network, the instruction to transmit the data content to the second electronic device.

9. The system of claim 8, wherein the operations further comprise:

before generating the instruction to transmit the data content to the second electronic device, authorizing the second electronic device to access the data content,

wherein the instruction to transmit the data content to the second electronic device is generated in response to authorizing the second electronic device.

10. The system of claim 8, wherein the operations further comprise:

before transmitting the instruction to transmit the data content to the second electronic device;

determining privacy controls associated with the data content;

determining that (I) the data content includes personally identifiable information associated with a user of the first electronic device, and (II) that the privacy controls associated with the data content include a restriction to share the personally identifiable information associated with the user of the first electronic device; and

generating instructions to the first electronic device to remove the personally identifiable information associated with the user of the first electronic device from the data content to be transmitted to the second electronic device.

11. The system of claim 8, wherein the operations further comprise:

detecting the first electronic device and the second electronic device within a field of view of a capture device; and

determining, based on data obtained from the capture device, a distance between the first electronic device and the capture device and a distance between the second electronic device and the capture device.

12. The system of claim 8, wherein the operations further comprise:

determining that the touch input at the display interface of the first electronic device and the touch input at the display interface of the second electronic device were performed by the same user.

13. The system of claim 8, wherein the operations further comprise:

determining that the touch input at the display interface of the first electronic device and the touch input at the display interface of the second electronic device were performed by the different users.

14. The system of claim 8, wherein determining that the user associated with the first electronic device has performed the gesture associated with the request to transfer the data content comprises:

detecting, using a capture device, one or more body movements of the user; and

determining that the one or more body movements of the user correspond to a request to transfer data content.

15. One or more non-transitory computer-readable storage media comprising instructions, which, when executed by one or more computers, cause the one or more computers to perform actions comprising:

determining that a display interface of a first electronic device is displaying a graphical representation of data content;

determining that a user associated with the first electronic device has performed a gesture associated with a request to transfer the data content, the gesture corresponding to a selection of

the graphical representation of data content through a touch input received at the display interface of the first electronic device;

determining that a display interface of a second electronic device has received a touch input;

generating an instruction to transmit the data content from the first electronic device to the second electronic device based on (i) the touch input received at the display interface of the first electronic device, (ii) the touch input at the display interface of the second electronic device, and (iii) the gesture associated with the request to transfer the data content; and

transmitting, to the first electronic device through a wireless computer network, the instruction to transmit the data content to the second electronic device.

16. The one or more non-transitory computer-readable storage media of claim 15, wherein the operations further comprise:

before generating the instruction to transmit the data content to the second electronic device, authorizing the second electronic device to access the data content,

wherein the instruction to transmit the data content to the second electronic device is generated in response to authorizing the second electronic device.

17. The one or more non-transitory computer-readable storage media of claim 15, wherein the operations further comprise:

before transmitting the instruction to transmit the data content to the second electronic device;

determining privacy controls associated with the data content;

determining that (I) the data content includes personally identifiable information associated with a user of the first electronic device, and (II) that the privacy controls associated with the data content include a restriction to share the personally identifiable information associated with the user of the first electronic device; and

generating instructions to the first electronic device to remove the personally identifiable information associated with the user of the first electronic device from the data content to be transmitted to the second electronic device.

18. The one or more non-transitory computer-readable storage media of claim 15, wherein the operations further comprise:

detecting the first electronic device and the second electronic device within a field of view of a capture device; and

determining, based on data obtained from the capture device, a distance between the first electronic device and the capture device and a distance between the second electronic device and the capture device.

19. The one or more non-transitory computer-readable storage media of claim 15, wherein the operations further comprise one of:

determining that the touch input at the display interface of the first electronic device and the touch input at the display interface of the second electronic device were performed by the same user;

or

determining that the touch input at the display interface of the first electronic device and the touch input at the display interface of the second electronic device were performed by the different users.

20. The one or more non-transitory computer-readable storage media of claim 15, wherein determining that the user associated with the first electronic device has performed the gesture associated with the request to transfer the data content comprises:

detecting, using a capture device, one or more body movements of the user; and

determining that the one or more body movements of the user correspond to a request to transfer data content.