Multi sensory input to improve hands-free actions of an electronic device

• US 10,306,174 B1
• Filed: 09/15/2014
• Issued: 05/28/2019
• Est. Priority Date: 09/15/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Grafton, James, Kent, James
• Primary Examiner(s): Brown, Vernal U
• Application Number:
  US14/486,342
• Time in Patent Office: 1,716 Days
• Field of Search: 340 1253
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/44226 on social networks
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18

Abstract

In one general aspect, a method can include detecting at least one indicator of user-initiated interaction with a computing device, obtaining data related to a demographic of a user of the computing device, identifying a current state of the computing device, determining that content displayed on a first display device included in the computing device is to be casted to a second display device separate from the computing device based on the at least one indicator of the user-initiated interaction with the computing device, the data related to a demographic of a user of the computing device, and the current state of the computing device, and casting the content displayed on the first display device to the second display device.

Claims

1. A method, comprising:

obtaining, by a first device, data representing a demographic characteristic of a user of the first computing device;

receiving, by the first device and from a database remote from the first device, untethered aggregated demographic data for a group of individuals; and

casting content playing on the first device to a second device separate from the first device,

the casting being triggered by;

detecting at least one indicator of user-initiated interaction with the first device;

identifying, by the first device, that the first device is currently playing the content; and

determining that the data representing the demographic characteristic of the user of the first device is included in the untethered aggregated demographic data.

2. The method of claim 1, wherein the casting is further triggered by determining that a probability associated with the casting exceeds a threshold value.

3. The method of claim 1, further comprising receiving confirmation that the first device is to perform the casting before the first device casts the content.

4. The method of claim 1, further comprising receiving, after the casting, an indication of a response of the user to the casting.

5. The method of claim 4, further comprising storing the indication of the response in association with the casting.

6. The method of claim 1, further comprising:

identifying at least one user preference associated with the first device and included in an account accessible by the user of the first device; and

wherein the casting is further based on the at least one user preference.

7. The method of claim 6, further comprising:

receiving, after the casting, an indication of a response of the user to the casting; and

updating the at least one user preference based on the received indication of the response of the user.

8. The method of claim 1, wherein the untethered aggregated demographic data for the group of individuals is obtained from one or more demographic data records for a common subset of demographics for the group of individuals.

9. The method of claim 1, wherein the untethered aggregated demographic data for the group of individuals is used to predict an action to be performed on the first device.

10. The method of claim 1, wherein the user-initiated interaction with the first device is a physical interaction of the user with the first device.

11. The method of claim 1, wherein the user-initiated interaction is a movement of the first device by the user of the first device.

12. The method of claim 1, wherein identifying a current state of the first device is based on receiving data indicative of at least one identified condition associated with the first device.

13. The method of claim 12, wherein a current state of the computing device is identified as at least one of playing music, playing video content, or displaying an image on the first display device.

14. The method of claim 12, wherein the data indicative of at least one identified condition associated with the first device is received from one or more sensors included in the first device.

15. The method of claim 1, wherein the casting further includes:

displaying the content on the second device without displaying the content on the first device; and

allowing the first device to control the displaying of the content on the second device.

16. The method of claim 1, wherein the casting is further triggered by a proximity of the first device with the second device.

17. An apparatus, comprising:

a plurality of sensors configured to provide data indicative of at least one identified condition associated with a first device;

a location detection module configured to identify a proximity of the first device with respect to a second device;

a device context module configured to identify a context of the first device;

a gesture detection module configured to identify at least one gesture associated with the first device;

a memory configured to store at least one characteristic associated with a user of the first device; and

an activity module configured to cast content from the first device to the second device, the casting being triggered by a combination of;

the identified proximity,

the identified context,

the identified at least one gesture,

the data indicative of movement of the first device, and

determining that data representing a demographic characteristic of a user of the first device is included in untethered aggregated demographic data.

18. The apparatus of claim 17, wherein the plurality of sensors includes at least one of a temperature sensor, an ambient light sensor, a magnetic field sensor, a gyroscope, a heart rate sensor, a relative humidity sensor, an accelerometer, a pressure point sensor, an altimeter, a barometer, an infrared (IR) sensor, or a step detector.

19. The apparatus of claim 17, wherein the at least one gesture associated with the first device includes at least one of a rotation of the first device, a shaking of the first device, a squeezing of the first device, an up-and-down movement of the first device, a side-to-side movement of the first device, a back-and-forth-movement of the first device, or a contact of a surface of the first device.

20. A non-transitory, machine-readable medium having instructions stored thereon, the instructions, when executed by a processor, cause a first device to:

obtain data representing a demographic characteristic of a user of the first device;

identify at least one preference of the user of the first device;

receive, from a database remote from the first device, untethered aggregated demographic data for a group of individuals;

cast content playing on the first device to a second device separate from the first device, the casting being triggered by;

detecting at least one indicator of user-initiated interaction with the first device;

identifying, by the first device, that the first device is currently playing the content;

the identified at least one user preference, and

determining that the data representing the demographic characteristic of the user of the first device is included in the untethered aggregated demographic data.

21. The non-transitory, machine-readable medium of claim 20, wherein the casting is further based on determining that a probability associated with the providing of the content exceeds a threshold value.

22. The non-transitory, machine-readable medium of claim 20, the instructions further causing the first device to receive confirmation that the first device is to provide the content before the first device provides the content.

23. The non-transitory, machine-readable medium of claim 20, the instructions further causing the first device to receive, after providing the content, an indication of a response of the user to the providing of the content.

24. The non-transitory, machine-readable medium of claim 20, wherein identifying a current state of the first device is based on receiving data indicative of at least one identified condition associated with the first device.

25. The non-transitory, machine-readable medium of claim 20, wherein the casting is further triggered by a proximity of the first device with the second device.