Agent Interfaces for Interactive Electronics that Support Social Cues

• US 20150138333A1
• Filed: 02/28/2012
• Published: 05/21/2015
• Est. Priority Date: 02/28/2012
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: DeVaul, Richard Wayne, Aminzade, Daniel
• Application Number:
  US13/407,159
• Publication Number:
  US 20150138333A1
• Time in Patent Office: Days
• US Class Current: 348/78
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 1/32 for obtaining desired directional characteristic only specially adapted for hearing aids H04R25/40
• CPC Class Codes: H04R 2201/025 Transducer mountings or cabinet supports enabling variable orientation of transducer of cabinet

Abstract

An anthropomorphic device, perhaps in the form factor of a doll or toy, may be configured to control one or more media devices. Upon reception or a detection of a social cue, such as movement and/or a spoken word or phrase, the anthropomorphic device may aim its gaze at the source of the social cue. In response to receiving a voice command, the anthropomorphic device may interpret the voice command and map it to a media device command. Then, the anthropomorphic device may transmit the media device command to a media device, instructing the media device to change state.

Claims

1. A method comprising:

an anthropomorphic device detecting a social cue, wherein the anthropomorphic device includes a camera and a microphone, and wherein detecting the social cue comprises the camera detecting a gaze directed toward the anthropomorphic device;

the anthropomorphic device aiming the camera and the microphone based on the direction of the gaze;

while the gaze is directed toward the anthropomorphic device, the anthropomorphic device receiving an audio signal via the microphone; and

based on receiving the audio signal while the gaze is directed toward the anthropomorphic device, the anthropomorphic device (i) transmitting a media device command to a media playback device, wherein the media playback device is separate from the anthropomorphic device, and (ii) providing an acknowledgement of the audio signal, wherein the media device command is based on the audio signal and instructs the media playback device to play out selected content.

2. The method of claim 1, wherein the anthropomorphic device comprises a head, and wherein the camera and the microphone are attached to the head.

3. The method of claim 1, wherein the audio signal is a voice command that directs the anthropomorphic device to change a state of the media playback device, and wherein the media device command instructs the media playback device to change the state.

4. The method of claim 1, wherein detecting the social cue further comprises identifying a user associated with the gaze directed toward the anthropomorphic device.

5. The method of claim 4, wherein identifying the user comprises:

performing facial recognition on the user to determine an identity of the user; and

based on the identity of the user, determining that the user has permission to use the anthropomorphic device.

6. The method of claim 5, wherein the anthropomorphic device has access to a profile of the user, wherein the profile contains one or more preferences of the user that map audio signals to media device commands, and wherein transmitting the media device command to the media playback device is based on looking up the audio signal in the mapping to find the media device command.

7. The method of claim 1, further comprising:

the anthropomorphic device also receiving, via the camera, a non-audio signal, wherein

transmitting the media device command to the media playback device is also based on receiving the non-audio signal.

8. The method of claim 1, wherein receiving the audio signal comprises filtering the audio signal from background noise received with the audio signal.

9. The method of claim 1, wherein the anthropomorphic device also includes a speaker,

and wherein providing the acknowledgment comprises producing a sound via the speaker.

10. The method of claim 1, further comprising:

in response to detecting the social cue, the anthropomorphic device transitioning from a sleep mode to an active mode, wherein the anthropomorphic device uses less power when in the sleep mode than when in the active mode.

11. The method of claim 10, further comprising:

after receiving the audio signal, the anthropomorphic device detecting inactivity for a given period of time; and

in response to detecting inactivity for the given period of time, the anthropomorphic device transitioning from the active mode to the sleep mode.

12. The method of claim 1, wherein aiming the camera and the microphone based on the direction of the gaze comprises aiming the camera and the microphone at a source of the gaze.

13. The method of claim 1, further comprising:

a second anthropomorphic device detecting a second social cue, wherein the second anthropomorphic device includes a second camera and a second microphone, and wherein detecting the second social cue comprises the second camera detecting a second gaze directed toward the second anthropomorphic device;

the second anthropomorphic device aiming the second camera and the second microphone based on the direction of the second gaze;

while the second gaze is directed toward the second anthropomorphic device, the second

anthropomorphic device receiving, via the second microphone, a second audio signal; and

based on receiving the second audio signal while the second gaze is directed toward the second anthropomorphic device, the second anthropomorphic device (i) transmitting a second media device command to the media playback device, and (ii) providing a second acknowledgement of the second audio signal, wherein the second media device command is based on the second audio signal.

14. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by an anthropomorphic computing device, cause the anthropomorphic computing device to perform operations comprising:

detecting a social cue at the anthropomorphic computing device, wherein the anthropomorphic computing device includes a camera and a microphone, and wherein detecting the social cue comprises the camera detecting a gaze directed toward the anthropomorphic computing device;

aiming the camera and the microphone based on the direction of the gaze;

while the gaze is directed toward the anthropomorphic computing device, receiving an audio signal via the microphone; and

based on receiving the audio signal while the gaze is directed toward the anthropomorphic computing device, (i) transmitting a media device command to a media playback device, wherein the media playback device is separate from the anthropomorphic device, and (ii) providing an acknowledgement of the audio signal, wherein the media device command is based on the audio signal and instructs the media playback device to play out selected content.

15. The article of manufacture of claim 14, wherein the audio signal is a voice command that directs the anthropomorphic computing device to change a state of the media playback device, and wherein the media device command instructs the media playback device to change the state.

16. The article of manufacture of claim 14, wherein detecting the social cue further comprises identifying a user associated with the gaze directed toward the anthropomorphic computing device.

17. The article of manufacture of claim 16, wherein identifying the user comprises:

performing facial recognition on the user to determine an identity of the user; and

based on the identity of the user, determining that the user has permission to use the anthropomorphic computing device.

18. The article of manufacture of claim 17, wherein the anthropomorphic computing device has access to a profile of the user, wherein the profile contains one or more preferences of the user that map audio signals to media device commands, and wherein transmitting the media device command to the media playback device is based on looking up the audio signal in the mapping to find the media device command.

19. The article of manufacture of claim 14, wherein the operations further comprise:

in response to detecting the social cue, transitioning from a sleep mode to an active mode, wherein the anthropomorphic computing device uses less power when in the sleep mode than when in the active mode.

20. A method comprising:

an anthropomorphic device detecting a first audio signal, wherein the anthropomorphic device includes a camera and a microphone array, and wherein detecting the first audio signal comprises the microphone array detecting the first audio signal;

the anthropomorphic device determining that the first audio signal encodes at least one pre-determined activation keyword;

in response to determining that the first audio signal encodes the at least one pre-determined activation keyword, the anthropomorphic device (i) processing the first audio signal to determine a source direction of the first audio signal, and (ii) aiming the camera at the source direction of the first audio signal;

while the camera is aimed at the source direction of the first audio signal, the anthropomorphic device receiving a second audio signal via the microphone array;

based on at least one of input from the camera and the second audio signal, the anthropomorphic device determining that the first audio signal and the second audio signal are from a common source; and

in response to determining that the first audio signal and the second audio signal are from the common source, the anthropomorphic device (i) transmitting a media device command to a media playback device, wherein the media playback device is separate from the anthropomorphic device, and (ii) providing an acknowledgement of the second audio signal, wherein the media device command is based on the second audio signal and instructs the media playback device to play out selected content.