Assistant device with active and passive experience modes

• US 10,102,856 B2
• Filed: 05/18/2017
• Issued: 10/16/2018
• Est. Priority Date: 01/20/2017
• Status: Active Grant

• Current Assignee: Essential Products, Inc.
• Original Assignee: Essential Products, Inc.
• Inventors: Segal, Mara Clair, Roman, Manuel, Desai, Dwipal, Rubin, Andrew E.
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/599,407
• Publication Number:
  US 20180211659A1
• Time in Patent Office: 516 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Ambient assistance is described. An assistant device can operate in an active experience mode in which a response to a user'"'"'s speech is provided if it includes a hardware activation phrase. Based on characteristics of the environment, the mode can be adjusted to a passive experience mode in which a response to speech is provided even if it does not include the hardware activation phrase.

Claims

1. A home assistant device, comprising:

a camera;

one or more processors; and

memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to;

operate the home assistant device in a passive experience mode, the passive experience mode corresponding to the home assistant device providing a first response to a first speech based on the first speech not including a hardware activation phrase;

determine characteristics of an environment of the home assistant device using the camera, the characteristics of the environment including determining that a user is engaged with playback of media content in the environment; and

switch operation of the home assistant device from the passive experience mode to an active experience mode based on the determined characteristics including that the user is engaged with the playback of the media content, the active experience mode corresponding to the home assistant device providing a second response to a second speech, the second speech including the hardware activation phrase.

2. An electronic device, comprising:

one or more processors; and

memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to;

operate the electronic device in a passive experience mode providing a first response to first speech based on the first speech not including a hardware activation phrase;

determine characteristics of an environment of the electronic device, the characteristics of the environment including a determination that a user is engaged with playback of media content; and

adjust operation of the electronic device from the passive experience mode to an active experience mode based on the characteristics including that the user is engaged with the playback of the media content, the active experience mode configured to provide a second response to second speech including the hardware activation phrase.

3. The electronic device of claim 2, wherein the hardware activation phrase indicates an intention of the user providing the second speech to interact with the electronic device.

4. The electronic device of claim 2, wherein the electronic device further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

5. The electronic device of claim 2, wherein the processor is configured to execute the instructions such that the processor and memory are configured to:

determine characteristics of the second speech, wherein the adjusting operation of the electronic device from the passive experience mode to the active experience mode is further based on the characteristics of the second speech.

6. The electronic device of claim 2, wherein the characteristics of the environment include a number of people within the environment.

7. A method, comprising:

operating an electronic device in a passive experience mode, the passive experience mode configured to provide a first response to a first speech not including a hardware activation phrase;

determining, by a processor, characteristics of an environment of the electronic device the characteristics of the environment including a determination that a user is involved in activity related to playback of media content in the environment; and

adjusting operation of the electronic device from the passive experience mode to an active experience mode based on the characteristics including that the user is involved in activity related to the playback of the media content in the environment, the active experience mode providing a second response to second speech based on the second speech including the hardware activation phrase.

8. The method of claim 7, wherein the hardware activation phrase indicates an intention of a user providing the second speech to interact with the electronic device.

9. The method of claim 7, wherein the electronic devices further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

10. The method of claim 9, wherein the characteristics of the environment indicate that the user is looking at or towards the electronic device.

11. The method of claim 7, further comprising:

determining characteristics of the second speech, wherein the adjusting operation of the electronic device from the passive experience mode to the active experience mode is further based on the characteristics of the second speech.

12. The method of claim 7, wherein the characteristics of the environment include a number of people within the environment.

13. The method of claim 7, further comprising:

receiving information from a display device regarding playback of media content in the environment; and

adjusting operation of the electronic device from the active experience mode to the passive experience mode based on the information regarding the playback of the media content in the environment.

14. A computer program product, comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:

operate the computing device in a passive experience mode providing a first response to first speech based on the first speech not including a hardware activation phrase;

determine characteristics of an environment of the computing device, the characteristics of the environment including a determination that a user is engaged with playback of media content; and

adjust operation of the computing device from the passive experience mode to an active experience mode based on the characteristics including that the user is engaged with playback of media content, the active experience mode configured to provide a second response to second speech including the hardware activation phrase.

15. The computer program product of claim 14, wherein the hardware activation phrase indicates an intention of the user providing the second speech to interact with the computing device.

16. The computer program product of claim 14, wherein the computing device further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

17. The computer program product of claim 16, wherein the characteristics of the environment indicate that the user is looking at or towards the computing device.

18. The computer program product of claim 14, the computer program instructions cause the one or more computing devices to:

determine characteristics of the second speech, wherein the adjusting operation of the computing device from the passive experience mode to the active experience mode is further based on the characteristics of the second speech.

19. The computer program product of claim 14, wherein the characteristics of the environment include a number of people within the environment.