AMBIENT ASSISTANT DEVICE

• US 20180211659A1
• Filed: 05/18/2017
• Published: 07/26/2018
• Est. Priority Date: 01/20/2017
• Status: Active Grant

• Current Assignee: Essential Products, Inc.
• Original Assignee: Essential Products, Inc.
• Inventors: Segal, Mara Clair, Roman, Manuel, Desai, Dwipal, Rubin, Andrew E.
• Granted Patent: US 10,102,856 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Ambient assistance is described. An assistant device can operate in an active experience mode in which a response to a user'"'"'s speech is provided if it includes a hardware activation phrase. Based on characteristics of the environment, the mode can be adjusted to a passive experience mode in which a response to speech is provided even if it does not include the hardware activation phrase.

Claims

1. A home assistant device, comprising:

a camera;

one or more processors; and

memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to;

operate the home assistant device in an active experience mode, the active experience mode corresponding to the home assistant device providing a first response to first speech based on the first speech including a hardware activation phrase indicating an expectation of a user providing the first speech to receive the first response responsive to the first speech;

determine characteristics of an environment of the home assistant device using the camera, the characteristics of the environment including determining that the user is looking at or towards the home assistant device; and

switch operation of the home assistant device from the active experience mode to a passive experience mode based on the determined characteristics, the passive experience mode corresponding to the home assistant device providing a second response to second speech, the second speech not including the hardware activation phrase.

2. An electronic device, comprising:

one or more processors; and

memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to;

operate the electronic device in an active experience mode providing a first response to first speech based on the first speech including a hardware activation phrase;

determine characteristics of an environment of the electronic device, the characteristics of the environment including a determination that a user that provided the first speech is looking at or towards the electronic device; and

adjust operation of the electronic device from the active experience mode to a passive experience mode based on the characteristics, the passive experience mode configured to provide a second response to second speech not including the hardware activation phrase.

3. The electronic device of claim 2, wherein the hardware activation phrase indicates an expectation of a user providing the first speech to receive the first response responsive to the first speech

4. The electronic device of claim 2, wherein the electronic device further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

6. The electronic device of claim 2, wherein the processor is configured to execute the instructions such that the processor and memory are configured to:

determine characteristics of the second speech, wherein the adjusting operation of the electronic device from the active experience mode to the passive experience mode is further based on the characteristics of the second speech.

7. The electronic device of claim 2, wherein the characteristics of the environment include activities occurring within the environment.

8. The electronic device of claim 2, wherein the characteristics of the environment include a number of people within the environment.

5. (canceled)

9. A method, comprising:

operating an electronic device in an active experience mode providing a first response to first speech based on the first speech including a hardware activation phrase;

determining, by a processor, characteristics of an environment of the electronic device the characteristics of the environment including a determination that a user that provided the first speech is providing a second speech that is spoken at or towards the electronic device; and

adjusting operation of the electronic device from the active experience mode to a passive experience mode based on the characteristics, the passive experience mode configured to provide a second response to the second speech not including the hardware activation phrase.

10. The method of claim 9, wherein the hardware activation phrase indicates an expectation of a user providing the first speech to receive the first response responsive to the first speech

11. The method of claim 9, wherein the electronic devices further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

12. The method of claim 11, wherein the characteristics of the environment indicate that a user is looking at or towards the electronic device.

13. The method of claim 9, further comprising:

determining characteristics of the second speech, wherein the adjusting operation of the electronic device from the active experience mode to the passive experience mode is further based on the characteristics of the second speech.

14. The method of claim 9, wherein the characteristics of the environment include activities occurring within the environment.

15. The method of claim 9, wherein the characteristics of the environment include a number of people within the environment.

16. A computer program product, comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:

operate the computing device in an active experience mode providing a first response to first speech based on the first speech including a hardware activation phrase, wherein the first response to the first speech is a speech-based response;

determine characteristics of an environment of the computing device; and

adjust operation of the computing device from the active experience mode to a passive experience mode based on the characteristics, the passive experience mode configured to provide a second response to second speech not including the hardware activation phrase, wherein the second response is a visual-based response.

17. The computer program product of claim 16, wherein the hardware activation phrase indicates an expectation of a user providing the first speech to receive the first response responsive to the first speech

18. The computer program product of claim 16, wherein the computing device further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

19. The computer program product of claim 18, wherein the characteristics of the environment indicate that a user is looking at or towards the computing device.

20. The computer program product of claim 16, the computer program instructions cause the one or more computing devices to:

determine characteristics of the second speech, wherein the adjusting operation of the computing device from the active experience mode to the passive experience mode is further based on the characteristics of the second speech.

21. The computer program product of claim 16, wherein the characteristics of the environment include activities occurring within the environment.

22. The computer program product of claim 16, wherein the characteristics of the environment include a number of people within the environment.