Privacy mode for always-on voice-activated information assistant

• US 9,472,206 B2
• Filed: 06/05/2014
• Issued: 10/18/2016
• Est. Priority Date: 06/17/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Ady, Roger W.
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US14/296,545
• Publication Number:
  US 20140372126A1
• Time in Patent Office: 866 Days
• Field of Search: 704/201, 704/270, 704/275, 704/270.1, 704/263, 704/208
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A user device and method discriminately provides audible responses to a voice command received by a user device that supports voice activation. The method includes detecting a first pre-established, audible activation command that activates the user device. In response to detecting the first pre-established, audible activation command, the method includes producing a first audible acknowledgement within loudspeaker proximity of the user device and then monitoring for detection of at least one second, audible acknowledgement produced by another user device within a pre-set time interval, which detection would indicate that the other user device is also responding. The method includes processing and responding to a received audible command in response to not detecting. However, in response to detecting, the method includes triggering entry into a privacy mode of audible command input and producing a privacy mode announcement via at least one of a display and a sound producing component.

Claims

1. A method comprising:

detecting, by a user device, a pre-established, audible activation command that activates the user device;

in response to detecting the pre-established, audible activation command, producing, by the user device, a first audible acknowledgement that is audible at least within a loudspeaker proximity of the user device;

monitoring, by the user device, for detection of at least one second audible acknowledgement produced by another user device within a pre-set time interval;

in response to not detecting the at least one second audible acknowledgement within the pre-set time interval;

processing, by the user device, a received audible command; and

produce, by the user device, a response to the received audible command;

in response to detecting the at least one second audible acknowledgement within the pre-set time interval;

triggering, by the user device, entry into a privacy mode of audible command input; and

producing, by the user device and for output via one or more of a display or a sound producing component, a privacy mode announcement.

2. The method of claim 1, wherein producing the privacy mode announcement comprises indicating that multiple devices were detected and indicating that loudspeaker playback is disabled.

3. The method of claim 1, wherein processing and producing the response to the received audible command comprises:

measuring a volume magnitude of the received audible command;

comparing the volume magnitude to a loudness threshold that is pre-selected to indicate when a user is speaking directly into the user device; and

in response to the volume magnitude exceeding the loudness threshold, processing and producing the response to the received audible command.

4. The method of claim 1, wherein producing the privacy mode announcement comprises:

generating a challenge requesting confirmation that the pre-established, audible activation command originated from an authorized user of the user device;

receiving a confirmation response to the challenge; and

in response to the received confirmation response being verified as a pre-established confirmation response that is assigned to the user device, processing and responding to the received audible command.

5. The method of claim 4, wherein:

generating the challenge requesting confirmation that the first pre-established, audible activation command originated from the authorized user of the user device comprises producing a challenge query that solicits the pre-established confirmation response as an audible response that is detectable at least within the loudspeaker proximity of the user device;

receiving an audible confirmation response to the challenge query; and

verifying that the pre-established, audible activation command originated from the authorized user of the user device by comparing the received audible confirmation response to the pre-established confirmation response for a match.

6. The method of claim 5, wherein the verifying comprises comparing the received audible confirmation response to a pre-selected one of a specific identifier assigned to the user device or a pre-recorded name of the authorized user.

7. The method of claim 4, wherein:

generating the challenge requesting confirmation that the pre-established, audible activation command originated from the authorized user of the user device comprises producing an audible challenge query that is audible within the loudspeaker proximity of the user device;

receiving the confirmation response to the challenge includes detecting a tactile user input at a user interface of the user device; and

the method further comprises verifying that the pre-established, audible activation command originated from the authorized user of the user device based on a user providing the tactile user input within a preset time after producing the audible challenge query.

8. The method of claim 7, wherein:

the tactile user input comprises a pre-established tactile input from among at least one of (i) a selection of a touch screen control, (ii) an actuation of a push button, (iii) a pattern of shaking the user device, or (iv) a tracing of at least one symbol on a touch screen.

9. The method of claim 4, wherein prior to detecting the pre-established, audible activation command, the method further comprises:

receiving a control input at a user interface of the user device to perform one of (a) modifying or (b) adding a pre-established confirmation response assigned to one of the user device or the authorized user;

receiving a new confirmation response from the authorized user; and

updating the pre-established confirmation response to match the new confirmation response.

10. The method of claim 4, wherein:

producing the first audible acknowledgement that is audible at least within the loudspeaker proximity of the user device comprises producing the first audible acknowledgement via a first sound producing component;

monitoring for detection of the at least one second audible acknowledgement produced by another device within the pre-set time interval comprises monitoring, via a first sound receiving component, for detection of the at least one second audible acknowledgement;

the method further comprises;

concurrently monitoring, via a second sound receiving component, for detection of the at least one second audible acknowledgement produced within the pre-set time interval; and

determining that the at least one second audible acknowledgement is from another user device based upon receiving the second audible acknowledgement at both the first sound receiving component and the second sound receiving component with at least one of a loudness variation or a timing delay that indicates a displaced source from the first sound producing component.

11. The method of claim 4, further comprising:

switching at least a portion of a voice-activated information assistant of the user device into a power saving mode while a voice monitor remains activated,

wherein detecting the pre-established, audible activation command comprises detecting the pre-established, audible activation command by the voice monitor, and

wherein producing the first audible acknowledgement comprises that is audible at least within the loudspeaker proximity of the user device comprises, in response to detecting the pre-established, audible activation command, the voice monitor activating the voice-activated information assistant to produce the first audible acknowledgement.

12. A user device, comprising:

at least one sound producing component;

at least one sound receiving component;

a user interface comprising a display;

at least one processor that is communicatively coupled to the user interface, the at least one sound receiving component, and the at least one sound producing component;

a memory configured to store instructions that, when executed by the at least one processor, configure the user device to;

detect, via the at least one sound receiving component, a pre-established, audible activation command that activates the user device;

in response to detecting the pre-established, audible activation command, produce, via the at least one sound producing component, a first audible acknowledgement that is audible at least within a loudspeaker proximity of the user device;

monitor, via the at least one sound receiving component, for detection of at least one second audible acknowledgement produced by another user device within a pre-set time interval;

in response to not detecting the at least one second audible acknowledgement within the pre-set time interval;

process, via the at least one sound receiving component, a received audible command; and

produce a response to the received audible command;

in response to detecting the at least one second audible acknowledgement within the pre-set time interval;

trigger entry into a privacy mode of audible command input; and

produce, for output via one or more of the display or the at least one sound producing component, a privacy mode announcement.

13. The user device of claim 12, wherein the instructions, when executed by the at least one processor, further configure the user device to:

produce the privacy mode announcement by indicating that multiple devices were detected and indicating that loudspeaker playback is disabled; and

process and respond to a subsequently received command in response to receiving a tactile control input to the user interface of the user device.

14. The user device of claim 12, wherein the instructions, when executed by the at least one processor, further configure the user device to:

measure a volume magnitude of the recevied audible command;

compare the volume magnitude to a loudness threshold that is pre-selected to indicate when a user is speaking directly into the user device; and

in response to the volume magnitude exceeding the loudness threshold, process and produce the response to the received audible command.

15. The user device of claim 12, wherein the instructions, when executed by the at least one processor, further configure the user device to:

generate a challenge requesting confirmation that the pre-established, audible activation command originated from an authorized user of the user device;

receive a confirmation response to the challenge; and

in response to the received confirmation response being verified as a pre-established confirmation response that is assigned to the user device, process and respond to the received audible activation command.

16. The user device of claim 15, wherein the instructions, when executed by the at least one processor, further configure the user device to:

generate the challenge requesting confirmation that the first pre-established, audible activation command originated from the authorized user of the user device by producing, via the at least one sound producing component, a challenge query that solicits the pre-established confirmation response as an audible response that is detectable within the loudspeaker proximity of the user device;

receive, via the at least one sound receiving component, an audible confirmation response to the challenge query; and

verify that the pre-established, audible activation command originated from the authorized user of the user device by comparing the received audible confirmation response to the pre-established confirmation response for a match.

17. The user device of claim 16, wherein the instructions, when executed by the at least one processor, further configure the user device to:

verify by comparing the received audible confirmation response to a pre-selected one of a specific identifier assigned to the user device or a pre-recorded name of the authorized user.

18. The user device of claim 15,

wherein the instructions, when executed by the at least one processor, further configure the user device to:

generate the challenge requesting confirmation that the pre-established, audible activation command originated from the authorized user of the user device by producing, via the at least one sound producing component, an audible challenge query that is audible within the loudspeaker proximity of the user device;

receive the confirmation response to the challenge includes detecting a tactile user input at the user interface of the user device; and

verify that the pre-established, audible activation command originated from the authorized user of the user device based on a user providing the tactile user input within a preset time after producing the audible challenge query.

19. The user device of claim 18, wherein:

the tactile user input comprises a pre-established tactile input from among at least one of (i) a selection of a touch screen control, (ii) an actuation of a push button, (iii) a pattern of shaking the user device, or (iv) a tracing of at least one symbol on a touch screen.

20. The user device of claim 15,

wherein prior to the at least one sound receiving component detecting the pre-established, audible activation command, the instructions, when executed by the at least one processor, further configure the user device to:

receive a control input at a user interface of the user device to perform one of (a) modifying or (b) adding a pre-established confirmation response assigned to one of the user device or the authorized user;

receive, via the at least one sound receiving component, a new confirmation response from the authorized user; and

update the pre-established confirmation response to match the new confirmation response.

21. The user device of claim 15, wherein:

the at least one sound producing component comprises a first sound producing component and a second sound producing component;

the at least one sound receiving component comprises a first sound receiving component and a second sound producing component; and

the instructions, when executed by the at least one processor, further configure the user device to;

produce the first audible acknowledgement within the loudspeaker proximity of the user device via the first sound producing component;

monitor, via the first sound receiving component, for detection of the at least one second audible acknowledgement produced by another device within the pre-set time interval;

concurrently monitor, via the second sound receiving component, for detection of the at least one second audible acknowledgement produced within the pre-set time interval; and

determine that the at least one second audible acknowledgement is from another user device based upon receiving the second audible acknowledgement at both the first sound receiving component and the second sound receiving component with at least one of loudness variation or a timing delay that indicates a displaced source from the first sound producing component.

22. The user device of claim 15,

wherein the instructions, when executed by the at least one processor, further the user device to:

switch at least a portion of a voice-activated information assistant of the user device into a power saving mode while a voice monitor remains activated;

detect the pre-established, audible activation command by the voice monitor; and

in response to detecting the pre-established, audible activation command, activate, via the voice monitor, the voice-activated information assistant produces the first audible acknowledgement within the loudspeaker proximity of the user device.