SYSTEM AND METHOD FOR SPEECH-ENABLED PERSONALIZED OPERATION OF DEVICES AND SERVICES IN MULTIPLE OPERATING ENVIRONMENTS

• US 20170116986A1
• Filed: 06/19/2015
• Published: 04/27/2017
• Est. Priority Date: 06/19/2014
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH
• Original Assignee: Robert Bosch GmbH
• Inventors: Weng, Fuliang, Shen, Zhongnan, Bei, Gengyan, Schulze, Katrin, Angkititrakul, Pongtep, Xiong, Nikita
• Granted Patent: US 10,410,630 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

A system provides multi-modal user interaction. The system is configured to detect acoustic events to perform context-sensitive personalized conversations with the speaker. Conversation or communication among the speakers or devices is categorized into different classes as confidential, partially anonymous, or public. When exchange with cloud infrastructure is needed, a clear indicator is presented to the speaker via one or more modalities. Furthermore, different dialog strategies are employed in situations where conversation failures, such as misunderstanding, wrong expectation, emotional stress, or memory deficiencies, occur.

Claims

1. A human computer interaction (HCI) system comprising:

at least one user input interface;

at least one output interface;

an automated device;

and a control system operatively connected to the at least one user input interface, the at least one output interface, and the automated device, the control system being configured to;

receive a series of spoken inputs from a user with the at least one input interface;

identify the user using a continuous authentication process with a privacy and security management module in the control system;

predict a level of authority for the user within a hierarchy of a plurality of users of the system;

operate the automated device based on a command from the user in the series of spoken inputs only in response to the level of authority for the user being great enough to authorize the operation of the automated device.

2. The system of claim 1, the control system being further configured to:

identify the user as a child in a family hierarchy;

generate a request message specifying the command from the child for a parent in the family hierarchy with the at least one output interface;

receive a spoken response to the request message from another user;

identify the other user as the parent using the continuous authentication process with the privacy and security management module; and

operate the automated device based on the command from the child only in response to the spoken response from the parent approving the requested command.

3. A human computer interaction (HCI) system comprising:

at least one user input interface;

at least one output interface; and

a control system operatively connected to the at least one user input interface and the at least one output interface, the control system being configured to;

receive spoken input corresponding to a name of a user during a progressive user enrollment process;

generate a phonetic pronunciation sequence corresponding to the name of the user; and

generate an audible output with the at least one output interface including a synthesized version of the name of the user based on the phonetic pronunciation sequence prior to receiving a spelling of the name of the user during the progressive enrollment process.

4. A human computer interaction (HCI) system comprising:

at least one user input interface;

at least one output interface; and

a control system operatively connected to the at least one user input interface and the at least one output interface, the control system being configured to;

receive spoken input from a user with the at least one user input interface;

identify a status of the user with reference to the spoken input from the user; and

update an output dialog to produce an output with the at least one output interface to complement the status of the user.

5. The system of claim 4, the control system being further comprising:

identify the status of the user with reference to a rate of speech for the user input; and

update the output dialog to produce an abbreviated output in response to the rate of speech exceeding a predetermined threshold.

6. A human computer interaction (HCI) system comprising:

at least one user input interface;

at least one output interface; and

a control system operatively connected to the at least one user input interface and the at least one output interface, the control system being configured to;

receive environmental sounds from an environment around the at least one user input interface;

identify an acoustic event based on the environmental sounds using an acoustic event recognition module in the control system; and

generate an output with the at least out output interface including an identification of the environmental sounds based on the identified acoustic event and a request to reduce or eliminate the environmental sound to improve voice input to the at least one user input interface.

7. A human computer interaction (HCI) system comprising:

at least one user input interface;

at least one output interface;

a plurality of devices; and

a control system operatively connected to the at least one user input interface, the at least one output interface, and the plurality of devices, the control system being configured to;

receive a request to add an additional device to the plurality of devices;

receive registration data from the additional device;

generate a dialog with the output interface to request user configuration data for the additional device based on the registration data;

receive input from a user including the user configuration data for the additional device in response to the dialog; and

operate the additional device with the user configuration data.

8. The system of claim 7, the control system being further configured to:

receive a request with the user input interface to remove one device in the plurality of devices from operation; and

delete configuration data corresponding to the one device to remove the one device from operation with the HCI system.

9. A human computer interaction (HCI) system comprising:

at least one user input interface;

at least one output interface; and

a control system operatively connected to the at least one user input interface and the at least one output interface, the control system being configured to;

receive spoken input from a user with the at least one input interface including biometric data in the voice of the user and spoken authentication data;

retrieve with a security management module plurality of key segments for an authentication key corresponding to the user over a network from a plurality of distributed online data storage services;

generate the authentication key using the plurality of key segments; and

accept additional input from the user only in response to the biometric data and spoken authentication data corresponding to the authentication key.