Methods and Systems for Communicating Content to Connected Vehicle Users Based Detected Tone/Mood in Voice Input

• US 20160104486A1
• Filed: 11/24/2015
• Published: 04/14/2016
• Est. Priority Date: 04/22/2011
• Status: Active Grant

• Current Assignee: Emerging Automotive, LLC
• Original Assignee: Albert S. Penilla, Angel A. Penilla
• Inventors: Penilla, Angel A., Penilla, Albert S.
• Granted Patent: US 9,493,130 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 16/0373 Voice control in general G10L
• CPC Class Codes: G01C 21/3641 Personalized guidance, e.g. limited guidance on previously travelled routes
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/04 : Training, enrolment or model building
• CPC Class Codes: G10L 17/06 : Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 2015/223 : Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 : of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 : of application context
• CPC Class Codes: G10L 21/0208 : Noise filtering
• CPC Class Codes: G10L 25/45 : characterised by the type of analysis window
• CPC Class Codes: G10L 25/57 : for processing of video signals
• CPC Class Codes: G10L 25/63 : for estimating an emotional state
• CPC Class Codes: G10L 25/84 : for discriminating voice from noise
• CPC Class Codes: G10L 25/90 : Pitch determination of speech signals
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/306 : User profiles

Abstract

Methods, systems and cloud processing are provided for coordinating and processing user input provided to vehicles during use. One example is for processing voice inputs at a vehicle to identify a mood of a user and then modifying or customizing the vehicle response based on the detected mood, physical characteristic and/or physiological characteristic of the user. One example includes sending, to a cloud processing server, data from the vehicle. The vehicle includes an on-board computer for processing instructions for the vehicle and processing wireless communication to exchange data with the cloud processing server. The method then receives, at the vehicle, data for a user account to use the vehicle. The cloud processing server uses the user account to identify a user profile of a user. Then, receiving, from the cloud processing server, voice profiles for the user profile. Each voice profile is associated with a tone identifier or audio signature. The voice profiles for the user are learned from a plurality of voice inputs made to the vehicle by the user in one or more prior sessions of use of the vehicle. The method further includes receiving, by on-board computer, a voice input. For the voice input, processing, by on-board computer, the voice input to identify a voice profile that is correlated to the voice input and generating a vehicle response for the voice input. The vehicle response is moderated based on the tone identifier of the identified voice profile. In one example, the tone identifier is used to infer a mood of the user, and the moderation of the vehicle response assists in selecting a type of response by the vehicle and/or setting made to a vehicle system.

Claims

10. A method executed by a processor of electronics of a vehicle, comprising,

receiving voice input of a user at a microphone of the vehicle to command input or request information from a voice input interface of the vehicle, the voice input interface being in communication with the microphone;

analyzing the voice input of the user by electronics of the vehicle to identify a voice profile for the voice input, the voice profile being associated with a tone identifier;

processing the tone identifier by electronics of the vehicle to select a vehicle response from among a plurality of vehicle responses for the voice input, the selection of the vehicle response using a mapping of tone identifiers to select ones of the plurality of vehicle responses,

the mapping being customized for the user, and the mapping being stored in association with a user account on the vehicle and on storage of a cloud processing system,

the vehicle having electronics for communicating with the cloud processing system for managing settings in the user account, the settings including the mapping, the electronics of the vehicle is configured to send data regarding the user responses to the cloud processing system for processing, and the mapping of tone identifiers to select ones of the plurality of vehicle responses being updated based on one or more processed user responses that are responsive to one or more of the plurality of vehicle responses returned for one or more voice inputs over time, wherein the processed user responses are analyzed by the cloud processing system to identify which of the plurality of vehicle responses should be updated in the mapping to improve the customization for the user;

receiving, by the electronics of the vehicle from the cloud processing system updates to the settings for the user account, the updates to the settings including updates to the mapping based on the one or more voice inputs over time; and

processing, by the electronics of the vehicle, the updates to the mapping to select a vehicle response for the voice input, the vehicle response is configured to make an operational setting for a vehicle system function on the vehicle.

1. A method for processing voice inputs at a vehicle, comprising,

(a) sending, to a cloud processing server, data from the vehicle, the vehicle including an on-board computer for processing instructions for the vehicle and processing wireless communication to exchange data with the cloud processing server;

(b) receiving, at the vehicle, data for a user account to use the vehicle, the cloud processing server uses the user account to identify a user profile of a user;

(c) receiving, from the cloud processing server, voice profiles for the user profile, each voice profile being associated with a tone identifier, the voice profiles for the user being learned from a plurality of voice inputs made to the vehicle by the user in one or more prior sessions of use of the vehicle;

(d) receiving, by on-board computer, a voice input, and for the voice input,

(i) processing, by on-board computer, the voice input to identify a voice profile that is correlated to the voice input;

(ii) generating a vehicle response for the voice input, the vehicle response is moderated based on the tone identifier of the identified voice profile.

2. The method of claim 1, wherein the tone identifiers identify a mood of the user.

3. The method of claim 2, wherein the mood of the user includes one or more of a normal mood, a frustrated mood, an agitated mood, an upset mood, a hurried mood, an urgency mood, a rushed mood, a stressed mood, a calm mood, a passive mood, a sleepy mood, a happy mood, or an excited mood, or combinations of two or more thereof.

4. The method of claim 1, wherein the tone identifiers identify a dialect of the user.

5. The method of claim 4, wherein the dialect of the user includes one or more of a language dialect, a country dialect, a geographic region dialect, an age dialect, or combinations of two or more thereof.

6. The method of claim 1, wherein the user account is accessed using login data at the vehicle, the login data defined by credentials of the user.

7. The method of claim 6, wherein the login data includes one or more of a password, a wireless login code, a biometric identifier, a gesture, or a pairing between a device of the user and logic of the on-board computer.

8. The method of claim 1, wherein the on-board computer is configured select the vehicle response from among a plurality of alternate vehicle response for the voice input as defined by predefined mapping between the tone identifier and the plurality of alternate vehicle response.

9. The method of claim 1, wherein the voice profiles and associated tone identifiers are stored in storage of the cloud processing server and a copy of select ones of the voice profiles and associated tone identifiers is stored in storage of the vehicle, wherein updates to the voice profiles and tone identifiers is processed periodically or upon login by the user profile at the vehicle.

11. The method of claim 10, wherein a plurality of voice profiles are associated with the user.

12. The method of claim 10, wherein receiving the voice input further includes,

capturing an audio sample of the voice input;

processing the audio sample to remove ambient noise;

identifying frequency and magnitude characteristics in the audio sample

generating signature markers for the audio sample based on the frequency and magnitude characteristics.

13. The method of claim 12, wherein the analyzing of the voice input to identify voice profiles includes,

filtering the voice profiles associated with the user account of the user to find a voice profile having a threshold level of signature markers to the audio sample associated with the voice input.

14. The method of claim 12, wherein the tone identifiers identify a mood of the user.

15. The method of claim 14, wherein the mood of the user includes one or more of a normal mood, a frustrated mood, an agitated mood, an upset mood, a hurried mood, an urgency mood, a rushed mood, a stressed mood, a calm mood, a passive mood, a sleepy mood, a happy mood, an excited mood, or combinations of two or more thereof.

16. The method of claim 10, wherein the voice profile use based on the tone assists in moderating when to provide additional information to the user or when to avoid providing additional information to the user.

17. The method of claim 10, wherein the tone identifier is associated with an audio signature.

18. The method of claim 10, wherein the mapping is obtained from a user'"'"'s data graph that is constructed from learning.

14-1. The method of claim 10, wherein the tone identifies when additional information is to be provided or not provided to the user based on the tone.

15-1. The method of claim 14, wherein the additional information includes advertising data, or discount data, or related information that is in addition to the requested information or instructed command found in the voice input.

19. A vehicle including an on-board computer for processing voice input, comprising,

a microphone interfaced with the on-board computer;

cache memory for storing a sample of audio data received from the microphone, the audio data being a voice input directed to the vehicle; and

a processor of the on-board computer configured process the sample of audio data to identify markers in frequency and magnitude, the markers used to define an audio signature for the voice input, the audio signature being used to identify a voice profile, the voice profile used to identify a vehicle response for the voice input, the voice profile is associated with tone of voice used in the voice input,

wherein the vehicle response acts to direct a vehicle system function to take an action based on the voice input;

wherein the vehicle response is moderated based on the tone of voice.

20. The vehicle of claim 19,

wherein the voice profile is one of a plurality of voice profiles of a user associated with predefined tones of voice, and each voice profile is mapped to select vehicle responses;

wherein the tones of voice are associated to moods of the user include one or more of a normal mood, a frustrated mood, an agitated mood, an upset mood, a hurried mood, an urgency mood, a rushed mood, a stressed mood, a calm mood, a passive mood, a sleepy mood, a happy mood, or an excited mood, or combinations of two or more thereof.