METHODS AND VEHICLES FOR PROCESSING VOICE COMMANDS AND MODERATING VEHICLE RESPONSE

• US 20170140757A1
• Filed: 01/31/2017
• Published: 05/18/2017
• Est. Priority Date: 04/22/2011
• Status: Active Grant

• Current Assignee: Emerging Automotive, LLC
• Original Assignee: Emerging Automotive, LLC
• Inventors: Penilla, Angel A., Penilla, Albert S.
• Granted Patent: US 10,424,296 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 16/0373 Voice control in general G10L
• CPC Class Codes: G01C 21/3641 Personalized guidance, e.g. limited guidance on previously travelled routes
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/45 characterised by the type of analysis window
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G10L 25/63 : for estimating an emotional state
• CPC Class Codes: G10L 25/84 : for discriminating voice from noise
• CPC Class Codes: G10L 25/90 : Pitch determination of speech signals
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/306 : User profiles

Abstract

Methods, systems and cloud processing are provided for coordinating and processing user input provided to vehicles during use. One example is a method for processing voice inputs at a vehicle is provided. The method includes determining a profile for a user of the vehicle using electronics of the vehicle. The profile is part of a user account and the vehicle is provided with wireless communication circuitry for accessing a server over a network for exchanging information regarding the vehicle, use of the vehicle and use of the profile. The method includes receiving via a microphone of the vehicle a voice command from the user of the vehicle. The electronics of the vehicle processing the voice command to enable a two-way conversation exchange between the user and the vehicle. The method acts to access, by electronics of the vehicle, data for learned behavior of the user. The learned behavior is associated to the profile for the user. The method processes the voice command to identify a type command. The type of command is one of an instruction to make a change to a setting associated with the vehicle or a request for interfacing with a remote service over the network. The method includes identifying a vehicle response for implementing the type of command identified for the voice command and processing, by the electronics of the vehicle, the vehicle response. The vehicle response is moderated based at least in part on the learned behavior of the user.

Claims

1. A method for processing voice inputs at a vehicle, comprising,

determining a profile for a user of the vehicle using electronics of the vehicle, the profile being part of a user account, the vehicle provided with wireless communication circuitry for accessing a server over a network for exchanging information regarding the vehicle, use of the vehicle and use of the profile;

receiving via a microphone of the vehicle a voice command from the user of the vehicle, the electronics of the vehicle processing the voice command to enable a two-way conversation exchange between the user and the vehicle;

accessing, by electronics of the vehicle, data for learned behavior of the user, the learned behavior being associated to the profile for the user;

processing the voice command to identify a type of command, the type of command being one of an instruction to make a change to a setting associated with the vehicle or a request for interfacing with a remote service over the network;

identifying a vehicle response for implementing the type of command identified for the voice command; and

processing, by the electronics of the vehicle, the vehicle response, the vehicle response being moderated based at least in part on the learned behavior of the user.

2. The method of claim 1, wherein the vehicle response is further based on current use of the vehicle, use of the vehicle includes one or more of geolocation of the vehicle, speed of the vehicle, direction of the vehicle, a route of the vehicle, a time of day, occupants in the vehicle, historical use of the vehicle, online data of the user accessible to the vehicle or server, or combinations of two or more thereof.

3. The method of claim 1, wherein determining the profile for the user includes processing credential data obtained from the user, the credential data used by the electronics of the vehicle to associated the user to the user account and to the profile, the electronics of the vehicle configured to interface over the network with the server for use of the profile and associated updates to the profile as made via use of the vehicle or updates associated with said learned behavior.

4. The method of claim 1, wherein the voice command processed using electronics of the vehicle is at least partially processed by the server, the processing of the voice command includes identifying a tone identifier, the tone identifier used to indicated a mood of the user of the vehicle.

5. The method of claim 4, wherein the vehicle response is further moderated based on the mood of the user, wherein the mood of the user is learned from user actions or reactions monitored over a period of time and from data obtained from sensors of the vehicle.

6. The method of claim 1, wherein the voice command processed using electronics of the vehicle is at least partially processed by the server, the processing of the voice command includes identifying a mood of the user, the mood includes one or more of a normal mood, a frustrated mood, an agitated mood, an upset mood, a hurried mood, an urgency mood, a rushed mood, a stressed mood, a calm mood, a passive mood, a sleepy mood, a happy mood, or an excited mood, or combinations of two or more thereof, and said mood is configured to change over time.

7. The method of claim 1, wherein the voice command processed using electronics of the vehicle is at least partially processed by the server, the processing of the voice command includes identifying a dialect of the user, the dialect includes one or more of a language dialect, a geographic region dialect, an age dialect, or combinations of two or more thereof.

8. The method of claim 1, wherein the user account is accessed using login data at the vehicle, the login data defined by credentials of the user, wherein updates to the profiles is processed periodically or upon login to the vehicle.

9. The method of claim 8, wherein the login data includes one or more of a password, a wireless login code, a biometric identifier, a gesture, or a pairing between a device of the user and logic of the electronics of the vehicle.

10. The method of claim 1, wherein the voice command processed using electronics of the vehicle is at least partially processed by the server, the processing of the voice command includes identifying a physical profile of the user, the physical profile includes a touch characteristic of the user when interfacing with a graphical user interface of the vehicle, such that the vehicle response is further moderated based on the touch characteristic.

11. The method of claim 1, wherein processing the voice command further includes,

capturing an audio sample of the voice command;

processing the audio sample to remove ambient noise;

identifying frequency and/or magnitude characteristics in the audio sample to identify a voice profile for the voice command.

12. The method of claim 1, wherein said leaned behavior are generated based on express inputs by the user and/or learned behavior that is inferred from inputs and/or settings made over time to a vehicle when the user account of the user was active on vehicle, the server stores at least some of the inputs and/or settings made to the vehicle in a database for processing said learned behavior, said processing is configured to use machine learning.

13. The method of claim 1, further comprising,

monitoring a physical characteristic of the user of the vehicle for a period of time that includes when the voice command was received from the user of the vehicle, the physical characteristic that is monitored being used to further moderate the vehicle response.

14. The method of claim 13, wherein the physical characteristic is detected using one or more sensors/devices of the vehicle or devices worn by the user of the vehicle, the physical characteristic includes one or more of heart rate of the user, motions of a body of the user, eye gaze patterns, body heat, stress indicators, voice micro tremors, voice inflections or pitches, emotional sounds, skin temperature, changes in face heat, slurred speech, camera captured facial expressions, touch patterns, retina scan sensing, gesture detection, or a combination of two or more thereof.

15. The method of claim 1, further comprising,

executing a machine learning algorithm by the server to process a history of inputs received from the vehicle for the user when using vehicle via the profile of the user account, the machine learning algorithm used to identify patterns in inputs made by the user based on a context of use of the vehicle, the patterns being used to predict use patterns of the vehicle by the user and moderate at least one aspect of the vehicle response that is responsive to the voice command, the voice command being one of more spoken words that is part of the two-way conversation exchange between the user and the vehicle.

16. The method of claim 15, wherein the context of use of the vehicle includes identifying geolocation data of the vehicle and inputs or actions made that correspond to said geolocation data.

17. The method of claim 15, further comprising,

capturing, via the microphone, a conversation between the user of the vehicle and a passenger of the vehicle, the conversation being analyzed to provide recommendations.

18. The method of claim 1, wherein the voice command processed using electronics of the vehicle is at least partially processed by the server, the voice command being processed to identify a current voice profile associated with a current voice interaction with the vehicle, the voice profile being indicative of a mood of the user, the mood of the user includes one or more of a normal mood, a frustrated mood, an agitated mood, an upset mood, a hurried mood, an urgency mood, a rushed mood, a stressed mood, a calm mood, a passive mood, a sleepy mood, a happy mood, or an excited mood, or combinations of two or more thereof, the moderation of the vehicle response includes changing a speed or type of response rendered by said vehicle to at least partially correlate said vehicle response to the mood identified by the current voice profile.

19. A method for processing voice inputs at a vehicle, comprising,

determining a profile for a user of the vehicle using electronics of the vehicle, the profile being part of a user account, the vehicle provided with wireless communication circuitry for accessing a server over a network for exchanging information regarding the vehicle, use of the vehicle and use of the profile;

receiving via a microphone of the vehicle a voice command from the user of the vehicle, the electronics of the vehicle processing the voice command to enable a two-way conversation exchange between the user and the vehicle;

accessing, by electronics of the vehicle, data for learned behavior of the user, the learned behavior being associated to the profile for the user;

processing the voice command to identify a type of command, the type of command being one of an instruction to make a change to a setting associated with the vehicle or a request for interfacing with a remote service over the network;

identifying a vehicle response for implementing the type of command identified for the voice command; and

processing, by the electronics of the vehicle, the vehicle response, the vehicle response being moderated based at least in part on the learned behavior of the user, the vehicle response includes outputting one or more voice responses via a speaker of said vehicle responsive to one or more voice commands captured by said microphone, the one or more voice commands and said one or more voice responses being part of the two-way conversation exchange between the user and the vehicle, wherein the request for interfacing with the remote service includes accessing a search engine to obtain information to be presented to the user via the vehicle as said vehicle response or provide information to said remote service responsive to said one or more voice commands captured by said microphone;

wherein a machine learning algorithm is executed by the server to process a history of inputs received from the vehicle for the user when using vehicle via the profile of the user account, the machine learning algorithm used to identify patterns in inputs made by the user based on a context of use of the vehicle.

20. The method of claim 19, wherein determining the profile for the user includes processing credential data obtained from the user, the credential data used by the electronics of the vehicle to associated the user to the user account and to the profile, the electronics of the vehicle configured to interface over the network with the server for use of the profile and associated updates to the profile as made via use of the vehicle or updates associated with said learned behavior.