EMOTIVE ADVISORY SYSTEM AND METHOD

• US 20080269958A1
• Filed: 04/28/2008
• Published: 10/30/2008
• Est. Priority Date: 04/26/2007
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Gusikhin, Oleg Yurievitch, Filev, Dimitar Petrov, Syed, Fazal Urrahman, Klampfl, Erica, Schunder, Mark, Macneille, Perry, Giuli, Thomas, Tonshal, Basavaraj, Chon, Yifan
• Granted Patent: US 9,811,935 B2
• Time in Patent Office: Days
• US Class Current: 701/1
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/744 Displaying an avatar, e.g. an animated cartoon character
• CPC Class Codes: B60W 2040/089 Driver voice
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Information about a device may be emotively conveyed to a user of the device. Input indicative of an operating state of the device may be received. The input may be transformed into data representing a simulated emotional state. Data representing an avatar that expresses the simulated emotional state may be generated and displayed. A query from the user regarding the simulated emotional state expressed by the avatar may be received. The query may be responded to.

Claims

1. An emotive advisory system for use by an occupant of an automotive vehicle, the system comprising:

a computer configured to

receive input indicative of an operating state of the vehicle and input indicative of an emotional state of the occupant,

generate (i) data representing an avatar having an appearance and (ii) data representing a spoken statement for the avatar having at least one speech characteristic, the appearance and the at least one speech characteristic conveying a simulated emotional state of the avatar to the occupant, the simulated emotional state depending on at least one of the operating state of the vehicle and the emotional state of the occupant, and

output the data representing the avatar for visual display and the data representing the statement for the avatar for audio play.

2. The system of claim 1 wherein the computer is further configured to receive a query from the occupant regarding the avatar'"'"'s simulated emotional state.

3. The system of claim 1 wherein the computer is further configured to determine whether a predetermined event has occurred and to query the occupant via spoken dialog if the predetermined event has occurred.

4. The system of claim 3 wherein the computer is further configured to receive a spoken response to the query from the occupant.

5. The system of claim 4 wherein the computer is further configured to alter an operating parameter of the vehicle based on the spoken response to the query.

6. The system of claim 1 wherein the input indicative of an emotional state of the occupant is based on at least one of audio and visual characteristics of the occupant.

7. The system of claim 1 wherein the computer is further configured to determine whether a predetermined event has occurred, to access information from sources remote from the vehicle if the predetermined event has occurred and to provide at least a portion of the accessed information to the occupant visually or audibly.

8. The system of claim 2 wherein the computer is further configured to respond to the query.

9. An emotive information system for a user of an automotive vehicle, the system comprising:

at least one sensor configured to detect at least one of audio and visual characteristics of the user;

a computer configured to

receive the detected information,

receive information about an operating state of the vehicle,

generate data representing a simulated emotional state based on at least a portion of the received information,

receive a spoken query from the user regarding the simulated emotional state, and

respond to the spoken query; and

an interface configured to convey the simulated emotional state to the user.

10. The system of claim 9 wherein the simulated emotional state is conveyed to the user via an avatar.

11. The system of claim 10 wherein the computer is further configured to selectively display the avatar based on an operating state of the vehicle.

12. The system of claim 9 wherein the computer is further configured to determine whether a predetermined event has occurred and to initiate a spoken dialog with the user if the predetermined event has occurred.

13. The system of claim 9 wherein the at least one sensor includes an open microphone.

14. The system of claim 9 wherein the at least one sensor is further configured to detect a button press from the user.

15. A method for emotively conveying information to a user of a device comprising:

receiving input indicative of an operating state of the device;

transforming the input into data representing a simulated emotional state;

generating data representing an avatar that expresses the simulated emotional state, the simulated emotional state conveying information to the user about the operating state of the device;

displaying the avatar;

receiving a query from the user regarding the simulated emotional state expressed by the avatar; and

responding to the query.

16. The method of claim 15 wherein the response to the query includes at least one speech characteristic indicative of the simulated emotional state expressed by the avatar.

17. The method of claim 15 further comprising determining whether a predetermined event has occurred and initiating a spoken dialog with the user regarding the operating state of the device if the predetermined event has occurred.

18. The method of claim 17 wherein the spoken dialog includes at least one speech characteristic indicative of the simulated emotional state expressed by the avatar.

19. An emotive information system for a user of an automotive vehicle comprising:

means for detecting an operating state of the vehicle;

means for generating data representing a simulated emotional state based on at least a portion of the detected information; and

means for expressing the simulated emotional state to the user, the simulated emotional state conveying information to the user about the operating state of the vehicle.

20. A method for emotively conveying information to a user of a device comprising:

a step for receiving input indicative of an operating state of the device;

a step for transforming the input into data representing a simulated emotional state;

a step for generating data representing an avatar that expresses the simulated emotional state, the simulated emotional state conveying information to the user about the operating state of the device; and

a step for displaying the avatar.

21. A method for emotively conveying information to a user of an automotive vehicle comprising:

receiving input indicative of an operating state of the vehicle;

transforming the input into data representing a simulated emotional state;

generating data representing a spoken statement that expresses the simulated emotional state, the simulated emotional state conveying information to the user about the operating state of the vehicle;

playing the spoken statement;

receiving a query from the user regarding the simulated emotional state; and

responding to the query.

22. An emotive advisory system for a passenger vehicle comprising:

a microphone;

a speaker;

a data source; and

a computer in communication with the microphone, speaker and data source, the computer configured to

(a) detect an emotional state of a vehicle driver from a combination of the driver'"'"'s voice pattern and dialog semantics received through the microphone, and

(b) generate natural language speech in real time to the driver through the speaker in response to the emotional state and information obtained from the data source.

23. An emotive advisory system for a passenger vehicle comprising:

a microphone;

a vehicle controller; and

a computer in communication with the microphone and vehicle controller, the computer configured to

(a) detect an emotional state of a vehicle driver from a combination of the driver'"'"'s voice pattern and dialog semantics received through the microphone, and

(b) command the vehicle controller to adjust a vehicle function in response to the emotional state and speech from the driver.

24. A computer-implemented method of emotive interaction between a driver of a passenger vehicle and the vehicle in real time, the method comprising:

(a) conducting bi-directional communication with the driver;

(b) detecting an emotional state of the driver from a combination of the driver'"'"'s voice pattern and dialog semantics; and

(c) adjusting a vehicle function in response to the emotional state and responses from the driver.