Emotive advisory system and method

• US 9,811,935 B2
• Filed: 04/28/2008
• Issued: 11/07/2017
• Est. Priority Date: 04/26/2007
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Filev, Dimitar Petrov, Gusikhin, Oleg Yurievitch, Klampfl, Erica, Chon, Yifan, Syed, Fazal Urrahman, Macneille, Perry, Schunder, Mark, Giuli, Thomas, Tonshal, Basavaraj
• Primary Examiner(s): MUSTAFA, IMRAN K
• Application Number:
  US12/110,712
• Publication Number:
  US 20080269958A1
• Time in Patent Office: 3,480 Days
• Field of Search: 701 1
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/744 Displaying an avatar, e.g. an animated cartoon character
• CPC Class Codes: B60W 2040/089 Driver voice
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Information about a device may be emotively conveyed to a user of the device. Input indicative of an operating state of the device may be received. The input may be transformed into data representing a simulated emotional state. Data representing an avatar that expresses the simulated emotional state may be generated and displayed. A query from the user regarding the simulated emotional state expressed by the avatar may be received. The query may be responded to.

Claims

1. An emotive advisory system for use by an occupant of an automotive vehicle, the system comprising:

a computer configured to

receive input indicative of an operating state of the vehicle and input indicative of an emotional state of the occupant,

generate data representing an avatar having an appearance and data representing a spoken statement for the avatar having at least one speech characteristic, the appearance and the at least one speech characteristic (i) depending on at least one of the operating state of the vehicle and the emotional state of the occupant and (ii) conveying a simulated emotional state of the avatar to the occupant to prompt the occupant to pose a query regarding a cause of the avatar'"'"'s simulated emotional state,

receive input indicative of a query from the occupant regarding the avatar'"'"'s simulated emotional state,

in response to the query, generate data representing another spoken statement for the avatar and including information about the cause of the avatar'"'"'s simulated emotional state, and

output the data representing the avatar for visual display and the data representing the statements for the avatar for audio play.

2. The system of claim 1 wherein the computer is further configured to determine whether a predetermined event has occurred and to query the occupant via spoken dialog if the predetermined event has occurred.

3. The system of claim 2 wherein the computer is further configured to receive a spoken response from the occupant to the computer'"'"'s query.

4. The system of claim 3 wherein the computer is further configured to alter an operating parameter of the vehicle based on the spoken response to the computer'"'"'s query.

5. The system of claim 1 wherein the input indicative of an emotional state of the occupant is based on at least one of audio and visual characteristics of the occupant.

6. The system of claim 1 wherein the computer is further configured to determine whether a predetermined event has occurred, to access information from sources remote from the vehicle if the predetermined event has occurred and to provide at least a portion of the accessed information to the occupant visually or audibly.

7. An emotive information system for a user of an automotive vehicle, the system comprising:

at least one sensor configured to detect at least one of audio and visual characteristics of the user;

a computer configured to

receive the detected information,

receive information about an operating state of the vehicle, and

generate data representing a simulated emotional state based on at least a portion of the received information; and

an interface configured to convey the simulated emotional state to the user to prompt the user to pose a query regarding a cause of the simulated emotional state, wherein the computer is further configured to receive a query from the user regarding the simulated emotional state and generate data representing a response to the query including information about the cause of the simulated emotional state.

8. The system of claim 7 wherein the simulated emotional state is conveyed to the user via an avatar.

9. The system of claim 8 wherein the computer is further configured to selectively display the avatar based on an operating state of the vehicle.

10. The system of claim 7 wherein the computer is further configured to determine whether a predetermined event has occurred and to initiate a spoken dialog with the user if the predetermined event has occurred.

11. The system of claim 7 wherein the at least one sensor includes an open microphone.

12. The system of claim 7 wherein the at least one sensor is further configured to detect a button press from the user.