Emotive advisory system acoustic environment

• US 8,649,533 B2
• Filed: 10/02/2009
• Issued: 02/11/2014
• Est. Priority Date: 10/02/2009
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: MacNeille, Perry Robinson, Gusikhin, Oleg Yurievitch, Theisen, Kacie Alane
• Primary Examiner(s): Ensey, Brian
• Assistant Examiner(s): Yu, Norman
• Application Number:
  US12/572,364
• Publication Number:
  US 20110083075A1
• Time in Patent Office: 1,593 Days
• Field of Search: 381/92, 381/310, 348 1401- 1416
• US Class Current: 381/110
• CPC Class Codes: B60K 2360/11 Instrument graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2360/148 Instrument input by voice
• CPC Class Codes: B60K 2370/11 Graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2370/148 Input by voice
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming

Abstract

An emotive advisory system for use by one or more occupants of an automotive vehicle includes a directional speaker array, and a computer. The computer is configured to determine an audio direction, and output data representing an avatar for visual display. The computer is further configured to output data representing a spoken statement for the avatar for audio play from the speaker array such that the audio from the speaker array is directed in the determined audio direction. A visual appearance of the avatar and the spoken statement for the avatar convey a simulated emotional state.

Claims

1. An emotive advisory system for use by one or more occupants of an automotive vehicle, the system comprising:

a display device;

a directional speaker array; and

a computer configured to

receive input indicative of an operating state of the vehicle and input indicative of an emotional state of an occupant,

generate (i) data representing an avatar having an appearance and (ii) data representing a spoken statement for the avatar having at least one speech characteristic, the appearance and the at least one speech characteristic conveying a simulated emotional state of the avatar to the occupant, the simulated emotional state depending on at least one of the operating state of the vehicle and the emotional state of the occupant,

determine an audio direction,

output the data representing the statement for the avatar for audio play on the directional speaker array such that the audio from the directional speaker array is directed in the determined audio direction, and

output the data representing the avatar for visual display on the display device such that the avatar orientation corresponds to the audio direction, wherein the directional speaker array is located proximate to the display device such that the audio from the directional speaker array appears to the occupant to come from the avatar and wherein the audio direction is determined such that the audio direction and the corresponding avatar orientation direct attention of the occupant to an object by directing the audio and a gaze of the avatar toward the object.

2. The system of claim 1 wherein the directional speaker array comprises:

a sound bar including a plurality of speakers.

3. The system of claim 1 further comprising:

a natural language interface for receiving input in the form of natural language.

4. The system of claim 1 wherein the computer is configured to determine the audio direction such that the audio direction and the corresponding avatar orientation direct the audio and a gaze of the avatar toward a particular occupant being addressed.

5. The system of claim 1 wherein the computer is configured to determine the audio direction such that the audio direction and the corresponding avatar orientation express an emotion by looking down and speaking, or by looking up and speaking at the occupant.

6. The system of claim 1 further comprising:

a plurality of microphones, wherein the computer is configured to locate an occupant who is speaking.

7. The system of claim 6 wherein the computer is configured to determine the audio direction such that the audio direction directs the audio toward the located occupant.

8. The system of claim 6 wherein the computer is configured to perform echo cancellation on signals received at the plurality of microphones.

9. The system of claim 6 wherein the computer is configured to perform noise cancellation on signals received at the plurality of microphones.

10. The system of claim 9 wherein the computer is configured to perform noise cancellation on signals received at the plurality of microphones to enhance sound from the located occupant.

11. The system of claim 1 further comprising:

a plurality of microphones, wherein the computer is configured to locate one or more occupants who are speaking, and to determine which occupants, if any, are speaking to the avatar.

12. The system of claim 1 further comprising:

a plurality of microphones, wherein the computer is configured to perform active noise cancellation based on signals received at the plurality of microphones by modifying audio from the directional speaker array.

13. The system of claim 1 further comprising:

a plurality of microphones, wherein the computer is configured to;

output data representing an impulse sound for audio play on the directional speaker array,

detect the impulse sound at the plurality of microphones, and

calibrate a channel model of transmission between the directional speaker array and the plurality of microphones.

14. The system of claim 1 wherein the computer is configured to determine the audio direction so as to provide an audio effect.

15. The system of claim 14 wherein the computer is configured to provide reverb.

16. An emotive advisory system for use by one or more occupants of an automotive vehicle, the system comprising:

a directional speaker array;

a display device; and

a computer configured to

receive input indicative of an operating state of the vehicle and input indicative of an emotional state of an occupant,

generate (i) data representing an avatar having an appearance and (ii) data representing a spoken statement for the avatar having at least one speech characteristic, the appearance and the at least one speech characteristic conveying a simulated emotional state of the avatar to the occupant, the simulated emotional state depending on at least one of the operating state of the vehicle and the emotional state of the occupant,

determine an audio direction,

output the data representing the statement for the avatar for audio play on the directional speaker array such that the audio from the directional speaker array is directed in the determined audio direction,

output the data representing the avatar for visual display on the display device such that the avatar orientation corresponds to the audio direction, and

locate an object in the vehicle,

wherein the audio direction is determined such that the audio direction and the corresponding avatar orientation direct the audio and a gaze of the avatar toward the object to direct attention of the occupant to the object.

17. A vehicle comprising:

A directional speaker array; and

A computer configured to;

Output data representing a spoken statement for audio play from the speaker array, and

output data representing an avatar for visual display, and

determine an audio direction,

output the data representing the statement for the avatar for audio play on the directional speaker array such that the audio from the directional speaker array is directed in the determined audio direction,

output the data representing the avatar for visual display on the display device such that the avatar orientation corresponds to the audio direction, and

locate an object in the vehicle

wherein the audio direction is determined such that the audio direction and the corresponding avatar orientation direct the audio and a gaze of the avatar toward the object to direct attention of the occupant to the object.