Sensor fusion system architecture

• US 20040167740A1
• Filed: 02/26/2004
• Published: 08/26/2004
• Est. Priority Date: 03/21/2002
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Skrbina, David, Kronen, Stanley Thomas
• Granted Patent: US 6,889,171 B2
• Time in Patent Office: Days
• US Class Current: 702/127
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 in the back of the vehicles
• CPC Class Codes: G01S 2013/93274 on the side of the vehicles

Abstract

A sensor system for generating information useful for determining the environment surrounding a vehicle includes multiple sensing zones, with each zone using sensors operating in different electromagnetic bands. The outputs of the sensors are fused to provide more useful information than that obtainable from either sensor, taken alone.

Claims

1. A vehicle sensor fusion architecture including:

A plurality of paired input sensors for sensing vehicle environment, each pair including one sensor from a first band of the electromagnetic spectrum and a second sensor from a second band of the electromagnetic spectrum and each sensor providing an electrical sensor output indicative of the vehicle environment;

A time tagging module for tagging each electrical sensor output with the time the sensor information was gathered; and

A central fusion module coupled to receive said sensor outputs from a paired input sensor and having a logic capability to process the paired input sensor output to generate a fusion output for providing information characterizing the vehicle environment.

2. A vehicle sensor fusion architecture as recited in claim 1 wherein said input sensors comprise a visual camera and a radar sensor.

3. A vehicle sensor fusion architecture as recited in claim 1 wherein said central fusion module comprises logic means for combining a plurality of paired sensor inputs from a plurality of signals generated by said paired input sensors.

4. A vehicle sensor fusion architecture as recited in claim 3 further comprising a time tagging module so that information data can be extrapolated and interpolated to the same point in time for a fusion action combining information to take place.

5. A vehicle sensor fusion architecture as recited in claim 4 further comprising a current track file generator, and a comparator for comparing the time tagged data to the current track file data, and an updating means for updating the track files if a match is found.

6. A method for generating information characterizing the environment around a vehicle comprising the steps of:

Designating a plurality of information gathering zones around the vehicle;

Selecting a first sensor type from a first band portion of the electromagnetic spectrum and selecting a second sensor type from a second bank of the electromagnetic spectrum;

Deploying at least one A sensor from the first sensor type and one B sensor from the second sensor type in each information gathering zone;

Time tagging information generated from the A and B sensors so as to indicate when the information characterizing the ambient was gathered;

Selecting information A (t1), characteristic of a time t1, from sensor A;

Selecting information B (t2), characteristic of a time t2, from sensor B which corresponds to substantially the same time as information A (t1); and

Fusing information A (t1) and B (t2) so as to characterize a target in the environment of the vehicle.

7. A method for generating information as recited in claim 6 further comprising the steps of:

Selecting from the steps of interpolating and extrapolating the information of B (t2);

Interpolating the information of B (t2) to be at the same time tag moment as A (t1); and

Extrapolating the information of B (t2) to be at the same time tag moment as A (t1).

8. A method for generating information characterizing the environment, including a target, around a motor vehicle including the steps of:

Designating four information gathering zones A, B C, and D around the vehicle;

Selecting a radar sensor type and selecting a camera sensor type;

Deploying one radar sensor and one camera sensor in each information-gathering zone A, B, C, and D;

Time tagging information generated from the radar and camera sensors so as to indicate when the information characterizing the target was gathered;

Selecting information A (t) from the radar sensor;

Selecting information B (t) from the camera sensor which corresponds in substantially the same time to information A (t); and

Fusing information A (t) and B (t) so as to characterize the target in the environment of the vehicle.

9. A method for generating information as recited in claim 8 further comprising the steps of:

Selecting from the steps of interpolating and extrapolating the information of B (t);

Interpolating the information of B (t) to be at the same time tag moment as A (t); and

Extrapolating the information of B (t) to be at the same time tag moment as A (t).