Method and system for sensor signal fusion

• US 7,266,477 B2
• Filed: 06/22/2005
• Issued: 09/04/2007
• Est. Priority Date: 06/22/2005
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Foessel, Alex Dietrich
• Primary Examiner(s): Cosimano; Edward R
• Application Number:
  US11/158,546
• Publication Number:
  US 20070005306A1
• Time in Patent Office: 804 Days
• Field of Search: 340/436, 340/945, 340/952, 340/954, 340/961, 340/988, 342/29, 342/52, 342/53, 342/54, 342/56, 342/159, 382/100, 382/103, 701/1, 701/300, 701/301, 702/127, 702/150, 702/189, 702/190, 702/191
• US Class Current: 702/189
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G05D 1/028 using a RF signal

Abstract

The first sensor collects a first group of first sensor readings of first fixed size dependent on a range of a first sensor and safeguarding requirements of a vehicle. A second sensor collects a second group of second sensor readings of second fixed size dependent on a range of a second sensor and the safeguarding requirements. A reference frame manager references the first group and the second group of readings to a vehicle coordinate reference frame. An integration module establishes a composite grid referenced to the ground based on a spatially aligned integration of the readings of the first group and the second group. A update module refreshes the composite grid upon material movement of the vehicle such that the global occupancy of each cell in the composite grid varies in accordance with the particular location of the vehicle on the terrain.

Claims

1. A method for detecting an obstacle based on sensor fusion, the method comprising:

collecting a first group of first sensor readings of a first size by transmitting a first electromagnetic signal, capable of reflection from an obstacle, from a vehicle at a transmission time and by receiving a reflection of the first electromagnetic signal from the obstacle at the vehicle at a reception time, the first sensor readings based on an elapsed time between the transmission time and the reception time;

collecting a second group of second sensor readings of a second size using a second electromagnetic signal or an ultrasonic signal;

referencing the first group and the second group of readings to a vehicle coordinate reference frame;

establishing a composite grid referenced to vehicle coordinate reference frame or ground based on a spatially and temporally aligned integration of the readings of the first group and the second group, the composite grid comprising a multi-dimensional array of cells, each cell associated with an overall state of occupancy; and

refreshing the composite grid for obstacle detection upon the sooner of a regular update interval or material movement of the vehicle such that the overall state of each cell in the composite grid varies in accordance with the particular location of the vehicle in the environment.

2. The method according to claim 1 further comprising:

determining whether material movement of the vehicle is detected by a location-determining receiver with a dilution of precision as the desired degree of reliability; and

refreshing the composite grid in accordance with the material movement detected by the location-determining receiver if position data of the location-determining receiver meets or exceeds a desired degree of reliability.

3. The method according to claim 1 further comprising:

determining whether material movement of the vehicle is detected by a location-determining receiver with a desired degree of reliability; and

refreshing the composite grid in accordance with the material movement detected by at least one of an odometer, a dead-reckoning system and an accelerometer if position data of the location-determining receiver fails to meet a desired degree of reliability.

4. The method according to claim 1 wherein the establishing further comprises:

determining a first mapping of the first group of first sensor readings with respect to the composite occupancy grid;

determining a second mapping of the second group of second sensor readings with respect to the composite occupancy grid; and

establishing a composite grid reference to the vehicle coordinate reference frame or a ground reference frame, where the composite grid is based on the first mapping and the second mapping.

5. The method according to claim 1 wherein the first mapping and the second mapping are determined during initialization of a data processing system.

6. The method according to claim 1 further comprising:

maintaining a current vehicular heading via a steering system of the vehicle if an obstacle is present in a first radial zone based on the composite grid.

7. The method according to claim 1 further comprising:

decreasing a vehicular speed or velocity of the vehicle to a target speed or velocity via at least one of a propulsion system and a braking system of the vehicle if an obstacle is present within a second radial zone associated with the vehicle based on the composite grid;

the second radial zone having a lesser radius from the vehicle than a first radial zone.

8. The method according to claim 1 further comprising:

stopping movement of the vehicle via a braking system of the vehicle if an obstacle is present within a third radial zone associated with the vehicle based on the composite grid or if a current vehicular direction qualifies as an actual or potential collision course with the obstacle;

the third radial zone having a lesser radius than a first radial zone and a second radial zone.

9. The method according to claim 1 wherein the material movement comprises a movement of the vehicle that is greater than or equal to some fraction of a cell width, cell height, or another cell dimension.

10. The method according to claim 1 wherein the refreshed composite grid comprises output data for control or guidance of the vehicle by a vehicle data processing system or a driver of the vehicle.

11. The method according to claim 1 wherein the material movement is determined with a corresponding desired degree of reliability.

12. The method according to claim 1 wherein the composite grid comprises a composite occupancy grid and wherein the overall state comprises at least one of an occupied state, an unoccupied state, a probability of occupancy, and a probability of non-occupancy.

13. The method according to claim 12 further comprising:

determining a guidance plan based the presence of one or more obstacles in accordance with the composite grid.

14. A system for detecting an obstacle based on sensor fusion, the system comprising:

a first sensor for collecting a first group of first sensor readings of a first size by transmitting a first electromagnetic signal, capable of reflection from an obstacle, from a vehicle at a transmission time and by receiving a reflection of the first electromagnetic signal from the obstacle at the vehicle at a reception time, the first sensor readings based on an elapsed time between the transmission time and the reception time;

a second sensor for collecting a second group of second sensor readings of a second size using a second electromagnetic signal or an ultrasonic signal;

a reference frame manager for referencing the first group and the second group of readings to a vehicle coordinate reference frame;

an integration module for establishing a composite grid referenced to vehicle coordinate reference frame or ground based on a spatially and temporally aligned integration of the readings of the first group and the second group, the composite grid comprising a multi-dimensional array of cells, each cell associated with an overall state of occupancy; and

an update module for refreshing the composite grid for obstacle detection upon the sooner of a regular update interval or material movement of the vehicle such that the overall state of each cell in the composite grid varies in accordance with the particular location of the vehicle in the environment.

15. The system according to claim 14 further comprising a location-determining receiver for determining position data associated with the vehicle;

wherein the update module is arranged to refresh the composite grid in accordance with the material movement detected by the location-determining receiver if position data of the location-determining receiver meets or exceeds a desired degree of reliability.

16. The system according to claim 14 wherein the update module is arranged to refresh the composite grid in accordance with the material movement detected by at least one of an odometer, a dead-reckoning system and an accelerometer if position data of the location-determining receiver fails to meet a desired degree of reliability.

17. The system according to claim 14 further comprising:

a steering system of the vehicle;

a guidance module for maintaining current vehicular heading via the steering system if an obstacle is present in a first radial zone based on the composite grid.

18. The system according to claim 14 further comprising:

a propulsion system of the vehicle;

a braking system of the vehicle;

a guidance module for decreasing a vehicular speed or velocity of the vehicle to a target speed or velocity via at least one of the propulsion system and the braking system if an obstacle is present within a second radial zone associated with the vehicle based on the composite grid;

the second radial zone having a lesser radius from the vehicle than a first radial zone.

19. The system according to claim 14 further comprising:

a braking system of the vehicle;

a guidance module for stopping movement of the vehicle via the breaking system if an obstacle is present within a third radial zone associated with the vehicle based on the composite grid or if a current vehicular direction qualifies as an actual or potential collision course with the obstacle;

the third radial zone having a lesser radius than a first radial zone and a second radial zone.

20. The system according to claim 14 wherein the material movement comprises a movement of the vehicle that is greater than or equal to some fraction of a cell width, cell height, or another cell dimension.

21. The system according to claim 14 wherein the refreshed composite grid comprises output data for control or guidance of the vehicle by a vehicle data processing system or a driver of a vehicle.

22. The system according to claim 14 wherein the material movement is determined with a corresponding desired degree of reliability.

23. The system according to claim 14 wherein the composite grid comprises a composite occupancy grid and wherein the overall state comprises at least one of an occupied state, an unoccupied state, a probability of occupancy, and a probability of non-occupancy.

24. The system according to claim 23 further comprising:

a guidance module for determining a guidance plan based the presence of one or more obstacles in accordance with the composite grid.

25. The system according to claim 14 wherein the integration module determines a first mapping of the first group of first sensor readings with respect to the composite occupancy grid and a second mapping of the second group of second sensor readings with respect to the composite occupancy grid.

26. The system according to claim 25 wherein the first mapping and the second mapping are determined during initialization of the system.