OBJECT RECOGNITION SYSTEM HAVING RADAR AND CAMERA INPUT

• US 20140203959A1
• Filed: 01/18/2013
• Published: 07/24/2014
• Est. Priority Date: 01/18/2013
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: KRIEL, Bradley Scott, Morris, Daniel
• Granted Patent: US 9,052,393 B2
• Time in Patent Office: Days
• US Class Current: 342/52
• CPC Class Codes: G01S 13/726 Multiple target tracking
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/93 for anti-collision purposes
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A object recognition system has a camera configured to generate image data of an environment of a machine and a radar device configured to generate radar data of the environment of the machine. The object recognition system also has a processor configured to detect an object in the environment based on the radar data, map the radar data to a portion of the image data corresponding to the detected object, and classify the detected object using the mapped portion of the image data.

Claims

1. A object recognition system comprising:

a camera configured to generate image data of an environment of a machine;

a radar device configured to generate radar data of the environment of the machine; and

,

a processor configured to;

detect an object in the environment based on the radar data;

map the radar data to a portion of the image data corresponding to the detected object; and

,

classify the detected object using the mapped portion of the image data.

2. The system of claim 1, wherein the processor is configured to classify the detected object as at least one of a person, a vehicle, or equipment.

3. The system of claim 1, wherein the processor is further configured to track the detected object.

4. The system of claim 1, wherein the processor is further configured to determine whether to classify the detected object based at least in part on whether the mapped portion of the image data is similar to image data of a previously tracked object.

5. The system of claim 1, further comprising a display, and wherein the processor is further configured to render the image data on the display.

6. The system of claim 5 wherein the processor is further configured to render a first indication of the detected object when the detected object is classified as a first type of object and a second indication of the detected object when the detected object is classified as a second type of object.

7. The system of claim 6 wherein the first indication is a first text string and the second indication is a second text string.

8. The system of claim 1, wherein the processor is further configured to:

determine a priority for classifying the object based at least in part on the radar data.

9. The system of claim 8, wherein the processor is further configured to:

determine an operational state of the machine;

determine whether the operational state indicates that a collision is likely to occur between the machine and the detected object; and

generate an alert when it is determined that the collision may occur.

10. A method for recognizing objects in an environment of a machine using a radar device and an image device, the method comprising:

accessing radar data of the radar device;

detecting an object in the environment of the machine based on the radar data;

accessing image data of the image device,

mapping the radar data to a portion of the image data corresponding to the detected object; and

,

classifying the detected object using the mapped portion of the image data.

11. The method of claim 10 wherein the detected object is classified as at least one of a person, a vehicle, or equipment.

12. The method of claim 10 further including tracking the detected object.

13. The method of claim 12 further including determining whether to classify the detected object based at least in part on whether the mapped portion of the image data is similar to image data of a previously tracked object.

14. The method of claim 10 further including rendering the image data on a display within the machine.

15. The method of claim 14 wherein the rendering includes providing an indication of the detected object.

16. The method of claim 14 wherein the indication is a first indication when the detected object is classified as a first type of object and a second indication when the detected object is classified as second type of object.

17. The method of claim 16, wherein the first indication is a first text string and the second indication is a second text string.

18. The method of claim 10 further including:

determining a priority for classifying the object based at least in part on the radar data.

19. The method of claim 17, further including:

determining an operational state of the machine;

determining whether the operational state indicates that a collision is likely to occur between the machine and the detected object; and

generating an alert when it is determined that the collision may occur.

20. A mobile machine comprising:

a cabin;

a display disposed within the cabin;

a frame;

a camera connected the frame and configured to generate image data of an environment of the mobile machine;

a radar device connected to the frame and configured to generate radar data of the environment of the mobile machine; and

,

a processor in communication with the camera, the radar device, and the display, the processor configured to;

detect one or more objects in the environment based on the radar data;

map the radar data to a portion of the image data corresponding to the detected object;

determine respective priorities for the one or more objects based at least in part on an operational state of the mobile machine;

classify a selected object of the one or more objects, the selected object having the highest respective priority of the one or more objects, by;

determining whether the selected object is being tracked based at least in part on the mapped portion of the image data for the selected object, and

when the selected object is not being tracked, assigning the selected object a classification; and

,

render the image data, an indication of the selected object, and an indication of the classification of the selected object on the display.