Method and system for detecting objects using far infrared images

• US 8,194,920 B2
• Filed: 02/16/2007
• Issued: 06/05/2012
• Est. Priority Date: 02/16/2007
• Status: Expired due to Fees

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Ebenstein, Samuel, Smith, Gregory, Prakah-Asante, Kwaku, Rodin, Yelena, Zorka, Nicholas
• Primary Examiner(s): Wu, Jingge
• Assistant Examiner(s): YENTRAPATI, AVINASH
• Application Number:
  US11/675,799
• Publication Number:
  US 20080197284A1
• Time in Patent Office: 1,936 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: B60R 21/34 Protecting non-occupants of a vehicle, e.g. pedestrians B60R19/02 takes precedence
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An object detection system is disclosed in at least one embodiment. The system includes a far IR sensor operable to sense thermal radiation of objects and surroundings in a field of view and to generate a far IR image in response thereto, and an image processing device operable to receive and process the far IR image to detect the presence of one or more objects in the field of view. The image processing device can be configured to process the far IR image by generating an initial threshold image based on the far IR image and an initial threshold value, iteratively obtaining a number of successive threshold images based on the far IR image and a number of successively increased threshold values, and determining the presence of one or more objects in the field of view based on the threshold images and threshold values.

Claims

1. An object detection system, the system comprising:

a far IR sensor operable to sense thermal radiation of objects and surroundings in a field of view and to generate a far IR image in response thereto; and

an image processing device operable to receive and process the far IR image, the image processing device is-configured to;

process the far IR image by generating an initial threshold image based on the far IR image and an initial threshold value;

iteratively obtain a number of successive threshold images based on the far IR image and a number of successively increased threshold values, each increase resulting in an image whose pixels represent objects having higher temperatures; and

obtain a number of largest component threshold images based on the number of successive threshold images and the number of successively increased threshold values;

for each of the number of successively increased threshold values, calculate a centroid angle within the field of view from the number of largest component threshold images to obtain a number of centroid angle values;

calculate a plurality of centroid values based on the number of centroid angle values and the number of successively increased threshold values;

calculate a plurality of distances between the plurality of centroid values;

determine a centroid convergence condition as defined by the plurality of distances between the plurality of centroid values, wherein a plurality of distances below a common distance value for successive centroid values signal a convergence by approaching and maintaining a relatively consistent distance as the threshold approaches a maximum; and

determine the presence or absence of an object in the field of view based on the initial threshold image, the number of successive threshold images, the initial threshold value, the successive threshold values, and the centroid convergence condition.

2. The object detection system of claim 1, wherein the image processing device is further configured to perform connected component analysis on each of the initial threshold image and the number of threshold images to obtain the number of largest component threshold images.

3. The object detection system of claim 1, wherein the image processing device is further configured to:

determine a centroid angle convergence position if the centroid convergence condition is satisfied; and

determine the presence or absence of the object in the field of view based on the initial threshold image, the number of successive threshold images, the initial threshold value, the successive threshold values, and the centroid convergence position.

4. The object detection system of claim 3, wherein whether the centroid convergence condition is satisfied is based on the plurality of distances between at least three centroid values.

5. The object detection system of claim 1, wherein the image processing device is further configured to determine the largest component pixel value count for each of the largest component threshold images.

6. The object detection system of claim 5, wherein the image processing device is further configured to determine the presence of a consistency condition defined as a consistent pixel count for the largest component threshold images as a function of the successively increased threshold values by analyzing the largest component pixel count for each of the largest component threshold images as a function of the initial threshold value and the successively incremented threshold values.

7. The object detection system of claim 6, wherein the image processing device is further configured to determine the presence or absence of the one or more objects based on the centroid convergence condition and the consistency condition being satisfied.

8. The object detection system of claim 1, wherein the far IR sensor is a 80×

60 far IR camera.

9. The object detection system of claim 1, wherein the pixel dimensions of the far IR image are 60×

20.

10. The object detection system of claim 1, wherein the one or more detected objects is a thermal object of interest.

11. The object detection system of claim 1, wherein the one or more detected objects is a pedestrian or an animal.

12. The object detection system of claim 1, wherein the far IR sensor is configured to generate a gray scale image.

22. The object detection system of claim 1 wherein the image processing device is further configured to:

for each image having a centroid, determine a distance between the centroids of each image; and

determine if the centroid angle convergence condition is satisfied based on the distance between the centroids.

13. A method of detecting an object, the method comprising:

receiving a far IR image generated by a far IR sensor operable to sense thermal radiation of objects and surroundings in a field of view;

processing the far IR image by generating an initial threshold image based on the far IR image and an initial IR sensitivity threshold value, iteratively obtaining a number of successive threshold images based on the far IR image and a number of successively increased threshold values, each increase resulting in an image whose pixels represent objects having higher temperatures;

obtaining a number of largest component threshold images based on the number of successive threshold images and the number of successively increased threshold values;

for each of the number of successively increased threshold values, calculating a centroid angle within the field of view from the number of largest component threshold images to obtain a number of centroid angle values;

calculating a plurality of centroid values based on the number of centroid angle values and the number of successively increased threshold values;

calculating a plurality of distances between the plurality of centroid values;

determining a centroid convergence condition as defined by the plurality of distances between the plurality of centroid values, wherein a plurality of distances below a common distance value for successive centroid values signal a convergence by approaching and maintaining a relatively consistent distance as the threshold approaches a maximum; and

determining the presence or absence of an one or more objects in the field of view based on the initial threshold image, the number of successive threshold images, the initial threshold value, and the successive threshold values, and the centroid convergence condition.

14. The method of claim 13, further comprising performing connected component analysis on each of the initial threshold image and the number of threshold images to obtain the number of largest component threshold images.

15. The method of claim 13, further comprising:

determining a centroid angle convergence position if the centroid convergence condition is satisfied; and

determining the presence or absence of the object in the field of view based on the initial threshold image, the number of successive threshold images, the initial threshold value, the successive threshold values, and the centroid convergence position.

16. The method of claim 13, wherein whether the centroid convergence condition is satisfied is based on the plurality of distances between at least three centroid values.

17. The method of claim 13, further comprising determining the largest component pixel count for each of the largest component threshold values.

18. The method of claim 17, further comprising determining the presence of a consistency condition defined as a consistent pixel count for the largest component threshold images as a function of the successively increased threshold values by analyzing the largest component pixel count for each of the largest component threshold images as a function of the initial threshold value and the successively incremented threshold values.

19. The method of claim 18, further comprising determining the presence or absence of the object based on the centroid convergence condition and the consistency condition being satisfied.

20. An object detection system, the system comprising:

a far IR sensor operable to sense thermal radiation of objects and surroundings in a field of view and to generate a far IR image in response thereto;

a secondary sensor selected from the group consisting of a vision sensor operable to sense objects and surroundings in the field of view and to generate a vision sensor image in response thereto, and a laser sensor operable to sense objects and surroundings in the field of view and to generate a laser sensor image in response thereto; and

a processing device operable to receive and process the far IR image and the vision sensor image or the laser sensor image to detect the presence or absence of an object in the field of view, the image processing device is configured to;

process the far IR image by generating an initial threshold image based on the far IR image and an initial threshold value;

iteratively obtaining a number of successive threshold images based on the far IR image and a number of successively increased threshold values, each increase resulting in an image whose pixels represent objects having higher temperatures;

perform a comparison of a pixel count of each of the largest component threshold images to the successively increased threshold values;

determine the largest component pixel count for each of the largest component threshold images based on the comparison;

for the largest component threshold images, determine a presence of a consistency condition, including at least a plurality of image centroids having distances below a common distance value for successive iterations of increased threshold values, signaling a convergence by approaching and maintaining a relatively consistent distance as the threshold approaches a maximum, defining consistent largest component pixel counts of the images as a function of the successively increased threshold values; and

determine the presence or absence of the objects in the field of view based on the initial threshold image, the number of successive threshold images, the initial threshold value, the successive threshold values, and the consistency condition.

21. The object detection system of claim 20 wherein the image processing device is further configured to:

if the consistency condition is satisfied, determine a threshold value boundary defining a position of the consistent largest component pixel values; and

determine the presence or absence of the object in the field of view based on the initial threshold image, the number of successive threshold images, the initial threshold value, the successive threshold values, and the threshold value boundary.