Object recognition system

• US 6,590,521 B1
• Filed: 10/18/2000
• Issued: 07/08/2003
• Est. Priority Date: 11/04/1999
• Status: Expired due to Fees

• Current Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Inoue, Shigeru, Izawa, Hisaya, Saka, Masakazu, Aoki, Tomoyoshi
• Primary Examiner(s): Gregory, Bernarr E.
• Application Number:
  US09/691,614
• Time in Patent Office: 993 Days
• Field of Search: 342/70, 342/71, 342/72, 342/89, 342/90, 342/175, 342/192-197, 342/52-59, 340/901, 340/902, 340/903, 701/23, 701/24-28, 701/300, 701/301, 701/302, 356/4.01, 356/5.01, 356/5-2-
• US Class Current: 342/70
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 2013/932 using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/9325 for inter-vehicle distance regulation, e.g. navigating in platoons
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An object recognition system including a radar, an image sensor and a controller is provided. The radar determines the position of an object, and the image sensor captures an image of the object. The controller sets a processing area within the image captured by the image sensor based on the position of the object determined by the radar and a predetermined size for the object to be recognized. The controller extracts horizontal and vertical edges from the processing area, and preferably judges whether each of the extracted edges belongs to the object based on characteristics of the object to be recognized. The controller then recognizes the outline of the object based on the edges judged to belong to the object. The object can be recognized by determining upper, lower, left and right ends of the object. On the other hand, the controller recognizes lane lines defining the lane in which the vehicle mounting the system of the invention is running. Thus, the relative position of the object to the lane lines is determined by comparing the intersections between a horizontal line corresponding to the lower end of the object and the lane lines with the left and right ends of the object.

Claims

1. An object recognition system comprising:

a radar for determining a position of an object;

an image sensor for capturing an image of the object; and

a controller programmed to position a processing area within the image captured by the image sensor with respect to the position of the object determined by the radar, to recognize the outline of the object based on edges extracted from the processing area, to recognize lane lines defining a lane in which the vehicle mounting the system is running, and to determine a relative position of the object to the lane lines based on the outline of the object and the recognized lane lines,

wherein the size of the processing area is predetermined to adapt to a possible width and height of the object to be recognized.

2. The system of claim 1, wherein the outline of the object is recognized by determining upper, lower, left, and right ends of the object.

3. The system of claim 2, wherein the relative position is determined by comparing the intersections between a horizontal line corresponding to the lower end of the object and the lane lines with the left and right ends of the object.

4. The system of claim 3, wherein the edges comprises horizontal edges and vertical edges.

5. The system of claim 4, wherein the upper and lower ends of the object are determined from the horizontal edges, and the left and right ends of the object are determined from the vertical edges.

6. The system of claim 1, said controller is further programmed to detect a road area by examining a change in intensity for pixels within the image;

and

wherein the lane lines are determined from an outline of the detected road area.

7. The system of claim 1, wherein the size of the processing area is predetermined allowing for a margin.

8. The method for recognizing an object comprising steps of:

determining a position of the object;

capturing an image of the object;

positioning a processing area within the image with respect to the determined position of the object;

extracting edges from the processing area;

recognizing an outline of the object based on the extracted edges;

recognizing lane lines defining a lane in which the vehicle implementing the method is running; and

determining a relative position of the object to the lane lines based on the recognized outline of the object and the recognized lane lines,

wherein the size of the processing area is predetermined to adapt to a possible width and height of the object to be recognized.

9. The method of claim 8, further comprising a step of determining upper, lower, left, and right ends of the object;

and

wherein the outline of the object is recognized by determined upper, lower, left, and right ends of the object.

10. The method of claim 9, further comprising a step of comparing the intersections between a horizontal line corresponding to the lower end of the object and the lane lines with the left and right ends of the object;

and

wherein the relative position is determined based on the result of the comparison.

11. The method of claim 10, wherein the edges comprises horizontal edges and vertical edges.

12. The method of claim 11, wherein the upper and lower ends of the object are determined from the horizontal edges, and the left and right ends are determined from the vertical edges.

13. The method of claim 12, further comprising a step of judging whether each of the extracted horizontal and vertical edges belongs to the object based on characteristics of the object to be recognized;

and

wherein the step of recognizing the outline of the object is performed on the horizontal and vertical edges judged to belong to the object.

14. The method of claim 8, further comprising a step of detecting a road area by examining a change in intensity for pixels within the image;

and

wherein the lane lines are determined from an outline of the detected road area.

15. An object recognition system comprising:

a radar for determining a position of an object;

an image sensor for capturing an image of the object;

means for positioning a processing area within the image with respect to the position determined by the radar;

means for extracting edges from the processing area;

means for recognizing an outline of the object based on the extracted edges;

means for recognizing lane lines defining a lane in which the vehicle mounting the system is running; and

means for determining a relative position of the object to the lane lines based on the recognized outline of the object and the recognized lane lines,

wherein the size of the processing area is predetermined to adapt to a possible width and height of the object to be recognized.

16. The system of claim 15, wherein the outline of the object is recognized by determining upper, lower, left, and right ends of the object.

17. The system of claim 16, wherein the relative position is determined by comparing the intersections between a horizontal line corresponding the lower end of the object and the lane lines with the left and right ends of the object.

18. The system of claim 17, wherein the edges comprises horizontal edges and vertical edges.

19. The system of claim 18, wherein the upper and lower ends are determined from the horizontal edges, and the left and right ends are determined from the vertical edges.

20. The system of claim 15, further comprising means for detecting a road area by examining a change in intensity for pixels within the image;

and

wherein the lane lines are determined from the outline of the detected road area.