Vehicle mounted image processor and method of use

• US 7,512,494 B2
• Filed: 05/12/2006
• Issued: 03/31/2009
• Est. Priority Date: 05/13/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Nishiuchi, Hidekazu
• Primary Examiner(s): Tran; Khoi H.
• Assistant Examiner(s): MANCHO, RONNIE M
• Application Number:
  US11/432,894
• Publication Number:
  US 20060256198A1
• Time in Patent Office: 1,054 Days
• Field of Search: 701/1, 701/300, 382/154, 382/103, 382/169, 348/208.1, 348/155
• US Class Current: 701/300
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vehicle-mounted detecting device and corresponding method for detecting a distance between the vehicle and an object outside the vehicle. The detecting device extracts edge intersection points from images of individual frames photographed by a camera. A set of edge intersection points for which the relative position in images does not change between frames is then specified as an observation point group; the movement of the specified observation point group in images is observed; and pitching by the vehicle is detected. Any two points with differing heights on an object present in an image are extracted as feature points, and after the effect of pitching is eliminated from the feature points, the position of a vanishing line in the image is calculated based on the ratio of the velocities of the feature points in images after the length between the two feature points and the effects of pitching are eliminated.

Claims

1. An image processor mounted to a vehicle for determining a distance between the vehicle and an object located outside the vehicle, the vehicle including imaging means for capturing images from in front of the vehicle, the image processor comprising:

velocity information calculation means for calculating velocity information from images photographed by the imaging means;

observation point group specification means operable for specifying, from the velocity information calculated by the velocity information calculation means, a set of observation points at which relative positions of velocity information do not change;

pitching detection means configured to detect pitching of the vehicle based on the set specified by the observation point group specification means; and

vanishing line calculation means configured to;

extract any two points of differing height of an object present in an image as feature points, eliminate the pitching detected by the pitching detection means from the feature points, and calculate the position of a vanishing line based on the velocity ratio of the feature points after the length between the two feature points and pitching are eliminated.

2. The image processor of claim 1, wherein the vanishing line calculation means is further configured to eliminate an effect of pitching by subtracting the velocity of the pitching detected by the pitching detection means from a vertically oriented velocity of the feature points in the image.

3. The image processor of claim 2, wherein the velocity information calculation means is configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

4. The image processor of claim 2, wherein the observation point specification means is configured to detect a velocity information set in which a change of vertical position in an image accompanying vehicle travel is less than 1pixel from the velocity information set in which the relative positions of velocity information in an image do not change between the images to specify the set as the observation point group.

5. The image processor of claim 4, wherein the velocity information calculation means is configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

6. The image processor of claim 4, wherein the observation point group specification means is configured to detect the velocity information set in which the change in vertical position orientation in an image accompanying vehicle travel is less than 1pixel while a frame rate of the imaging means is speeded up.

7. The image processor of claim 6, wherein the velocity information calculation means is configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

8. The image processor of claim 1, wherein the velocity information calculation means is configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

9. The image processor of claim 1, wherein the observation point specification means detects a velocity information set in which a change of vertical position in an image accompanying vehicle travel is less than 1pixel from the velocity information set in which the relative positions of velocity information in an image do not change between the images to specify the set as the observation point group.

11. A vehicle comprising the image processor of claim 1, and further comprising:

the imaging means for capturing images from in front of the vehicle; and

wherein the velocity information calculation means, the observation point group specification means, the pitching detection means and the vanishing line calculation means are included in a controller.

12. The vehicle of claim 11, wherein the controller is further configured to eliminate an effect of pitching by subtracting the velocity of the detected pitching from a vertically oriented velocity of the feature points.

13. The vehicle of claim 12, wherein the controller is further configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

14. The vehicle of claim 12, wherein the observation point specification means is configured to detect a velocity information set in which a change of vertical position in an image accompanying vehicle travel is less than 1pixel from the velocity information set in which the relative positions of velocity information in an image do not change between the images to specify the set as the observation point group.

15. The vehicle of claim 14, wherein the velocity information calculation means is configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

16. The vehicle of claim 14, wherein the observation point group specification means is configured to detect the velocity information set in which the change in vertical position orientation in an image accompanying vehicle travel is less than 1pixel while a frame rate of the imaging means is speeded up.

17. The vehicle of claim 16, wherein the velocity information calculation means is configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

18. The vehicle of claim 11, wherein the controller is further configured to perform edge extraction for images in individual frames photographed by the imaging means and extract a point at which an extracted edge in a vertical orientation and an extracted edge in a horizontal orientation intersect, and the edge intersection point that is extracted serves as the velocity information.

19. The vehicle of claim 11, wherein the observation point specification means detects a velocity information set in which a change of vertical position in an image accompanying vehicle travel is less than 1pixel from the velocity information set in which the relative positions of velocity information in an image do not change between the images to specify the set as the observation point group.

10. A method for determining a position of a vanishing line within an image captured by an image processor in a vehicle having an imaging means, the method comprising:

calculating velocity information for images captured by the imaging means;

selecting a set in which the relative positions of velocity information do not change from the calculated velocity information;

detecting pitching by the vehicle based on the specified velocity information set;

extracting as feature points, any two points of differing height on an object present in the image;

eliminating from the feature points the pitching detected by the pitching detection means; and

calculating the position of a vanishing line based on the velocity ratio of the feature points after the length between the two feature points and pitching are eliminated.