Image processing apparatus and method

• US 20030016287A1
• Filed: 07/17/2002
• Published: 01/23/2003
• Est. Priority Date: 07/18/2001
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Kubota, Susumu, Taniguchi, Yasuhiro, Nakayama, Hiromitsu
• Granted Patent: US 7,095,432 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation

Abstract

An image input unit inputs a plurality of images in a time series. Each input image includes a boundary line extending toward a vanishing point on a plane. A reverse projection image generation unit generates a reverse projection image of each input image by projecting information related to each input image onto the plane. A boundary line detection unit detects the boundary line from the reverse projection image by identifying a boundary line candidate on each reverse projection image.

Claims

1. Image processing apparatus, comprising:

an image input unit configured to input a plurality of images in a time series, each input image including a boundary line extending toward a vanishing point on a plane;

a reverse projection image generation unit configured to generate a reverse projection image of each input image by projecting information related to each input image onto the plane; and

a boundary line detection unit configured to detect the boundary line from the reverse projection image by identifying a boundary line candidate on the reverse projection image.

2. The image processing apparatus according to claim 1,

wherein said image input unit is installed on a vehicle moving relative to the plane and inputs each image as the vehicle moves in an advance direction.

3. The image processing apparatus according to claim 2,

wherein the boundary line is a white line and the plane is a road surface, and

wherein each input image includes at least two high contrast lines on the road plane, the two high contrast lines are a left high contrast line and a right high contrast line relative to the vehicle'"'"'s movement.

4. The image processing apparatus according to claim 1,

wherein the projected information comprises at least one of edge information and multivalue information, the edge information and multivalue information each relating to at least one pixel of the input image onto the plane.

5. The image processing apparatus according to claim 4,

wherein said reverse projection image generation unit calculates a differential vector of an intensity value of the pixel on the input image, calculates a normal vector of the pixel for a direction of the boundary line passing through the pixel, calculates an inner product value between the differential vector and the normal vector of the pixel on the input image, and projects the inner product value of the pixel onto the plane.

6. The image processing apparatus according to claim 5,

wherein said reverse projection image generation unit determines the direction of the boundary line passing through the pixel by using a curvature of the boundary line detected from the previous input image in the time series to calculate the normal vector.

7. The image processing apparatus according to claim 5,

wherein said reverse projection image generation unit calculates the inner product value between the differential vector and the normal vector for a left side edge and a right side edge of the boundary line.

8. The image processing apparatus according to claim 7,

wherein said reverse projection image generation unit sets an upper limit line and a lower limit line to define a reverse projection conversion area on the input image, and projects the inner product value of each of the pixels included in the reverse projection area onto the plane.

9. The image processing apparatus according to claim 7,

wherein said boundary line detection unit detects a first segment by connecting at least two positive peaks of the inner product value on the reverse projection image, detects a second segment by connecting at least two negative peaks of the inner product value on the reverse projection image, and extracts the first segment and the second segment as a first boundary line candidate segment from the reverse projection image.

10. The image processing apparatus according to claim 9,

wherein said boundary line detection unit identifies a second boundary line candidate segment having at least one similar attribute with the first boundary line candidate segment, and registers the similar attribute of the first and second boundary line candidate segments as a boundary line candidate in a boundary line candidate table.

11. The image processing apparatus according to claim 10,

wherein the registered attribute comprises at least one of an average edge intensity, a line width, a horizontal position having at least a starting point and an end point, a continuance time as the number of frames in which the boundary line candidate is continuously detected, a detection time as the number of frames passed since the boundary line candidate is first detected, and a disappearance time as the number of frames passed since the boundary line candidate is last detected.

12. The image processing apparatus according to claim 11,

wherein said boundary line detection unit decides that a new boundary line candidate segment detected from the reverse projection image is the same as the boundary line candidate registered in the boundary line candidate table if an attribute of the new boundary line candidate segment is similar to the registered attribute of the boundary line candidate, and updates the registered attribute of the boundary line candidate in the boundary line candidate table according to the attribute of the new boundary line candidate segment.

13. The image processing apparatus according to claim 12,

wherein said boundary line detection unit selects the boundary line as a left white line and a second boundary line as a right white line from the boundary line candidates registered in the boundary line candidate table according to the registered attribute of each boundary line candidate.

14. The image processing apparatus according to claim 13,

wherein said boundary line detection unit calculates the curvature of each of the two boundary lines, and extrapolates a depth part of each of the two boundary lines on the reverse projection image by using at least one of a circular arc and a parabola of the curvature.

15. The image processing apparatus according to claim 14,

wherein said boundary line detection unit projects each pixel of the two boundary lines onto the camera image, and determines a vanishing point by using a position of the two boundary lines on the camera image.

16. The image processing apparatus according to claim 15,

wherein said boundary line detection unit interpolates a new vanishing point by using the vanishing point of the present camera image and a vanishing point of the previous camera image.

17. The image processing apparatus according to claim 15,

wherein the vanishing point of the previous camera image is regarded as an origin coordinate of the present camera image, and

wherein said reverse projection image generation unit projects information of the present camera image onto the plane by using the origin coordinate of the present camera image.

18. The image processing apparatus according to claim 15,

further comprising:

a boundary line output unit configured to output the two boundary lines projected onto the camera image.

19. Image processing method, comprising:

inputting a plurality of images in a time series, each input image including a boundary line extracting toward a vanishing point on a plane;

generating a reverse projection image of each input image by projecting information related to each input image onto the plane; and

detecting the boundary line from the reverse projection image by identifying a boundary line candidate on the reverse projection image.

20. A computer program product, comprising:

a computer readable program code embodied in said product for causing a computer to process an image, said computer readable program code having;

a first program code to input a plurality of images in a time series, each input image including a boundary line extending toward a vanishing point on a plane;

a second program code to generate a reverse projection image of each input image by projecting information related to each input image onto the plane; and

a third program code to detect the boundary line from the reverse projection image by identifying a boundary line candidate on the reverse projection image.