Image processing apparatus and method

• US 7,095,432 B2
• Filed: 07/17/2002
• Issued: 08/22/2006
• Est. Priority Date: 07/18/2001
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Kubota, Susumu, Nakayama, Hiromitsu, Taniguchi, Yasuhiro
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US10/196,246
• Publication Number:
  US 20030016287A1
• Time in Patent Office: 1,497 Days
• Field of Search: 348/148, 348/143, 348/119, 348/171, 348/149, 348/118, 382/153, 382/158, 382/156, 382/159, 382/104, 382/199, 382/103, 382/106, 382/22
• US Class Current: 348/148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation

Abstract

An image input unit inputs a plurality of images in a time series. Each input image includes a boundary line extending toward a vanishing point on a plane. A reverse projection image generation unit generates a reverse projection image of each input image by projecting information related to each input image onto the plane. A boundary line detection unit detects the boundary line from the reverse projection image by identifying a boundary line candidate on each reverse projection image.

Claims

1. An image processing apparatus comprising:

an image input unit configured to input a plurality of images in a time series, each input image including a boundary line extending toward a vanishing point on a plane;

a reverse projection image generation unit configured to calculate a differential vector of an intensity value of a pixel on an input image, to calculate a normal vector of the pixel for a direction of the boundary line passing through the pixel, to calculate an inner product value between the differential vector and the normal vector of the pixel on the input image, and to project the inner product value of the pixel onto the plane, the plane onto which the inner product value is projected being a reverse projection image; and

a boundary line detection unit configured to detect the boundary line from the reverse projection image based on the inner product value.

2. The image processing apparatus according to claim 1,

wherein said reverse projection image generation unit determines the direction of the boundary line passing through the pixel by using a curvature of the boundary line detected from the previous input image in the time series to calculate the normal vector.

3. The image processing apparatus according to claim 1,

wherein said reverse projection image generation unit calculates the inner product value between the differential vector and the normal vector for a left side edge and a right side edge of the boundary line.

4. The image processing apparatus according to claim 3,

wherein said reverse projection image generation unit sets an upper limit line and a lower limit line to define a reverse projection conversion area on the input image, and projects the inner product value of each of the pixels included in the reverse projection area onto the plane.

5. The image processing apparatus according to claim 3,

wherein said boundary line detection unit detects a first segment by connecting at least two positive peaks of the inner product value on the reverse projection image, detects a second segment by connecting at least two negative peaks of the inner product value on the reverse projection image, and extracts the first segment and the second segment as a first boundary line candidate segment from the reverse projection image.

6. The image processing apparatus according to claim 5,

wherein said boundary line detection unit identifies a second boundary line candidate segment having at least one similar attribute with the first boundary line candidate segment, and registers the similar attribute of the first and second boundary line candidate segments as a boundary line candidate in a boundary line candidate table.

7. The image processing apparatus according to claim 6,

wherein the registered attribute comprises at least one of an average edge intensity, a line width, a horizontal position having at least a starting point and an end point, a continuance time as the number of frames in which the boundary line candidate is continuously detected, a detection time as the number of frames passed since the boundary line candidate is first detected, and a disappearance time as the number of frames passed since the boundary line candidate is last detected.

8. The image processing apparatus according to claim 7,

wherein said boundary line detection unit decides that a new boundary line candidate segment detected from the reverse projection image is the same as the boundary line candidate registered in the boundary line candidate table if an attribute of the new boundary line candidate segment is similar to the registered attribute of the boundary line candidate, and updates the registered attribute of the boundary line candidate in the boundary line candidate table according to the attribute of the new boundary line candidate segment.

9. The image processing apparatus according to claim 8,

wherein said boundary line detection unit selects the boundary line as a left white line and a second boundary line as a right white line from the boundary line candidates registered in the boundary line candidate table according to the registered attribute of each boundary line candidate.

10. The image processing apparatus according to claim 9,

wherein said boundary line detection unit calculates the curvature of each of the two boundary lines, and extrapolates a depth part of each of the two boundary lines on the reverse projection image by using at least one of a circular arc and a parabola of the curvature.

11. The image processing apparatus according to claim 10,

wherein said boundary line detection unit projects each pixel of the two boundary lines onto the camera image, and determines a vanishing point by using a position of the two boundary lines on the camera image.

12. The image processing apparatus according to claim 11,

wherein said boundary line detection unit interpolates a new vanishing point by using the vanishing point of the present camera image and a vanishing point of the previous camera image.

13. The image processing apparatus according to claim 11,

wherein the vanishing point of the previous camera image is regarded as an origin coordinate of the present camera image, and

wherein said reverse projection image generation unit projects information of the present camera image onto the plane by using the origin coordinate of the present camera image.

14. The image processing apparatus according to claim 11,

further comprising:

a boundary line output unit configured to output the two boundary lines projected onto the camera image.

15. An image processing method, comprising:

inputting a plurality of images in a time series, each input image including a boundary line extending toward a vanishing point on a plane;

calculating a differential vector of an intensity value of a pixel on an input image;

calculating a normal vector of the pixel for a direction of the boundary line passing through the pixel;

calculating an inner product value between the differential vector and the normal vector of the pixel on the input image;

projecting the inner product value of the pixel onto the plane, the plane onto which the inner product value is projected being a reverse projection image; and

detecting the boundary line from the reverse projection image based on the inner product value.

17. The image processing method according to claim 15,

wherein the second calculating step further comprises

determining the direction of the boundary line passing through the pixel by using a curvature of the boundary line detected from the previous input image in the time series to calculate the normal vector.

18. The image processing method according to claim 15,

wherein the third calculating step further comprises

calculating the inner product value between the differential vector and the normal vector for a left side edge and a right side edge of the boundary line.

19. The image processing method according to claim 18,

wherein the projecting step further comprises

setting an upper limit line and a lower limit line to define a reverse projection conversion area on the input image;

and

projecting the inner product value of each of the pixels included in the reverse projection area onto the plane.

20. The image processing method according to claim 18,

wherein the detecting step further comprises

detecting a first segment by connecting at least two positive peaks of the inner product value on the reverse projection image;

detecting a second segment by connecting at least two negative peaks of the inner product value on the reverse projection image; and

extracting the first segment and the second segment as a first boundary line candidate segment from the reverse projection image.

21. The image processing method according to claim 20,

wherein the detecting step further comprises

identifying a second boundary line candidate segment having at least one similar attribute with the first boundary line candidate segment;

and

registering the similar attribute of the first and second boundary line candidate segments as a boundary line candidate in a boundary line candidate table.

22. The image processing method according to claim 21,

wherein the registered attribute comprises at least one of an average edge intensity, a line width, a horizontal position having at least a starting point and an end point, a continuance time as the number of frames in which the boundary line candidate is continuously detected, a detection time as the number of frames passed since the boundary line candidate is first detected, and a disappearance time as the number of frames passed since the boundary line candidate is last detected.

23. The image processing method according to claim 22,

wherein the detecting step further comprises

deciding that a new boundary line candidate segment detected from the reverse projection image is the same as the boundary line candidate registered in the boundary line candidate table if an attribute of the new boundary line candidate segment is similar to the registered attribute of the boundary line candidate;

and

updating the registered attribute of the boundary line candidate in the boundary line candidate table according to the attribute of the new boundary line candidate segment.

24. The image processing method according to claim 23,

wherein the detecting step further comprises

selecting the boundary line as a left white line and a second boundary line as a right white line from the boundary line candidates registered in the boundary line candidate table according to the registered attribute of each boundary line candidate.

25. The image processing method according to claim 21,

wherein the detecting step further comprises

calculating a curvature of each of the two boundary lines and extrapolating a depth part of each of the two boundary lines on the reverse projection image by using at least one of a circular arc and a parabola of the curvature.

26. The image processing method according to claim 25,

wherein the detecting step further comprises

projecting each pixel of the two boundary lines onto the camera image;

and

determining a vanishing point by using a position of the two boundary lines on the camera image.

27. The image processing method according to claim 26,

wherein the detecting step further comprises

interpolating a new vanishing point by using the vanishing point of the present camera image and a vanishing point of the previous camera image.

28. The image processing method according to claim 26,

wherein the vanishing point of the previous camera image is regarded as an origin coordinate of the present camera image, and

wherein the projecting step further comprises

projecting information of the present camera image onto the plane by using the origin coordinate of the present camera image.

29. The image processing method according to claim 26,

further comprising:

outputting the two boundary lines projected onto the camera image.

16. A computer readable medium encoded with a computer readable program code for causing a computer to process an image, said computer readable program code having:

a first program code to input a plurality of images in a time series, each input image including a boundary line extending toward a vanishing point on a plane;

a second program code to calculate a differential vector of an intensity value of a pixel on an input image;

a third program code to calculate a normal vector of the pixel for a direction of the boundary line passing through the pixel;

a fourth program code to calculate an inner product value between the differential vector and the normal vector of the pixel on the input image; and

a fifth program code to project the inner product value of the pixel onto the plane.

30. The computer readable medium according to claim 16,

wherein the third program code further comprises a program code to determine the direction of the boundary line passing through the pixel by using a curvature of the boundary line detected from the previous input image in the time series to calculate the normal vector.

31. The computer readable medium according to claim 16,

wherein the fourth program code further comprises a program code to calculate the inner product value between the differential vector and the normal vector for a left side edge and a right side edge of the boundary line.

32. The computer readable medium according to claim 31,

wherein the fifth program code further comprises a program code to set an upper limit line and a lower limit line to define a reverse projection conversion area on the input image, and a program code to project the inner product value of each of the pixels included in the reverse projection area onto the plane.

33. The computer readable medium according to claim 32,

wherein the sixth program code further comprises a program code to detect a first segment by connecting at least two positive peaks of the inner product value on the reverse projection image, a program code to detect a second segment by connecting at least two negative peaks of the inner product value on the reverse projection image, and a program code to extract the first segment and the second segment as a first boundary line candidate segment from the reverse projection image.

34. The computer readable medium according to claim 33,

wherein the sixth program code further comprises a program code to identify a second boundary line candidate segment having at least one similar attribute with the first boundary line candidate segment, and a program code to register the similar attribute of the first and second boundary line candidate segments as a boundary line candidate in a boundary line candidate table.

35. The computer readable medium according to claim 34,

wherein the registered attribute comprises at least one of an average edge intensity, a line width, a horizontal position having at least a starting point and an end point, a continuance time as the number of frames in which the boundary line candidate is continuously detected, a detection time as the number of frames passed since the boundary line candidate is first detected, and a disappearance time as the number of frames passed since the boundary line candidate is last detected.

36. The computer readable medium according to claim 35,

wherein the sixth program code further comprises a program code to decide that a new boundary line candidate segment detected from the reverse projection image is the same as the boundary line candidate registered in the boundary line candidate table if an attribute of the new boundary line candidate segment is similar to the registered attribute of the boundary line candidate, and a program code to update the registered attribute of the boundary line candidate in the boundary line candidate table according to the attribute of the new boundary line candidate segment.

37. The computer readable medium according to claim 36,

wherein the sixth program code further comprises a program code to select the boundary line as a left white line and a second boundary line as a right white line from the boundary line candidates registered in the boundary line candidate table according to the registered attribute of each boundary line candidate.

38. The computer readable medium according to claim 37,

wherein the sixth program code further comprises a program code to calculate a curvature of each of the two boundary lines, and a program code to extrapolate a depth part of each of the two boundary lines on the reverse projection image by using at least one of a circular arc and a parabola of the curvature.

39. The computer readable medium according to claim 38,

wherein the sixth program code further comprises a program code to project each pixel of the two boundary lines onto the camera image, and a program code to determine a vanishing point by using a position of the two boundary lines on the camera image.

40. The computer readable medium according to claim 39,

wherein the sixth program code further comprises a program code to interpolate a new vanishing point by using the vanishing point of the present camera image and a vanishing point of the previous camera image.

41. The computer readable medium according to claim 39,

wherein the vanishing point of the previous camera image is regarded as an origin coordinate of the present camera image, and

wherein the fifth program code further comprises a program code to project information of the present camera image onto the plane by using the origin coordinate of the present camera image.

42. The image processing apparatus according to claim 39,

further comprising:

a seventh program code to output the two boundary lines projected onto the camera image.