Image processing apparatus and method

• US 20040057599A1
• Filed: 06/27/2003
• Published: 03/25/2004
• Est. Priority Date: 06/27/2002
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Okada, Ryuzo, Onoguchi, Kazunori
• Granted Patent: US 7,612,800 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An image processing apparatus includes an image input unit that receives time-series images, and a candidate area tracking unit that extracts a line segment in a specific direction, sets an obstacle candidate area in a vicinity of the detected line segment, and searches the vicinity of the line segment extracted in each frame, thus tracking the obstacle candidate area with low computational cost. Using the tracking result of tracking a group of three or more candidate areas, a plane assumption verifier determines to which of the preset planes these candidate areas belong. On the basis of the result by the plane assumption verifier, an obstacle detector detects an obstacle and estimates the position of the obstacle.

Claims

1. An image processing apparatus comprising:

means for inputting a plurality of frame images serving as video images;

means for detecting, from each frame image in the plurality of frame images, a straight-line component in a specific direction;

means for generating an obstacle candidate area as an image area in a vicinity of the detected straight-line component;

means for tracking the obstacle candidate area in an image succeeding each frame image in the plurality of frame images, and producing a tracking result for the obstacle candidate area;

means for determining, using the tracking result of three or more obstacle candidate areas, whether the three or more obstacle candidate areas belong to a specific plane and producing a determination result; and

means for detecting an obstacle based on the determination result.

2. The image processing apparatus according to claim 1, wherein the means for detecting, from each frame image in the plurality of frame images, detects a straight-line component in the horizontal direction.

3. The image processing apparatus according to claim 1, wherein the means for determining determines whether the three or more obstacle candidate areas belong to a horizontal plane.

4. The image processing apparatus according to claim 1, wherein the means for tracking includes:

means for setting an image area in a vicinity of the detected straight-line component as a search area for the obstacle candidate area; and

means for tracking the obstacle candidate area by comparing an image feature in the obstacle candidate area with an image feature in the search area.

5. The image processing apparatus according to claim 1, wherein the means for tracking eliminates the obstacle candidate area when a trajectory of the obstacle candidate area tracked over the plurality of frame images is not smooth.

6. The image processing apparatus according to claim 1, wherein the means for determining includes:

means for selecting a candidate area group consisting of three or more obstacle candidate areas; and

means for computing a fitness between the candidate area group and a predetermined plane and evaluating whether the candidate area group belongs to the predetermined plane based on the fitness.

7. The image processing apparatus according to claim 1, wherein means for detecting an obstacle detects when a number of obstacle candidate areas that are determined by the means for determining not to belong to the specific plane is greater than a predetermined number.

8. The image processing apparatus according to claim 1, further comprising:

means for estimating a position of the obstacle in a frame image based on a motion of the three or more obstacle candidate areas.

9. An image processing apparatus comprising:

a camera configured to input a plurality of frame images serving as video images;

a tracking unit configured to detect a straight-line component in a specific direction from each frame image in the plurality of frame images, to generate an obstacle candidate area as an image area in a vicinity of the straight-line component, to track the obstacle candidate area in an image succeeding each frame image in the plurality of frame images, and to produce a tracking result for the obstacle candidate area; and

a detector configured to determine, using the tracking result of three or more obstacle candidate areas, whether the plural obstacle candidate areas belong to a specific plane and configured to detect an obstacle based on the determination.

10. The image processing apparatus according to claim 9, wherein the tracking unit is configured to detect, from each frame image in the plurality of frame images, a straight-line component in a horizontal direction.

11. The image processing apparatus according to claim 9, wherein the detector is configured to determine whether the three or more obstacle candidate areas belong to a horizontal plane.

12. The image processing apparatus according to claim 9, wherein the tracking unit is configured to set an image area in a vicinity of the detected straight-line component as a search area for the obstacle candidate area and to track the obstacle candidate area by comparing an image feature in the obstacle candidate area with an image feature in the search area.

13. The image processing apparatus according to claim 9, wherein the tracking unit is configured to eliminate the obstacle candidate area when a trajectory of the obstacle candidate area tracked over the plurality of frame images is not smooth.

14. The image processing apparatus according to claim 9, wherein the detector is configured to select a candidate area group consisting of three or more obstacle candidate areas, to compute a fitness between the candidate area group and a predetermined plane, and to evaluate whether the candidate area group belongs to the predetermined plane based on the fitness.

15. The image processing apparatus according to claim 9, wherein the detector is configured to detect an obstacle when a number of obstacle candidate areas that are determined not to belong to the specific plane is greater than a predetermined number.

16. The image processing apparatus according to claim 9, wherein the detector is configured to estimate a position of the obstacle in a frame image based on a motion of the three or more obstacle candidate areas.

17. An image processing method comprising:

inputting a plurality of frame images serving as video images;

detecting, from each frame image in the plurality of frame images, a straight-line component in a specific direction;

generating an obstacle candidate area as an image area in a vicinity of the detected straight-line component;

tracking the obstacle candidate area in an image succeeding each frame image in the plurality of frame images, and producing a tracking result for the obstacle candidate area;

determining, using the tracking result of three or more obstacle candidate areas, whether the three or more obstacle candidate areas belong to a specific plane, and producing a determination result; and

detecting an obstacle based on the determination result.

18. The image processing method according to claim 17, wherein the detecting step detects, from each frame image in the plurality of frame images, a straight-line component in the horizontal direction.

19. The image processing method according to claim 17, wherein the determining step determines whether the three or more obstacle candidate areas belong to a horizontal plane.

20. The image processing method according to claim 17, wherein the tracking step includes:

setting an image area in a vicinity of the detected straight-line component as a search area for the obstacle candidate area; and

tracking the obstacle candidate area by comparing an image feature in the obstacle candidate area with an image feature in the search area.