Object recognition apparatus and object recognition method using epipolar geometry

• US 8,077,965 B2
• Filed: 02/22/2006
• Issued: 12/13/2011
• Est. Priority Date: 02/23/2005
• Status: Active Grant

• Current Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Original Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Inventors: Kakinami, Toshiaki, Sato, Jun
• Primary Examiner(s): Lu, Tom Y
• Assistant Examiner(s): CONWAY, THOMAS A
• Application Number:
  US11/884,728
• Publication Number:
  US 20080137940A1
• Time in Patent Office: 2,120 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/586 of parking space
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space

Abstract

An object recognition apparatus that processes images, as acquired by an imaging means (10) mounted on a moving object, in a first image plane and a second image plane of different points of view, and recognizes an object in the vicinity of the moving object. The object recognition apparatus comprises: a feature point detection means (42) that detects feature points in first and second image planes of an object image; a fundamental matrix determination means (43a) that determines, based on calculation of an epipole through the auto-epipolar property, a fundamental matrix that expresses a geometrically corresponding relationship based on translational camera movement, with not less than two pairs of feature points corresponding between the first and second image planes; and a three-dimensional position calculation means (43b) that calculates a three-dimensional position of an object based on the coordinates of the object in the first and second image planes and the determined fundamental matrix.

Claims

1. An object recognition apparatus processing images, as acquired by an imaging means mounted on a moving object, in a first image plane and a second image plane having different points of view, and recognizing an object in the vicinity of the moving object, the object recognition apparatus comprising:

a feature point detection means detecting feature points of an object image in the first image plane and the second image plane;

a fundamental matrix determination means determining a fundamental matrix expressing a geometrically corresponding relationship, based on translational camera movement, between the first image plane and the second image plane, from not less than two pairs of feature points corresponding between the first image plane and the second image plane, on the basis of calculation of epipoles through auto-epipolar property; and

a three-dimensional position calculation means calculating a three-dimensional position of the object based on the coordinates of the object in the first image plane and the second image plane and the determined fundamental matrix;

wherein the fundamental matrix determination means generates temporary fundamental matrices, takes feature points in which the distance between a straight line connecting the feature points corresponding to one another in the first image plane and the second image plane and the epipole is less than or equal to a predetermined value as properly-corresponding feature points, and determines the temporary fundamental matrix in which the number of properly corresponding feature points is greatest as an official fundamental matrix.

2. The object recognition apparatus according to claim 1,

wherein the feature point detection means detects an occluding contour of the object and detects points positioned on the contour lines of the occluding contour as the feature points.

3. The object recognition apparatus according to claim 1,

wherein the epipole is found using the temporary fundamental matrix generated based on two pairs of corresponding feature points selected at random in the first image plane and the second image plane, a process for finding the discrete value of the properly-corresponding feature points found using the epipole is repeated a set number of times, and the temporary fundamental matrix in which the discrete value is greatest is determined as the official fundamental matrix.

4. The object recognition apparatus according to claim 1,

wherein the moving object is an automobile, the object is a bumper of another automobile present in the vicinity of the automobile, and the feature point detection means detects an occluding contour of the bumper.

5. The object recognition apparatus according to claim 1,

wherein the imaging means is configured of a single camera, and

the first image plane and the second image plane are respectively created before and after translational movement of the camera, the translational movement resulting from movement of the moving object.

6. The object recognition apparatus according to claim 5, wherein evaluation of a state of movement of the camera is performed based on information from a moving object position information acquisition means provided in the moving object.

7. The object recognition apparatus according to claim 1, further comprising:

a frame memory storing the images of the first image plane and the second image plane acquired by the imaging means; and

an image processing portion detecting an occluding contour using epipolar geometry,

wherein the image processing portion detects and tracks feature points of the object in an image read from the frame memory, and determines an essential matrix using an epipole found from the feature points based on epipolar geometric calculation.

8. The object recognition apparatus according to claim 7,

wherein the image processing portion applies a Gaussian differential filter in the direction of the epipole on the images to be processed stored in the frame memory at the time of occluding contour detection.

9. An object recognition method for processing images, as acquired by an imaging means mounted on a moving object, in a first image plane and a second image plane having different points of view, and recognizing an object in the vicinity of the moving object, the object recognition method comprising:

a step of detecting feature points of a object image in the first image plane and the second image plane;

a step of determining a fundamental matrix expressing a geometrically corresponding relationship, based on translational camera movement, between the first image plane and the second image plane, from not less than two pairs of feature points corresponding between the first image plane and the second image plane, on the basis of calculation of epipoles through auto-epipolar property; and

a step of calculating a three-dimensional position of the object based on the coordinates of the object in the first image plane and the second image plane and the determined fundamental matrix;

wherein the step of determining a fundamental matrix includes generating temporary fundamental matrices, taking feature points in which the distance between a straight line connecting the feature points corresponding to one another in the first image plane and the second image plane and the epipole is less than or equal to a predetermined value as properly-corresponding feature points, and determining the temporary fundamental matrix in which the number of properly-corresponding feature points is greatest as an official fundamental matrix.

10. A non-transitory computer-readable medium storing an object recognition program that causes a computer to execute a function of processing images, as acquired by an imaging means mounted on a moving object, in a first image plane and a second image plane having different points of view, and recognizing an object in the vicinity of the moving object, wherein the object recognition program comprises:

a function of detecting feature points of a object image in the first image plane and the second image plane;

a function of determining a fundamental matrix expressing a geometrically corresponding relationship, based on translational camera movement, between the first image plane and the second image plane, from not less than two pairs of feature points corresponding between the first image plane and the second image plane, on the basis of calculation of epipoles through auto-epipolar property; and

a function of calculating a three-dimensional position of the object based on the coordinates of the object in the first image plane and the second image plane and the determined fundamental matrix;

wherein the function of determining a fundamental matrix includes generating temporary fundamental matrices, taking feature points in which the distance between a straight line connecting the feature points corresponding to one another in the first image plane and the second image plane and the epipole is less than or equal to a predetermined value as properly-corresponding feature points, and determining the temporary fundamental matrix in which the number of properly-corresponding feature points is greatest as an official fundamental matrix.