Moving object recognizing apparatus

• US 8,184,857 B2
• Filed: 01/24/2008
• Issued: 05/22/2012
• Est. Priority Date: 01/30/2007
• Status: Expired due to Fees

• Current Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Original Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Inventors: Akita, Tokihiko
• Primary Examiner(s): Wu, Jingge
• Assistant Examiner(s): Goradia, Shefali
• Application Number:
  US12/525,018
• Publication Number:
  US 20100027844A1
• Time in Patent Office: 1,580 Days
• Field of Search: 382/100, 382/103, 348/143, 348/148
• US Class Current: 382/103
• CPC Class Codes: G06T 7/269 using gradient-based methods

Abstract

The moving object recognizing apparatus includes a data acquisition unit for acquiring image data obtained in time series by an imaging device, a recognition unit for recognizing a moving object by executing a plurality of image processing operations on two of the image data obtained at different times, the plurality of image processing operations including an operation of calculating correlativity of a same recognition target pattern, and a reliability calculation unit for calculating recognition reliability indicative of reliability of recognition of the moving object, based upon at least one of the results of the respective image processing operations by the recognition unit.

Claims

1. A moving object recognizing apparatus comprising:

a data acquisition unit for acquiring image data obtained in time series by an imaging device;

a recognition unit for recognizing a moving object by executing a plurality of image processing operations on two of the image data obtained at different times, said plurality of image processing operations including an operation of calculating correlativity of a same recognition target pattern, said recognition unit including a point feature extraction unit for extracting a point feature used as said recognition target pattern from the image data, an optical flow calculation unit for calculating optical flows, based upon said point feature present in said two image data obtained at different times, and a grouping unit for grouping, from among the optical flows, optical flows belonging to one moving object, under a predetermined condition; and

a reliability calculation unit for calculating recognition reliability indicative of reliability of recognition of the moving object, based upon at least one of the results of the respective image processing operations by the recognition unit, said reliability calculation unit calculating the recognition reliability with using at least one of the group consisting of;

a plurality of primary intermediate reliabilities which are calculated based on the respective results of a plurality of the image processing operations including the image processing operations executed by said point feature extraction unit, said optical flow calculation unit, and said grouping unit and which indicate reliabilities of the results of the respective image processing operations;

a secondary intermediate reliability which is one selected from said primary intermediate reliabilities or a fusion of two or more of the primary intermediate reliabilities;

an average reliability obtained by averaging the secondary intermediate reliabilities of those optical flows whose secondary intermediate reliabilities are included within a predetermined higher range;

an object reliability calculated based upon correlativity between a size permissible for the moving object as one recognition target and a physical size delimited by an area where the optical flows grouped as belonging to the moving object are present, said object reliability indicating a degree of certainty whether the recognized moving object is truly the moving object to be recognized or not; and

a position precision reliability calculated based upon the degree of dispersion of the positions of the optical flows grouped as belonging to the one recognized moving object and indicating the degree of certainty of the position of this recognized moving object.

2. The moving object recognizing apparatus according to claim 1, wherein said optical flow calculation unit calculates said optical flow, based upon;

a target area around said point feature in previously acquired image data, and

an area included in image data acquired with lapse of a predetermined period after said previously acquired image data and having the highest correlativity value relative to said target area; and

said reliability calculation unit calculates, based on said correlativity value, optical flow reliability indicative of reliability of said calculated optical flow and then calculates said recognition reliability with using said optical flow reliability.

3. The moving object recognizing apparatus according to claim 2, wherein said grouping unit groups, as the optical flows belonging in one moving object, particular optical flows among said calculate optical flows, which satisfy the following conditions;

said each particular optical flow has an extension which is convergent to a single vanishing point within a predetermining convergence range; and

said each particular optical flow has a same external ratio within a predetermined tolerance range, said external ratio being relative to one end of said particular optical flow as an external dividing point and being determined between said particular optical flow and a line segment extending beyond the other end of said particular optical flow to said vanishing point;

said reliability calculation unit calculates grouping reliability indicative of reliability of said grouped optical flows belonging to said one moving object, based upon either one or both of convergence of the grouped optical flows and the total number of the grouped optical flows; and

said reliability calculation unit calculates said recognition reliability with using said grouping reliability.

4. The moving object recognizing apparatus according to claim 3, wherein said reliability calculation unit may calculate said convergence, based upon either one or both of deviation of each optical flow grouped according to each grouping criteria and dispersion of said deviation.

5. The moving object recognizing apparatus according to claim 2, wherein said point feature extraction unit calculates a point feature amount indicative of a characteristic property of said point feature in the image data;

said reliability calculation unit calculates a point feature reliability indicative of reliability of the point feature extracted based upon said point feature amount; and

said recognition reliability is calculated with using the point feature reliability.

6. The moving object recognizing apparatus according to claim 2, wherein said recognition unit executes the image processing operations in time series, according to the image data acquired in time series;

said reliability calculation unit calculates the recognition reliability according to said image processing operations executed in time series; and

the recognition reliability calculated at said reliability calculation unit based upon the latest image processing operation is calculated, based upon;

the result of the latest image processing operation, and

previously calculated recognition reliability calculated with using the result of the previous image processing operation.

7. The moving object recognizing apparatus according to claim 2, wherein said recognition unit calculates the instantaneous velocity of the moving object at each time and the average velocity thereof, based upon time differences of the image data acquired in time series and the position of the moving object obtained by executing the image processing operations in time series with using the image data;

and

said reliability calculation unit calculates continuity reliability indicative of the certainty whether the recognized moving object remains the same through the image processing operations executed in time series, based upon deviation amounts of the instantaneous velocities relative to the average velocity; and

the recognition reliability is calculated, with using the continuity reliability.

8. The moving object recognizing apparatus according to claim 1, wherein said grouping unit groups, as the optical flows belonging in one moving object, particular optical flows among said calculate optical flows, which satisfy the following conditions;

said each particular optical flow has an extension which is convergent to a single vanishing point within a predetermining convergence range; and

said each particular optical flow has a same external ratio within a predetermined tolerance range, said external ratio being relative to one end of said particular optical flow as an external dividing point and being determined between said particular optical flow and a line segment extending beyond the other end of said particular optical flow to said vanishing point;

said reliability calculation unit calculates grouping reliability indicative of reliability of said grouped optical flows belonging to said one moving object, based upon either one or both of convergence of the grouped optical flows and the total number of the grouped optical flows; and

said reliability calculation unit calculates said recognition reliability with using said grouping reliability.

9. The moving object recognizing apparatus according to claim 8, wherein said reliability calculation unit may calculate said convergence, based upon either one or both of deviation of each optical flow grouped according to each grouping criteria and dispersion of said deviation.

10. The moving object recognizing apparatus according to claim 9, wherein said point feature extraction unit calculates a point feature amount indicative of a characteristic property of said point feature in the image data;

said reliability calculation unit calculates a point feature reliability indicative of reliability of the point feature extracted based upon said point feature amount; and

said recognition reliability is calculated with using the point feature reliability.

11. The moving object recognizing apparatus according to claim 9, wherein said recognition unit executes the image processing operations in time series, according to the image data acquired in time series;

said reliability calculation unit calculates the recognition reliability according to said image processing operations executed in time series; and

the recognition reliability calculated at said reliability calculation unit based upon the latest image processing operation is calculated, based upon;

the result of the latest image processing operation, and

previously calculated recognition reliability calculated with using the result of the previous image processing operation.

12. The moving object recognizing apparatus according to claim 8, wherein said point feature extraction unit calculates a point feature amount indicative of a characteristic property of said point feature in the image data;

said reliability calculation unit calculates a point feature reliability indicative of reliability of the point feature extracted based upon said point feature amount; and

said recognition reliability is calculated with using the point feature reliability.

13. The moving object recognizing apparatus according to claim 8, wherein said recognition unit executes the image processing operations in time series, according to the image data acquired in time series;

said reliability calculation unit calculates the recognition reliability according to said image processing operations executed in time series; and

the recognition reliability calculated at said reliability calculation unit based upon the latest image processing operation is calculated, based upon;

the result of the latest image processing operation, and

previously calculated recognition reliability calculated with using the result of the previous image processing operation.

14. The moving object recognizing apparatus according to claim 8, wherein said recognition unit calculates the instantaneous velocity of the moving object at each time and the average velocity thereof, based upon time differences of the image data acquired in time series and the position of the moving object obtained by executing the image processing operations in time series with using the image data;

and

said reliability calculation unit calculates continuity reliability indicative of the certainty whether the recognized moving object remains the same through the image processing operations executed in time series, based upon deviation amounts of the instantaneous velocities relative to the average velocity; and

the recognition reliability is calculated, with using the continuity reliability.

15. The moving object recognizing apparatus according to claim 1, wherein said point feature extraction unit calculates a point feature amount indicative of a characteristic property of said point feature in the image data;

said reliability calculation unit calculates a point feature reliability indicative of reliability of the point feature extracted based upon said point feature amount; and

said recognition reliability is calculated with using the point feature reliability.

16. The moving object recognizing apparatus according to claim 15, wherein said recognition unit executes the image processing operations in time series, according to the image data acquired in time series;

said reliability calculation unit calculates the recognition reliability according to said image processing operations executed in time series; and

the recognition reliability calculated at said reliability calculation unit based upon the latest image processing operation is calculated, based upon;

the result of the latest image processing operation, and

previously calculated recognition reliability calculated with using the result of the previous image processing operation.

17. The moving object recognizing apparatus according to claim 1, wherein said recognition unit executes the image processing operations in time series, according to the image data acquired in time series;

said reliability calculation unit calculates the recognition reliability according to said image processing operations executed in time series; and

the recognition reliability calculated at said reliability calculation unit based upon the latest image processing operation is calculated, based upon;

the result of the latest image processing operation, and

previously calculated recognition reliability calculated with using the result of the previous image processing operation.

18. The moving object recognizing apparatus according to claim 1, wherein said recognition unit calculates the instantaneous velocity of the moving object at each time and the average velocity thereof, based upon time differences of the image data acquired in time series and the position of the moving object obtained by executing the image processing operations in time series with using the image data;

and

said reliability calculation unit calculates continuity reliability indicative of the certainty whether the recognized moving object remains the same through the image processing operations executed in time series, based upon deviation amounts of the instantaneous velocities relative to the average velocity; and

the recognition reliability is calculated, with using the continuity reliability.

19. The moving object recognizing apparatus according to claim 1, further comprising a reporting unit for reporting the recognition result of the moving object, in accordance with the recognition reliability.

20. The moving object recognizing apparatus according to claim 19, wherein the reporting unit

reports the recognition result, in an alarm mode enabling speedy recognition by the user through at least either one of the visual perception and the auditory perception, if the recognition reliability is greater than a predetermined first threshold value;

or

reports the recognition result, in a caution mode of lower recognizability than the alarm mode, if the recognition reliability is below the first threshold value and greater than a second threshold value set to be smaller than the first threshold value;

or

does not report the recognition result at all, if the recognition reliability is below the second threshold value.