Pedestrian detector and pedestrian detecting method

• US 20070274566A1
• Filed: 05/16/2007
• Published: 11/29/2007
• Est. Priority Date: 05/24/2006
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Fujimoto, Kazumi
• Granted Patent: US 7,667,581 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60R 21/34 Protecting non-occupants of a vehicle, e.g. pedestrians B60R19/02 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A pedestrian detecting method including: picking up an image of a scene in front of a vehicle; extracting characteristic points from the picked up image; calculating a velocity at which, and a direction in which, each of pixels representing the respective characteristic points moves in the image; expressing coordinates of each of the pixels in a coordinate system corresponding to an overhead view; determining whether the characteristic points represent a two-dimensional object or a three-dimensional object, and if it is determined that they represent the three-dimensional object, whether or not the three-dimensional object is a moving object, on the basis of the calculated velocity and direction of each of the pixels and the coordinates thereof in the overhead view; and determining whether or not the moving object is a pedestrian on the basis of change in the velocity at which the moving object moves in the image.

Claims

1. A pedestrian detector comprising:

an image pick up unit configured to pick up an image of a scene in front of a host vehicle; and

a controller including;

a characteristic point extractor configured to extract characteristic points from the image which has been picked up by the image pick up unit;

a movement information calculator configured to calculate a velocity at which, and a direction in which, each of pixels representing the respective characteristic points moves in the image, and to output the velocity and the direction as movement information;

a coordinate converter configured to express coordinates of each of the pixels in a coordinate system corresponding to an overhead view representing the image of the scene in front of the host vehicle obtained by coordinate conversion;

an object attribute determinator configured to determine that the characteristic points represent any one of a two-dimensional object or a three-dimensional object, and in a case where it is determined that the characteristic points represent the three-dimensional object, whether or not the three-dimensional object is a moving object, on the basis of the coordinates of each of the pixels in the overhead view, and on the basis of the movement information on each of the pixels; and

a pedestrian determinator configured to determine whether or not the moving object is a pedestrian on the basis of change in the velocity at which the moving object moves in the image in a case where it is determined that the characteristic points represent the moving object.

2. The pedestrian detector according to claim 1, wherein the pedestrian determinator calculates the variance of the velocity at which the moving object moves, and thus determines that the moving object is a pedestrian in a case where the variance is not smaller than a predetermined value.

3. The pedestrian detector according to claim 2, wherein the pedestrian determinator determines that the moving object is a pedestrian in a case where the variance of the velocity at which the moving object moves is smaller than the predetermined value, and concurrently in a case where change in the coordinates of the moving object in the image in a predetermined interval of time is not larger than a predetermined value while change in the velocity at which the moving object moves in the image in the predetermined interval of time is not smaller than a predetermined value.

4. The pedestrian detector according to claim 1, wherein the pedestrian determinator divides the moving object into an upper portion and a lower portion in the image, calculates the variance of a velocity at which the lower portion moves, and determines that the moving object is a pedestrian in a case where the variance is not smaller than a predetermined value.

5. The pedestrian detector according to claim 4, wherein the pedestrian determinator determines that the moving object is a pedestrian in a case where the variance of the velocity at which the lower portion of the moving object moves is smaller than the predetermined value, and concurrently in a case where the change in the coordinates of the moving object in the image in the predetermined interval of time is not larger than the predetermined value while the change in the velocity at which the moving object moves in the image in the predetermined interval of time is not smaller than the predetermined value.

6. The pedestrian detector according to claim 1, wherein the pedestrian determinator detects a top end and a bottom end of the moving object, calculates a maximum value of a velocity at which the top end moves and a maximum value of a velocity at which the bottom end moves, and thus determines that the moving object is a pedestrian in a case where a ratio of the maximum value of the top end to the maximum value of the bottom end is out of a predetermined range.

7. The pedestrian detector according to claim 6, wherein the pedestrian determinator determines that the moving object is a pedestrian in a case where the ratio of the maximum value of the velocity at which the top end of the moving object moves to the maximum value of the velocity at which the bottom end of the moving object moves is within the predetermined range, and concurrently in a case where the change in the coordinates of the moving object in the image in the predetermined interval of time is not larger than the predetermined value while the change in the velocity at which the moving object moves in the image in the predetermined interval of time is not smaller than the predetermined value.

8. The pedestrian detector according to claim 1,

further comprising a grouping unit configured to group into a single group pixels which are next to each other in the vertical direction in the image, and which move at their respective velocities being within a predetermined range,

wherein the object attribute determinator determines that the characteristic points represent any one of a two-dimensional object and a three-dimensional object on the basis of coordinates of each of the pixels, which are respectively situated uppermost and lowermost among the pixels grouped into the single group by the grouping unit, in the overhead view.

9. The pedestrian detector according to claim 8, wherein

the object attribute determinator determines that the characteristic points represent a two-dimensional object in a case where the pixels respectively situated uppermost and lowermost are situated in a single one of a plurality of sub-areas set up by dividing a predetermined area in the overhead view into a predetermined number of sub-areas, and

the object attribute determinator determines that the characteristic points represent the three-dimensional object in a case where the pixels respectively situated uppermost and lowermost are situated in mutually different ones of the plurality of sub-areas, or concurrently in a case where the pixel situated uppermost is situated out of the plurality of sub-areas in the overhead view whereas the pixel situated lowermost is situated in any one of the plurality of sub-areas in the overhead view.

10. The pedestrian detector according to claim 8, wherein

the object attribute determinator determines that a corresponding one of the pixels represents a candidate point for a two-dimensional object in a case where the pixel is situated in any one of a plurality of sub-areas set up by dividing a predetermined area in the overhead view into a predetermined number of sub-areas,

the object attribute determinator determines that the pixel represents a candidate point for a three-dimensional object in a case where the pixel is situated out of the plurality of sub-areas in the overhead view, and

the object attribute determinator determines that the candidate point for the two-dimensional object and the candidate point for the three-dimensional object represent the single three dimensional object in a case where the candidate point for the two-dimensional object and the candidate point for the three-dimensional object are situated next to each other in the vertical direction in the image.

11. The pedestrian detector according to claim 1, wherein the object attribute determinator performs a moving object determination of determining whether or not the three-dimensional object represents a moving object on the basis of the movement information on the pixel of each of the plurality of three-dimensional objects, and on the basis of the coordinates of the pixel in the image.

12. The pedestrian detector according to claim 11, wherein the object attribute determinator detects lane boundary lines indicating boundaries of a lane in which the host vehicle runs on the basis of the coordinates of the pixel, which is situated lowermost among the pixels in the vertical direction of the image, in the overhead view, and thus performs the moving object determination in each of zones divided by the lane boundary lines.

13. The pedestrian detector according to claim 1, wherein the object attribute determinator determines that a plurality of moving objects represent a single moving object in a case where the plurality of moving objects move in the same direction in the image while a ratio of velocities at which the plurality of moving objects move is within a predetermined range, and concurrently in a case where difference in the coordinates among the plurality of moving objects in the image is not larger than a predetermined value.

14. The pedestrian detector according to claim 1, wherein the characteristic points are edges of the object.

15. A pedestrian detecting method comprising:

picking up an image of a scene in front of a host vehicle;

extracting characteristic points from the image which has been picked up;

calculating a velocity at which, and a direction in which, each of pixels representing the respective characteristic points moves in the image, and outputting the velocity and the direction as movement information;

expressing coordinates of each of the pixels in a coordinate system corresponding to an overhead view representing the image of the scene in front of the host vehicle obtained by coordinate conversion;

determining whether the characteristic points represent any one of a two-dimensional object and a three-dimensional object, and in a case where it is determined that the characteristic points represent the three-dimensional object, whether or not the three-dimensional object is a moving object, on the basis of the coordinates of each of the pixels in the overhead view, and on the basis of the movement information on each of the pixels; and

determining whether or not the moving object is a pedestrian on the basis of change in the velocity at which the moving object moves in the image in a case where it is determined that the characteristic points represent the moving object.