Moving-state determining device

• US 20080013790A1
• Filed: 07/13/2007
• Published: 01/17/2008
• Est. Priority Date: 07/13/2006
• Status: Active Grant

• Current Assignee: Mitsubishi FUSO Truck & Bus Corporation (Daimler Truck Holding AG), KEIO University
• Original Assignee: Mitsubishi FUSO Truck & Bus Corporation (Daimler Truck Holding AG), KEIO University
• Inventors: Yamamoto, Keiichi, Saito, Hideo, Teshima, Tomoaki, Ihara, Toru, Ozawa, Shinji
• Granted Patent: US 7,965,871 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60W 40/109 Lateral acceleration
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A moving-state determining device acquires a first overhead image from a bird'"'"'s-eye image obtained at first image pickup timing and acquires a second overhead image, which can be regarded as the same road surface area as that of the first overhead image, for each of moving directions assumed for the moving body from a bird'"'"'s-eye image obtained at second image pickup timing different from the first image pickup timing by a predetermined time period. The device compares a degree of similarity between the first overhead image and each of a plurality of second overhead images to acquire the second overhead image with the largest similarity degree as the same road surface area as that of the first overhead image. The device acquires the moving direction and a lateral displacement amount in the moving from a coordinate conversion condition based on which the second overhead image has been obtained.

Claims

1. A moving-state determining device, comprising:

a camera mounted on a moving body for obtaining a bird'"'"'s-eye image by picking up a road surface on which the moving body moves from diagonally upward;

coordinate converting means for coordinate-converting the bird'"'"'s-eye image to acquire an overhead image viewing the road surface from right above;

bird'"'"'s-eye image obtaining means for obtaining a first bird'"'"'s-eye image picked up by the camera at a first image pickup timing and a second bird'"'"'s-eye image picked up by the camera at a second image pickup timing different from the first image pickup timing by a predetermined time period;

overhead image obtaining means which obtains a first overhead image from the first bird'"'"'s-eye image using the coordinate converting means, assumes a plurality of changes in the moving state of the moving body on the road surface during the predetermined time period and obtains a plurality of second overhead images that can be regarded as the same road surface area as that of the first overhead image based on the assumed changes in the moving state of the moving body and the second bird'"'"'s-eye image; and

moving-state change calculating means which calculates a degree of similarity between the first overhead image and each of the plurality of second overhead images, and compares the degrees of similarity to each other to obtain most probable change in the moving state of the moving body from the assumed change based on which the second overhead image with the largest degree of similarity is obtained.

2. The moving-state determining device according to claim 1,

wherein said degree of similarity between the first overhead image and the second overhead image is obtained by calculating a sum of absolute values of differences between the overhead images, and

said second overhead image with the largest degree of similarity with respect to the first overhead image is the second overhead image with the smallest value of the sum of the absolute value of the differences.

3. The moving-state determining device according to claim 1,

wherein said coordinate converting means regards the road surface on which the moving body moves as a plane and converts the bird'"'"'s-eye image to the overhead image using homography H indicating a geometric positional relation between a camera image obtained by picking up an image of the road surface at a given inclination φ

from a given height h and the road surface,

wherein homography H_P which is used for obtaining the first overhead image and homography H_N which is used for obtaining the second overhead image are acquired by the following equation with a moving distance of the moving body during the predetermined time period as L and a moving direction of the moving body as yaw angle θ

, when its internal parameter matrix A is known;

4. The moving-state determining device according to claim 3,

wherein said moving distance L of the moving body during the predetermined time period is obtained from a moving speed of the moving body measured by a speed sensor.

5. The moving-state determining device according to claim 1,

wherein said overhead image obtaining means acquires, from the second bird'"'"'s-eye images, the second overhead image which can be regarded as the same road surface area as that of the first overhead image for each of inclination in the longitudinal direction of the moving body in each of moving directions of the moving body assumed in advance, and

said moving-state change calculating means compares the similarity degrees between the first overhead image and each of the plurality of second overhead images to each other to identify the second overhead image with the largest similarity degree, and determines the moving direction and the inclination of the moving body based on which the identified second overhead image is obtained as the most probable change in the moving state of the moving body.

6. The moving-state determining device according to claim 5,

wherein homography H_N which is used for obtaining the second overhead image is acquired by the following equation with the inclination of the moving body in the longitudinal direction as a pitch angle φ

;

7. The moving-state determining device according to claim 6,

wherein said moving body is a vehicle,

said camera picks up a road image in front of the vehicle in the moving direction of the vehicle, and said moving-state determining device further comprises lateral displacement amount calculating means for calculating a lateral displacement amount with respect to the moving direction of the vehicle based on the most probable change in the moving state of the vehicle acquired by the moving-state change calculating means,

wherein said running-state determining device extracts the lateral displacement amount corresponding to drop in attention of a driver of the vehicle from the lateral displacement amount calculated by the lateral displacement amount calculating means and uses the extracted lateral displacement amount for detecting meandering of the vehicle.

8. The moving-state determining device according to claim 6,

wherein said moving body is a vehicle, and

said moving-state determining device further comprises illumination direction adjusting means for adjusting an illumination direction of a headlight of the vehicle based on the pitch angle φ

.

9. The moving-state determining device according to claim 1,

wherein said moving body is a vehicle, and

said moving-state determining device further comprises:

lateral displacement amount calculating means for calculating a lateral displacement amount with respect to the moving direction of the vehicle based on the most probable change in the moving state of the vehicle acquired by the traveling-state change calculating means, the lateral displacement amount calculating means extracting the lateral displacement amount corresponding to a curve of a road on which the vehicle is moving based on temporal change of the lateral displacement amount and estimating the direction of the curve based on the extracted lateral displacement amount; and

illumination direction adjusting means for adjusting an illumination direction of a headlight of the vehicle based on the direction of the curve estimated by the lateral displacement amount calculating means.