Aparatus, method, and recording medium for measuring distance in a real space from a feature point on the road

• US 9,070,191 B2
• Filed: 01/09/2012
• Issued: 06/30/2015
• Est. Priority Date: 02/18/2011
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Murashita, Kimitaka
• Primary Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US13/345,945
• Publication Number:
  US 20120213412A1
• Time in Patent Office: 1,268 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A storage medium storing a distance calculation program that causes a computer to execute operations, the operations includes; obtaining a first image captured by an image pickup device disposed on a vehicle; extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and calculating a first distance from the first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and a first angle of the point viewed from a center of the image pickup device in the real space, and setting information including an angle of depression of the image pickup device and a distance between the image pickup device and the road surface.

Claims

1. A non-transitory storage medium storing a distance calculation program that causes a computer to execute a process, the process comprising:

obtaining a first image captured by an image pickup device;

extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and

calculating a first distance from the first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and a first angle of the point viewed from a center of the image pickup device in the real space, the position of the point in the first image being a distance represented by a number of pixels counted from a center of the first image, and setting information including an angle of depression of the image pickup device, an angle in a lateral direction of the image pickup device, a rotation angle of the image pickup device, and a distance between the image pickup device and the road surface.

2. The non-transitory storage medium according to claim 1, wherein the extracting includes:

setting a region on which a process of extracting the first feature point is to be performed in an entire region of the first image in accordance with a vanishing point and a horizon included in the first image; and

extracting at least one feature point from the region.

3. A non-transitory storage medium storing a distance calculation program that causes a computer to execute a process, the process comprising:

obtaining a first image captured by an image pickup device;

extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and

calculating a first distance from the first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and a first angle of the point viewed from a center of the image pickup device in the real space, and setting information including an angle of depression of the image pickup device and a distance between the image pickup device and the road surface, wherein the calculating includes;

calculating the first angle of the first feature point on the road surface relative to the image pickup device in the real space in accordance with the data and the position of the first feature point included in the first image; and

calculating the first distance in the real space in accordance with a trigonometric function calculation using the first angle of the first feature point and the setting information of the image pickup device.

4. The non-transitory storage medium according to claim 3, wherein the process further comprises:

extracting a second feature point which satisfies the extraction condition from a second image which is captured by the image pickup device after the first image is captured;

calculating, when the first feature point in the first image and the second feature point in the second image indicate the same feature point of the same object, a second distance from the second feature point to the image pickup device in the real space in accordance with the data and the setting information; and

calculating a traveling amount of a vehicle in which the image pickup device is disposed in accordance with a difference between the first distance and the first angle obtained when the first image is captured and the second distance and a second angle obtained when the second image is captured.

5. The non-transitory storage medium according to claim 4,

wherein the extracting the first feature point includes:

setting a first region on which a process of extracting the first feature point is to be performed in an entire region of the first image in accordance with a vanishing point and a horizon included in the first image; and

extracting at least one first feature point from the first region, and

wherein the extracting the second feature point includes;

setting a second region larger than the first region in the second image; and

extracting at least one second feature point from the second region.

6. The non-transitory storage medium according to claim 4, wherein the process further comprises:

determining a motion of the vehicle in accordance with an optical flow in the first image and an optical flow in the second image, and

wherein the calculating the traveling amount includes;

calculating the traveling amount by a calculation method corresponding to the motion.

7. The non-transitory storage medium according to claim 6, wherein the determining the motion includes:

determining whether the vehicle is traveling straight or the vehicle is turning left or right, and

wherein the calculating the traveling amount includes;

executing a traveling-amount calculation process to be performed at a time when the vehicle is traveling straight when it is determined that the vehicle is traveling straight; and

executing a traveling-amount calculation process to be performed at a time when the vehicle is rounding a curve when it is determined that the vehicle is turning left or right.

8. The non-transitory storage medium according to claim 4, wherein the process further comprises:

determining a motion of the vehicle in accordance with the traveling amount, as a first traveling amount, obtained in accordance with the first and second images captured by the image pickup device serving as a first image pickup device and a second traveling amount obtained in accordance with images captured by a second image pickup device which is installed in the vehicle and which has setting information different from that of the first image pickup device.

9. A distance calculation apparatus comprising;

a processor to execute;

obtaining a first image captured by an image pickup device;

extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and

calculating a first distance from the extracted first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and an angle of the point viewed from a center of the image pickup device in the real space, the position of the point in the first image being a distance represented by a number of pixels counted from a center of the first image, and setting information including an angle of depression of the image pickup device, an angle in a lateral direction of the image pickup device, a rotation angle of the image pickup device, and a distance between the image pickup device and the road surface, and

a memory, coupled to the processor, to store the data and the setting information.

10. The distance calculation apparatus according to claim 9,

wherein the processor is configured, in the extracting, to

set a region on which a process of extracting the first feature point is performed in an entire region of the first image in accordance with a vanishing point and a horizon included in the first image, and

extract at least one feature point from the region.

11. A distance calculation apparatus comprising;

a processor to execute;

obtaining a first image captured by an image pickup device;

extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and

calculating a first distance from the extracted first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and an angle of the point viewed from a center of the image pickup device in the real space, and setting information including an angle of depression of the image pickup device and a distance between the image pickup device and the road surface, and

a memory, coupled to the processor, to store the data and the setting information,

wherein the processor is further configured, in the calculating, to

calculate the first angle of the first feature point on the road surface relative to the image pickup device in the real space in accordance with the data and the position of the first feature point included in the first image, and

calculate the first distance in the real space in accordance with a trigonometric function calculation using the first angle of the first feature point and the setting information of the image pickup device.

12. The distance calculation apparatus according to claim 11,

wherein the processor is further configured to

extract a second feature point which satisfies the extraction condition from a second image which is captured by the image pickup device after the first image is captured,

calculate, when the first feature point in the first image and the second feature point in the second image indicate the same feature point of the same object, a second distance from the second feature point to the image pickup device in the real space in accordance with the data and the setting information, and

calculate a traveling amount of a vehicle in which the image pickup device is disposed in accordance with a difference between the first distance and the first angle obtained when the first image is captured and the second distance and a second angle obtained when the second image is captured.

13. The distance calculation apparatus according to claim 12,

wherein the processor is further configured to

determine a motion of the vehicle in accordance with the traveling amount, as a first traveling amount, obtained in accordance with the first and second images captured by the image pickup device serving as a first image pickup device and a second traveling amount obtained in accordance with images captured by a second image pickup device which is installed in the vehicle and which has setting information different from that of the first image pickup device.

14. The distance calculation apparatus according to claim 12,

wherein the processor is configured, in the extracting the first feature point, to

set a region on which the process of extracting the first feature point is to be performed in an entire region of the first image in accordance with a vanishing point and a horizon included in the first image, and

extract at least one feature point from the region, and

the processor is configured, in the process of extracting the second feature point, to

set a second region larger than the first region in the second image, and

extract at least one second feature point from the second region.

15. The distance calculation apparatus according to claim 12,

wherein the processor is further configured to

determine a motion of the vehicle in accordance with an optical flow in the first image and an optical flow in the second image, and

calculate the traveling amount by a calculation method corresponding to the motion in the process of calculating a traveling amount.

16. The distance calculation apparatus according to claim 15,

wherein the processor is configured, in the determining the motion, to

determine whether the vehicle is traveling straight or the vehicle is turning left or right, and

the processor is configured, in the calculating a traveling amount, to

execute a traveling-amount calculation process to be performed at a time when the vehicle is traveling straight when it is determined that the vehicle is traveling straight, and

execute a traveling-amount calculation process to be performed at a time when the vehicle is rounding a curve when it is determined that the vehicle is turning left or right.

17. A method comprising:

obtaining a first image captured by an image pickup device;

extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and

calculating, by a processor, a first distance from the first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and a first angle of the point viewed from a center of the image pickup device in the real space, the position of the point in the first image being a distance represented by a number of pixels counted from a center of the first image, and setting information including an angle of depression of the image pickup device, an angle in a lateral direction of the image pickup device, a rotation angle of the image pickup device, and a distance between the image pickup device and the road surface.

18. The method according to claim 17, further comprising:

setting a region on which a process of extracting the first feature point is to be performed in an entire region of the first image in accordance with a vanishing point and a horizon included in the first image; and

extracting at least one feature point from the region.

19. A method comprising:

obtaining a first image captured by an image pickup device;

extracting a first feature point which satisfies an extraction condition used to determine a point on a road surface in the first image; and

calculating, by a processor, a first distance from the first feature point to the image pickup device in a real space in accordance with data obtained by associating a position of the point in the first image and a first angle of the point viewed from a center of the image pickup device in the real space, and setting information including an angle of depression of the image pickup device and a distance between the image pickup device and the road surface,

wherein the calculating includes

calculating the first angle of the first feature point on the road surface relative to the image pickup device in the real space in accordance with the data and the position of the first feature point included in the first image; and

calculating the first distance in the real space in accordance with a trigonometric function calculation using the first angle of the first feature point and the setting information of the image pickup device.

20. The method according to claim 19, further comprising:

extracting a second feature point which satisfies the extraction condition from a second image which is captured by the image pickup device after the first image is captured;

calculating, when the first feature point in the first image and the second feature point in the second image indicate the same feature point of the same object, a second distance from the second feature point to the image pickup device in the real space in accordance with the data and the setting information; and

calculating a traveling amount of a vehicle in which the image pickup device is disposed in accordance with a difference between the first distance and the first angle obtained when the first image is captured and the second distance and a second angle obtained when the second image is captured.