Vehicle environment monitoring device, vehicle environment monitoring method, control program and computer-readable recording medium

• US 7,190,281 B2
• Filed: 02/04/2005
• Issued: 03/13/2007
• Est. Priority Date: 02/06/2004
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.), Murakami Corporation
• Inventors: Sato, Hidenori, Nagasawa, Shunsuke, Unno, Noriyuki
• Primary Examiner(s): Pope; Daryl C
• Application Number:
  US11/049,757
• Publication Number:
  US 20050190952A1
• Time in Patent Office: 767 Days
• Field of Search: 340/903, 340/904, 340/937, 340/425.5, 340/435, 340/436, 700/300, 700/301
• US Class Current: 340/903
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A vehicle environment monitoring device comprises an imaging section for taking image data of an environment of an automobile, the image including at least two types of colors; a setting section for setting a specific area of the image data obtained by the imaging section, wherein the specific area has a strip shape longer in, and parallel to, a horizontal direction of a frame image and includes at least two types of colors; an extraction section for taking in image data of the specific area in units of one or a plurality of frame images taken by the imaging section in time series and extracting moving vector information based on the image data of the specific area; and a detection section for detecting a second vehicle which is present in an environment of a first vehicle having the vehicle environment monitoring device mounted thereon, based on the moving vector information.

Claims

1. A vehicle environment monitoring device, comprising:

an imaging section for taking image data of an environment of an automobile, the image data including at least two colors;

a setting section for setting a specific area of the image data obtained by the imaging section, wherein the specific area has a strip shape longer in, and parallel to, a horizontal direction of a frame image and includes at least two colors;

an extraction section for taking in image data of the specific area in units of one or a plurality of frame images taken by the imaging section in time series and extracting moving vector information based on the image data of the specific area; and

a detection section for detecting a second vehicle which is present in an environment of a first vehicle having the vehicle environment monitoring device mounted thereon, based on the moving vector information.

2. A vehicle environment monitoring device according to claim 1, wherein the specific area including the two colors encompasses a border between a windshield and a hood of the second vehicle.

3. A vehicle environment monitoring device according to claim 1, wherein the setting section sets the specific area of the image data obtained by the imaging section, such that the specific area does not encompass an image of the first vehicle and to encompass at least a part of the second vehicle.

4. A vehicle environment monitoring device according to claim 1, wherein:

the setting section sets a plurality of specific areas of the image data taken by the imaging section, wherein positions of the plurality of specific areas are the same in the horizontal direction and different in a vertical direction;

the extraction section takes in the image data of each of the specific areas and extracting moving vector information; and

the detection section adds vector values in an approach direction when the moving vector information in each of the plurality of specific areas represents the approach direction, and determining that there is at least one of the second vehicle approaching the first vehicle and the second vehicle running at the same speed as the first vehicle when a total sum of all the plurality of divided areas is larger than a predetermined threshold value.

5. A vehicle environment monitoring device according to claim 1, wherein the imaging section is attached to the first vehicle.

6. A vehicle environment monitoring device according to claim 5, wherein the imaging section is attached to such a position that the imaging section is capable of taking an image of at least one of a left rear environment and a right rear environment of the first vehicle.

7. A vehicle environment monitoring device according to claim 6, further comprising an input image switching section switchable to select either image data from a right side imaging section or image data from a left side imaging section, and supplying the selected image data to the extraction section.

8. A vehicle environment monitoring device according to claim 1, wherein the extraction section compares a sum of current image data and a sum of previous image data time t before (t>

0) for each of a plurality of divided areas obtained by dividing the specific area, extracting a divided area of the current image data and a divided area of the previous image data which have a minimum difference between the sums thereof, determining that the position of the extracted divided area of the previous image data is the position of the extracted divided area of the current image data time t before, and extracting a positional relationship between the extracted divided area of the previous image data and the extracted divided area of the current image data as the moving vector information.

9. A vehicle environment monitoring device according to claim 8, wherein the extraction section includes:

a dividing section for dividing the image data of the specific area which has been taken in into a plurality of divided areas, wherein the image data of the specific area is either image data taken by a right side imaging section or image data taken by a left side imaging section which is selected by switching in accordance with a direction indicated by a direction indicator;

a sum calculation section for calculating a sum of image data values of each of the plurality of divided areas;

a divided area extraction section for extracting a divided area of the previous image data, a sum of which has a minimum difference from a sum of each of the plurality of divided areas of the current image data;

a running determination section for determining whether the first vehicle is running or not based on vehicle speed information from the first vehicle;

an approach threshold value setting section for setting one of two threshold values as a predetermined threshold value when the first vehicle is running, and setting the other threshold value as the predetermined threshold value when the first vehicle is not running; and

a moving vector information determination section for comparing the predetermined threshold value with a positional relationship between the position of the extracted divided area of the previous image data and the position of the extracted divided area of the current image data;

when a value of the positional relationship is larger than the predetermined threshold value, determining the moving vector information as approaching vector information; and

when the value of the positional relationship is equal to or smaller than the predetermined threshold value, determining the moving vector information as separating vector information.

10. A vehicle environment monitoring device according to claim 8, wherein the extraction section compares a predetermined threshold value with the positional relationship between the extracted divided area of the previous image data and the extracted divided area of the current image data, thus determining whether the second vehicle in the divided areas is approaching, separating away from, or running at the same speed as, the first vehicle, and determining the moving vector information.

11. A vehicle environment monitoring device according to claim 10, wherein the extraction section refers to vehicle speed information obtainable from the first vehicle to determine whether the first vehicle is running or not, and when the first vehicle is running, setting the predetermined threshold value to be lower than the threshold value when the first vehicle is not running.

12. A vehicle environment monitoring device according to claim 1, wherein the detection section detects a relative moving state between the first vehicle and the second vehicle based on the moving vector information.

13. A vehicle environment monitoring device according to claim 12, wherein the detection section is capable of detecting that the second vehicle is approaching the first vehicle based on the moving vector information.

14. A vehicle environment monitoring device according to claim 12, wherein the detection section is capable of detecting that the second vehicle is running at the same speed as the first vehicle.

15. A vehicle environment monitoring device according to claim 12, wherein the detection section reads the moving vector information of each of the plurality of divided areas, and when a sum of approach direction vector values, among the moving vector information, is larger than a predetermined threshold value, detecting that there is the second vehicle approaching the first vehicle and/or the second vehicle running at the same speed as the first vehicle.

16. A vehicle environment monitoring device according to claim 1, further comprising a memory section for temporarily storing a result of the extraction performed by the extraction section, wherein the detection section reads the moving vector information from the memory section and uses the moving vector information for the detection.

17. A vehicle environment monitoring device according to claim 12, wherein the detection section includes:

a moving vector value calculation section for detecting whether the moving vector information indicates “

close”

or “

far”

in each of the plurality of divided areas, and when the moving vector information indicates “

close”

, adding a vector value in an approach direction; and

a second vehicle determination section for, when the sum of the vector values in the approach direction is larger than a determination threshold value, determining that there is the second vehicle approaching the first vehicle and/or there is the second vehicle running at the same speed as the first vehicle.

18. A vehicle environment monitoring device according to claim 12, further comprising an approach notification section for outputting an alarm in accordance with a relative moving state between the first vehicle and the second vehicle, the relative moving state being detected by the detection section.

19. A vehicle environment monitoring device according to claim 18, wherein:

the detection section detects whether or not there is the second vehicle approaching the first vehicle and/or the second vehicle running at the same speed as the first vehicle in a direction indicated by a direction indicator; and

the approach notification section outputs an alarm to the environment of the first vehicle when the detection section detects that there is at least one of the second vehicle approaching the first vehicle and/or the second vehicle running at the same speed as the first vehicle in the direction indicated by the direction indicator.

20. A vehicle environment monitoring method, comprising:

an imaging step of taking image data of an environment of an automobile as frame images in time series, the image data including at least two types of colors;

a setting step of setting a specific area of the image data in units of one or a plurality of frame images taken by the imaging section, wherein the specific area has a strip shape longer in, and parallel to, a horizontal direction of the frame images and includes at least two types of colors;

an extraction step of taking in image data of the specific area in units of one or a plurality of frame images taken by the imaging section and extracting moving vector information based on the image data of the specific area; and

a detection step of detecting a second vehicle which is present in an environment of a first vehicle having the vehicle environment monitoring device mounted thereon, based on the moving vector information.

21. A vehicle environment monitoring method according to claim 20, wherein the extraction step includes the steps of:

taking in a direction indicated by a direction indicator;

switching to select either image data taken by a right side imaging section or image data taken by a left side imaging section which corresponds to the direction which has been taken in;

taking in the image area of the set specific area among the image data of the frame image selected by switching;

dividing the specific area into a plurality of divided areas;

calculating a sum of image data values of each of the plurality of divided areas;

reading sum data stored in a memory section as previous image data time t before for each of the plurality of divided areas;

comparing a sum of current image data and a sum of the previous image data, and extracting a divided area of the previous image data, a sum of which has a minimum difference from a sum of each of the plurality of divided areas of the current image data;

determining whether the first vehicle is running or not based on vehicle speed information from the first vehicle;

setting one of two threshold values as a predetermined threshold value when the first vehicle is running, and setting the other threshold value as the predetermined threshold value when the first vehicle is not running; and

comparing the predetermined threshold value with a positional relationship between the position of the extracted divided area of the previous image data and the position of the extracted divided area of the current image data;

when a value of the positional relationship is larger than the predetermined threshold value, determining the moving vector information as approaching vector information; and

when the value of the positional relationship is equal to or smaller than the predetermined threshold value, determining the moving vector information as separating vector information.

22. A vehicle environment monitoring method according to claim 20, wherein the detection step includes the steps of:

detecting whether the moving vector information indicates “

close”

or “

far”

in each of the plurality of divided areas, and when the moving vector information indicates “

close”

, adding a vector value in an approach direction; and

when the sum of the vector values in the approach direction is larger than a determination threshold value, determining that there is the second vehicle approaching the first vehicle and/or there is the second vehicle running at the same speed as the first vehicle.

23. A control program for causing a computer to carry out a vehicle environment monitoring method according to claim 20.

24. A computer-readable recording medium having a control program according to claim 23 recorded thereon.