Mobile body surrounding surveillance apparatus, mobile body surrounding surveillance method, control program and computer-readable recording medium

• US 7,253,389 B2
• Filed: 02/01/2006
• Issued: 08/07/2007
• Est. Priority Date: 02/01/2005
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Yoshida, Shinichi
• Primary Examiner(s): LE, QUE TAN
• Application Number:
  US11/344,541
• Publication Number:
  US 20060171562A1
• Time in Patent Office: 552 Days
• Field of Search: 250/208.1, 250/221, 250/201.1, 250/559.05, 340435-438
• US Class Current: 250/208.1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A mobile body surrounding surveillance 1A apparatus according to the present invention includes: an image capturing section 2; a particular region setting section 31 for setting a particular region for image data captured by the image capturing section; an own mobile body movement direction detection section 11 for detecting a movement direction of the own mobile body and outputting a movement direction information indicating the detected movement direction; a movement vector information extraction section for taking the image data of the particular region for each of one or a plurality of frame images captured by the image capturing section in time series and adjusting a detection sensitivity based on the movement direction information so as to extract a movement vector information on another mobile body; and a change detection section 35 for detecting a moving state of the other mobile body which is present in the surrounding of the own mobile body based on the movement vector information.

Claims

1. A mobile body surrounding surveillance apparatus comprising:

an image capturing section for capturing an image in a surrounding of an own mobile body;

a particular region setting section for setting a particular region for image data captured by the image capturing section, the particular region being parallel to the horizontal direction of a frame image and rectangular having a long edge in the horizontal direction;

an own mobile body movement direction detection section for detecting a movement direction of the own mobile body and outputting a movement direction information indicating the detected movement direction;

a movement vector information extraction section for taking the image data of the particular region for each of one or a plurality of frame images captured by the image capturing section in time series and adjusting a detection sensitivity based on the movement direction information so as to extract a movement vector information on another mobile body; and

a change detection section for detecting a moving state of the other mobile body which is present in the surrounding of the own mobile body based on the movement vector information.

2. A mobile body surrounding surveillance apparatus according to claim 1, wherein the image capturing section is mounted on a position of the own mobile body such that the image capturing section can capture an image in a rear and side direction.

3. A mobile body surrounding surveillance apparatus according to claim 2, wherein the own mobile body is a vehicle, and the image capturing section is mounted on a side mirror or a door mirror of at least one of a side of a driver'"'"'s seat and a side opposite to the driver'"'"'s seat.

4. A mobile body surrounding surveillance apparatus according to claim 1, wherein the particular region setting section sets the particular region for the captured image data such that a captured image of the own mobile body is not included in the image data and at least a portion of a captured image of the other mobile body is included in the image data.

5. A mobile body surrounding surveillance apparatus according to claim 1, wherein the own mobile body movement direction detection section detects the movement direction of the own mobile body in accordance with an operation direction when a moving direction control section of the own mobile body is operated.

6. A mobile body surrounding surveillance apparatus according to claim 5, wherein the moving direction control section is a steering wheel, an operation angle information of the steering wheel increases or decreases in accordance with an operation angle in a positive direction or a negative direction, the positive direction being defined as a direction when the steering wheel is steered to the right and the negative direction is defined as a direction when the steering wheel is steered to the left.

7. A mobile body surrounding surveillance apparatus according to claim 1, wherein the movement vector information includes a movement amount and direction information indicating a moving state of the other mobile body which is present in the surrounding of the own mobile body.

8. A mobile body surrounding surveillance apparatus according to claim 1, wherein the movement vector information extraction section compares a sum value of a current image data and a sum value of a previous image data time t(t>

0) before for each of a plurality of divided regions of the particular region, respectively;

extracts divided regions between which the difference of the sum values is minimum;

determines a position of the extracted divided region of the previous image data as a position of the divided region of the current image data time t before; and

extracts a positional relationship between the extracted divided region of the previous image data and the extracted divided region of the current image data as the movement amount of the movement vector information and extracts the direction information of the movement vector information based on the movement direction information.

9. A mobile body surrounding surveillance apparatus according to claim 8, wherein the movement vector information extraction section determines whether the divided region of the current image data is shifted to the left or right with respect to the divided region of the previous image data based on the positional relationship between the extracted divided regions;

increases or decreases a sensitivity setting threshold which is used to extract the direction information of the movement vector information in accordance with the movement direction information;

compares the movement amount of the movement vector information and the sensitivity setting threshold; and

extracts the direction information of the movement vector information which indicates that the other mobile body is approaching the own mobile body, when the movement amount of the movement vector information is greater than or equal to the sensitivity setting threshold.

10. A mobile body surrounding surveillance apparatus according to claim 7, wherein the movement vector information extraction section includes:

a dividing section for dividing image data of the particular region which is a portion of image data of the captured image data into a plurality of divided regions;

a sum value calculating section for calculating a sum of the image data values for each divided region;

a divided region extraction section for extracting one of the divided regions of previous image data having a minimum difference in sum value between the previous image data and current image data, for each divided region of the current image data;

a left/right determination section for determining whether the divided region of the current image data which gives the minimum difference is located to the left or right with respect to the extracted divided region of the previous image data; and

a movement vector information determination section for determining a direction information of the movement vector information as left movement vector information when the divided region of the current image data is located to the left and as right movement vector information when the divided region of the current image data is located to the right.

11. A mobile body surrounding surveillance apparatus according to claim 10, wherein the own mobile body is a vehicle, and the left/right determination section includes:

an operation angle increasing determination section for determining whether or not the operation angle of a steering wheel of the own mobile body is increasing;

a detection sensitivity control section for, when the operation angle is increasing, increasing a sensitivity setting threshold used to determine whether or not the other mobile body is approaching the own mobile body based on the movement amount of the movement vector information so as to weaken the detection sensitivity for the movement vector information and, when the operation angle is not increasing, maintaining or decreasing the sensitivity setting threshold so as to maintain or strengthen the detection sensitivity for the movement vector information; and

a comparison and determination section for comparing the movement amount of the movement vector information with the sensitivity setting threshold and determines whether or not the movement amount of the movement vector information is greater than or equal to the sensitivity setting threshold so as to determine whether the divided region of the current image data which gives the minimum difference is located to the left or right.

12. A mobile body surrounding surveillance apparatus according to claim 10, wherein the own mobile body is a vehicle, and the left/right determination section includes:

an operation angle increasing determination section for determining whether or not the operation angle of a steering wheel of the own mobile body is increasing;

a detection sensitivity control section for, when the operation angle is increasing, decreasing a sensitivity setting threshold used to determine whether or not the other mobile body is approaching the own mobile body based on the movement amount of the movement vector information so as to strengthen the detection sensitivity for the movement vector information and, when the operation angle is not increasing, maintaining or increasing the sensitivity setting threshold so as to maintain or weaken the detection sensitivity for the movement vector information; and

a comparison and determination section for comparing the movement amount of the movement vector information with the sensitivity setting threshold and determines whether or not the movement amount of the movement vector information is greater than or equal to the sensitivity setting threshold so as to determine whether the divided region of the current image data which gives the minimum difference is located to the left or right.

13. A mobile body surrounding surveillance apparatus according to claim 1, wherein the change detection section detects a moving state of the other mobile body relative to the own mobile body based on the movement vector information.

14. A mobile body surrounding surveillance apparatus according to claim 1, wherein the change detection section reads the movement vector information for each of the plurality of divided regions of the particular region;

and detects that there exists the other mobile body which is dangerously approaching the own mobile body when the sum value of the number of the vector movement amounts of a predetermined direction information among the read movement vector information is greater than a predetermined determination threshold.

15. A mobile body surrounding surveillance apparatus according to claim 14, wherein the change detection section calculates, as the direction information, the number of vector in the left direction when the image capturing section is mounted on the right side of the own mobile body and calculates, as the direction information, the number of vector in the right direction when the image capturing section is mounted on the left side of the own mobile body.

16. A mobile body surrounding surveillance apparatus according to claim 1, further comprising a change informing section for outputting a warning in accordance with the moving state of the other mobile body relative to the own mobile body, the moving state being detected by the change detection section.

17. A mobile body surrounding surveillance apparatus according to claim 1, further comprising a storage section for temporarily storing a result extracted by the movement vector information extraction section, wherein the change detection section reads the result extracted by the movement vector information extraction section from the storage section for processing the change detection.

18. A mobile body surrounding surveillance method comprising:

an image capturing step of capturing an image data in a surrounding of an own mobile body as a frame image, the image data being captured by an image capturing section in time series;

a setting step of setting a particular region which is parallel to the horizontal direction of the frame image and rectangular having a long edge in the horizontal direction for each of one or a plurality of captured frame images;

an own mobile body movement direction detection step of detecting a movement direction of the own mobile body and outputting a movement direction information indicating the detected movement direction;

a movement vector information extraction step of taking the image data of the particular region for each of one or the plurality of the captured frame images and extracting a movement vector information on another mobile body based on the image data of the particular region and the movement direction information; and

a change detection step of detecting a moving state of the other mobile body which is present in the surrounding of the own mobile body based on the movement vector information.

19. A mobile body surrounding surveillance method according to claim 18, wherein the movement vector information extraction step includes:

a dividing step of dividing image data of the particular region which is a portion of image data of the captured image data into a plurality of divided regions;

a sum value calculating step of calculating a sum of the image data values for each divided region;

a divided region extraction step of extracting one of the divided regions of previous image data having a minimum difference in sum value between the previous image data and current image data, for each divided region of the current image data;

a left/right determination step of determining whether the divided region of the current image data which gives the minimum difference is located to the left or right with respect to the extracted divided region of the previous image data; and

a movement vector information determination step of determining a direction information of the movement vector information as left movement vector information when the divided region of the current image data is located to the left and as right movement vector information when the divided region of the current image data is located to the right.

20. A mobile body surrounding surveillance method according to claim 19, wherein the own mobile body is a vehicle, and the left/right determination step includes:

an operation angle increasing determination step of determining whether or not the operation angle of a steering wheel of the own mobile body is increasing;

a detection sensitivity control step of, when the operation angle is increasing, increasing a sensitivity setting threshold used to determine whether or not the other mobile body is approaching the own mobile body based on the movement amount of the movement vector information so as to weaken the detection sensitivity for the movement vector information and, when the operation angle is not increasing, maintaining or decreasing the sensitivity setting threshold so as to maintain or strengthen the detection sensitivity for the movement vector information; and

a comparison and determination step of comparing the movement amount of the movement vector information with the sensitivity setting threshold and determines whether or not the movement amount of the movement vector information is greater than or equal to the sensitivity setting threshold so as to determine whether the divided region of the current image data which gives the minimum difference is located to the left or right.

21. A mobile body surrounding surveillance method according to claim 19, wherein the own mobile body is a vehicle, and the left/right determination step includes:

an operation angle increasing determination step of determining whether or not the operation angle of a steering wheel of the own mobile body is increasing;

a detection sensitivity control step of, when the operation angle is increasing, decreasing the sensitivity setting threshold used to determine whether or not the other mobile body is approaching the own mobile body based on the movement amount of the movement vector information so as to strengthen the detection sensitivity for the movement vector information and, when the operation angle is not increasing, maintaining or increasing the sensitivity setting threshold so as to maintain or weaken the detection sensitivity for the movement vector information; and

a comparison and determination step of comparing the movement amount of the movement vector information with the sensitivity setting threshold and determines whether or not the movement amount of the movement vector information is greater than or equal to the sensitivity setting threshold so as to determine whether the divided region of the current image data which give the minimum difference is located to the left or right.

22. A computer-readable recording medium having recorded thereon a control program for causing a computer to execute each step of the mobile body surrounding surveillance method according to claim 18.