Image processing apparatus, image processing system, and image processing method

• US 8,503,729 B2
• Filed: 12/11/2011
• Issued: 08/06/2013
• Est. Priority Date: 02/24/2011
• Status: Active Grant

• Current Assignee: Socionext Inc.
• Original Assignee: Fujitsu Semiconductor Limited (Fujitsu Limited)
• Inventors: Kumagai, Akira
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/316,513
• Publication Number:
  US 20120219190A1
• Time in Patent Office: 604 Days
• Field of Search: None
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching

Abstract

An image processing apparatus: detects an object position included within an overlapping region of a first image corresponding to a vehicles'"'"'s traveling direction and a second image corresponding to a direction crossing the traveling direction; assigns the first image a first value when the object is in a first position, and a second value smaller than the first value when the object is in a second position more distant from a traveling direction axis of the vehicle than the first position, and assigns the second image a third value when the object is in the first position, and a fourth value larger than the third value when the object is in the second position; and determines a boundary so that the object is included within an image having a larger value than the other; and joins the first and second images.

Claims

1. An image processing apparatus comprising:

a detecting unit which detects a position of an object included within an overlapping region of a first image corresponding to a traveling direction of a vehicle or an opposite direction to the traveling direction and a second image corresponding to a direction crossing the traveling direction of the vehicle;

a processing unit which assigns the first image a first evaluation value when the object is in a first position, and a second evaluation value, which is smaller than the first evaluation value, when the object is in a second position, of which a distance from a traveling direction axis of the vehicle is larger than the same of the first position, and assigns the second image a third evaluation value when the object is in the first position, and a fourth evaluation value, which is larger than the third evaluation value, when the object is in the second position;

a determining unit which determines a boundary of the first and second images for joining the first and second images, so that the object is included within either of the first and second images, which has a larger evaluation value than the other; and

a generating unit which joins the first and second images along the boundary and generates an image, in which a region around the vehicle is viewed from a prescribed view point.

2. The image processing apparatus according to claim 1, wherein

the processing unit assigns the first and second images evaluation values, so that the evaluation values assigned to the first and second images when the object displaces in the opposite direction to the traveling direction of the vehicle are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces in the traveling direction.

3. The image processing apparatus according to claim 1, wherein

the processing unit assigns the first and second images evaluation values, so that the evaluation values assigned to the first and second images when the object displaces approaching to the traveling direction axis are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces departing from the traveling direction axis.

4. The image processing apparatus according to claim 1, wherein

the processing unit assigns the first and second images evaluation values, so that, in a case that the vehicle makes a turn to a turn direction, the evaluation values assigned to the first and second images when the object displaces in the turn direction are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces in the turn direction.

5. The image processing apparatus according to claim 1, wherein

the processing unit assigns the first and second images evaluation values, so that, in a case that the vehicle makes a turn to a turn direction, the evaluation values assigned to the first and second images when the object displaces by a first displacement amount per unit time at the first and second positions are respectively larger than the evaluation values assigned to the first and second images when the object displaces by a second displacement amount per unit time, which is smaller than the first displacement amount per unit time, at the first and second positions.

6. The image processing apparatus according to claim 1, wherein

the processing unit assigns the first and second images evaluation values, so that the evaluation values assigned to the first and second images when a size of the object is a first size are respectively larger than the evaluation values assigned to the first and second images when the size of the object is a second size which is smaller than the first size.

7. The image processing apparatus according to claim 1, wherein

the determining unit changes the previously determined boundary to a new determined boundary when a time-dependent change amount of a difference between the evaluation values assigned to the first and second images is equal to or greater than a reference amount, or maintains the previously determined boundary when the change amount is less than the reference amount.

8. The image processing apparatus according to claim 1, wherein

the determining unit changes the previously determined boundary to a newly determined boundary when the evaluation values assigned to the first and second images are equal to or larger than a reference value, or maintains the previously determined boundary when the evaluation values assigned to the first and second images are less than the reference value.

9. An image processing system comprising:

a detecting unit which detects a position of an object included within an overlapping region of a first image corresponding to a traveling direction of a vehicle or an opposite direction to the traveling direction and a second image corresponding to a direction crossing the traveling direction of the vehicle;

a processing unit which assigns the first image a first evaluation value when the object is in a first position, and a second evaluation value, which is smaller than the first evaluation value, when the object is in a second position of which a distance from a traveling direction axis of the vehicle is larger than the same of the first position, and assigns the second image a third evaluation value when the object is in the first position, and a fourth evaluation value, which is larger than the third evaluation value, when the object is in the second position;

a determining unit which determines a boundary of the first and second images for joining the first and second images, so that the object is included within either of the first and second images, which has a larger evaluation value than the other;

a generating unit which joins the first and second images along the boundary and generates an image, in which a region around the vehicle is viewed from a prescribed view point; and

a displaying apparatus which displays the image, in which the region around the vehicle is viewed from the prescribed view point.

10. The image processing system according to claim 9, wherein

the processing unit assigns the first and second images evaluation values, so that the evaluation values assigned to the first and second images when the object displaces in the opposite direction to the traveling direction of the vehicle are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces in the traveling direction.

11. The image processing system according to claim 9, wherein

the processing unit assigns the first and second images evaluation values, so that the evaluation values assigned to the first and second images when the object displaces approaching to the traveling direction axis are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces departing from the traveling direction axis.

12. The image processing system according to claim 9, wherein the processing unit assigns the first and second images evaluation values, so that, in a case that the vehicle makes a turn to a turn direction, the evaluation values assigned to the first and second images when the object displaces in the turn direction are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces in the turn direction.

13. The image processing system according to claim 9, wherein the processing unit assigns the first and second images evaluation values, so that, in a case that the vehicle makes a turn to a turn direction, the evaluation values assigned to the first and second images when the object displaces by a first displacement amount per unit time at the first and second positions are respectively larger than the evaluation values assigned to the first and second images when the object displaces by a second displacement amount per unit time, which is smaller than the first displacement amount per unit time, at the first and second positions.

14. The image processing system according to claim 9, wherein

the processing unit assigns the first and second images evaluation values, so that the evaluation values assigned to the first and second images when a size of the object is a first size are respectively larger than the evaluation values assigned to the first and second images when the size of the object is a second size which is smaller than the first size.

15. An image processing method comprising:

detecting a position of an object included within an overlapping region of a first image corresponding to a traveling direction of a vehicle or an opposite direction to the traveling direction and a second image corresponding to a direction crossing the traveling direction of the vehicle;

assigning the first image a first evaluation value when the object is in a first position, and a second evaluation value, which is smaller than the first evaluation value, when the object is in a second position of which a distance from a traveling direction axis of the vehicle is larger than the same of the first position;

assigning the second image a third evaluation value when the object is in the first position, and a fourth evaluation value, which is larger than the third evaluation value, when the object is in the second position;

determining a boundary of the first and second images for joining the first and second images, so that the object is included within either of the first and second images, which has a larger evaluation value than the other; and

joining the first and second images along the boundary to generate an image, in which a region around the vehicle is viewed from a prescribed view point.

16. The image processing method according to claim 15, wherein

the evaluation values are assigned to the first and second images, so that the evaluation values assigned to the first and second images when the object displaces in the opposite direction to the traveling direction of the vehicle are respectively larger than evaluation values assigned to the first and second images when the object at the same position displaces in the traveling direction.

17. The image processing method according to claim 15, wherein

the evaluation values are assigned to the first and second images, so that the evaluation values assigned to the first and second images when the object displaces approaching to the traveling direction axis are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces departing from the traveling direction axis.

18. The image processing method according to claim 15, wherein

the evaluation values are assigned to the first and second images, so that, in a case that the vehicle makes a turn to a turn direction, the evaluation values assigned to the first and second images when the object displaces in the turn direction are respectively larger than the evaluation values assigned to the first and second images when the object at the same position displaces in the turn direction.

19. The image processing method according to claim 15, wherein

the evaluation values are assigned to the first and second images, so that, in a case that the vehicle makes a turn to a turn direction, the evaluation values assigned to the first and second images when the object displaces by a first displacement amount per unit time at the first and second positions are respectively larger than the evaluation values assigned to the first and second images when the object displaces by a second displacement amount per unit time, which is smaller than the first displacement amount per unit time, at the first and second positions.