MULTI-LENS CAMERA SYSTEM, VEHICLE MOUNTING THE MULTI-LENS CAMERA SYSTEM, AND RANGE-FINDING METHOD EXECUTED BY THE MULTI-LENS CAMERA SYSTEM

• US 20130250109A1
• Filed: 03/18/2013
• Published: 09/26/2013
• Est. Priority Date: 03/21/2012
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: YOKOTA, Soichiro
• Granted Patent: US 9,420,151 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images

Abstract

A multi-lens camera system includes multiple camera units including respective optical filters and image sensors that acquire a captured image via the optical filter, each of the optical filters having multiple filter regions whose optical characteristics differ respectively, and an image processor, operably connected to the multiple camera unit, to execute different types of image processing on the captured image to form an image that contains multiple image regions whose characteristics differ depending on the optical characteristics of the filter region in which they are acquired.

Claims

1. A multi-lens camera system comprising:

multiple camera units including respective optical filters and image sensors that acquire captured images via the optical filters, each of the optical filters having multiple filter regions whose optical characteristics differ respectively; and

an image processor operably connected to the multiple camera units, to execute different types of image processing on the captured image to form an image that contains multiple image regions whose characteristics differ depending on the optical characteristics of the filter regions.

2. The multi-lens camera system according to claim 1, wherein the image processor performs a parallax calculation using image information for a specified image region.

3. The multi-lens camera system according to claim 2, wherein the image processor performs the parallax calculation for multiple image regions, and performs the parallax calculation based on different image information depending on the image regions.

4. The multi-lens camera system according to claim 2, wherein the image processor recognizes objects in the captured images by multiple different image recognitions method differing depending on the image regions.

5. The multi-lens camera system according to claim 4, wherein the image processor recognizes the objects in the captured image using parallax information obtained between first image regions in the captured images in the respective camera units and color information in a second image region.

6. The multi-lens camera system according to claim 4, wherein the image processor recognizes the objects in the captured image using parallax information obtained between first image regions in the captured images in the respective camera units and luminance information in a third image region.

7. A vehicle comprising:

a multi-lens camera system comprising;

multiple camera units including respective optical filters and image sensors that acquire captured image via the optical filter, each of the optical filter having multiple filter regions whose optical characteristics differ respectively; and

an image processor, operably connected to the multiple camera units, to process images into multiple image regions whose characteristics differ depending on the optical characteristics of the filter regions to recognize various types of targets positioned forward of the vehicle; and

a vehicle drive control unit to control steering and braking in the vehicle based on the recognition result provided by the image processor.

8. The vehicle according to claim 7, wherein the image processor of the multi-lens camera system divides the captured image into the multiple image regions having a central region where at least one of another vehicle in front, a pedestrian, a motorbike and bicycle is recognized as the target using parallax calculation.

9. The vehicle according to claim 8, wherein the image processor of the multi-lens camera system is configured to:

calculate parallax information for the central region in the captured image, using luminance information of the central region;

perform clustering using the parallax information;

recognize a dividing line on a road, using the luminance information of the central region;

modify the clustering results to correspond to the size of the vehicle in front, the pedestrian, the motorbike and the bicycle;

calculate a height of a road based on the dividing line to set the vehicle in front, the pedestrian, the motorbike and the bicycle on the road as target information;

modify the clustering results, using the target information;

classify the vehicle in front, the pedestrian, the motorbike and the bicycle in the clustered central region; and

recognize the target in the central region in the captured image in three dimensions.

10. The vehicle according to claim 9, wherein the image processor of the multi-lens camera system divides the captured image into the multiple image regions having an upper region where at least one of a traffic light and a traffic sign is recognized as the target using color information.

11. The vehicle according to claim 10, wherein the image processor of the multi-lens camera system configured to:

recognize the traffic light and the traffic sign in the upper region, using color information of the upper region while the image processor calculates the parallax information for the central region in the captured image, using the luminance information and performs clustering using the parallax information for the central region,

calculate parallax only for the area where the traffic light or the traffic sign is recognized in the upper region, using the color information of the upper region if needed, while the image processor modifies the clustering results to correspond to the size of another vehicle in front, the pedestrian, the motorbike, and the bicycle, calculates the height of the road based on the dividing line to set the vehicle in front, the pedestrian, the motorbike and the bicycle on the road as target information, modifies the clustering results, using the target information, and classifies the vehicle in front, the pedestrian, the motorbike and the bicycle in the clustered central region; and

recognize the target in the upper region of the captured image in three dimensions.

12. The vehicle according to claim 9, wherein the image processor of the multi-lens camera system divides the captured image into the multiple image regions having a lower region where a raindrop is detected, using luminance information.

13. The vehicle according to claim 12, wherein the image processor of the multi-lens camera system detects the raindrop for the lower region, while calculating parallax information and performing clustering using the parallax information for the central region.

14. A range-finding method in a multi-lens camera system that has multiple camera units including respective optical filters and image sensors that acquire captured image via the optical filters, each of the optical filter having multiple filter regions whose optical characteristics differ respectively;

the method comprising the steps of;

capturing images;

dividing each of the captured images into multiple image regions corresponding to the respective filter regions;

calculating parallax information for a first image region in the captured image where a first target object is recognized;

performing clustering using the parallax information for the first image region;

recognizing a second target object for a second image region in the captured image while calculating parallax information and performing clustering using the parallax information for the first image region;

modifying the clustering result;

calculating parallax information only for the area where the second target object is recognized in the second image region, while modifying the clustering result for the first image region; and

recognizing the targets in the first image region and the second image region in the captured image in three dimensions.

15. The range-finding method according to claim 14, further comprising:

detecting a raindrop for a third region in the captured image, while calculating parallax information and performing clustering using the parallax information for the first image region.