Brightness and color matching video from multiple-camera system

• US 10,166,921 B2
• Filed: 02/10/2015
• Issued: 01/01/2019
• Est. Priority Date: 02/11/2014
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH, Robert Bosch LLC (Robert Bosch GmbH)
• Original Assignee: Robert Bosch GmbH
• Inventors: Sypitkowski, Greg
• Primary Examiner(s): Hess, Michael J
• Application Number:
  US15/114,252
• Publication Number:
  US 20170166126A1
• Time in Patent Office: 1,421 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/173 Reversing assist
• CPC Class Codes: B60K 2360/188 Displaying information using colour changes
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/349 Adjustment of brightness
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/211 producing three-dimensional [3D] effects, e.g. stereoscopic images
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 9/646 : for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

Systems and methods for adjusting color and brightness in multi-camera systems are disclosed. The system uses four cameras, having overlapping fields of view, mounted on the four sides of a vehicle. Color errors are determined for areas where the images from the rear and side cameras'"'"' fields of view overlap. Color gain factors are determined and applied for each of the side cameras (using the rear camera as “master”) to match the colors of the video outputs from the side cameras and the rear camera. The gains for the front view camera are then adjusted using gain factors based on the matched video outputs from the side view cameras using the side cameras as the “master” to the front camera “slave.” In this way, the rear camera indirectly acts as the master for the front camera and all cameras are ultimately color-matched to the rear camera.

Claims

1. A surround view system for a vehicle, the system comprising:

a first camera, positioned on the vehicle to capture a first video image including a first region of interest;

a second camera, positioned on the vehicle to capture a second video image including a second region of interest, the second camera being positioned such that the first region of interest at least partially overlaps with the second region of interest;

a third camera, positioned on the vehicle to capture a third video image including a third region of interest, the third camera positioned such that the third region of interest at least partially overlaps with the second region of interest;

a controller configured to

receive the first video image;

receive the second video image;

determine a first overlap region, the first overlap region including a portion of the first video image where the first region of interest overlaps with the second region of interest;

determine a first color error for the second camera, the first color error being indicative of a color difference between the first video image and the second video image within the first overlap region;

determine a first color gain factor for the second camera based on the first color error;

generate a first adjusted video input stream for the second camera by applying the first color gain factor to reduce the first color error;

receive an adjusted second video image from the second camera including an adjusted second region of interest;

receive the third video image;

determine a second overlap region, the second overlap region including a portion of the third video image where the third region of interest overlaps with the adjusted second region of interest;

determine a second color error for the third camera, the second color error being indicative of a second color difference between the third video image and the adjusted second video image within the second overlap region;

determine a second color gain factor for the third camera based on the second color error; and

generate a second adjusted video input stream for the third camera using the second color gain factor to reduce the second color error,

wherein the controller is further configured to adjust the first color gain factor and the second color gain factor such that they are lesser than a maximum threshold and greater than a minimum threshold, and

wherein the minimum threshold and the maximum threshold for one gain factor are based on the values of the other gain factors.

2. The system of claim 1, wherein the controller is further configured to

repeat the steps of determining a first color error and determining a first color gain factor for each of a group of colors consisting of red, green, and blue to produce a first red gain factor, a first green gain factor, and a first blue gain factor;

and

generate a first adjusted video input stream for the second camera by applying the first red gain factor, the first green gain factor, and the first blue gain factor to reduce the first color error.

3. The system of claim 1, wherein the controller is further configured to

repeat the steps of determining a second color error and determining a second color gain factor for each of a group of colors consisting of red, green, and blue to produce a second red gain factor, a second green gain factor, and a second blue gain factor;

and

generate a second adjusted video input stream for the third camera by applying the second red gain factor, the second green gain factor, and the second blue gain factor to reduce the second color error.

4. The system of claim 1, wherein the controller is further configured to change a plurality of YC_bC_r to RGB color space conversion coefficients for the first and second adjusted video streams.

5. The system of claim 1, wherein the controller is further configured to apply a brightness gain to the first camera based on an average brightness.

6. A method for correcting color in multiple-camera systems, the method comprising:

receiving, from a first camera positioned on a vehicle, a first video image including a first region of interest;

receiving, from a second camera positioned on the vehicle, a second video image including a second region of interest, the second camera being positioned such that the first region of interest at least partially overlaps with the second region of interest;

determining, by a controller, a first overlap region, the first overlap region including a portion of the first video image where the first region of interest overlaps with the second region of interest;

determining, by the controller, a first color error for the second camera, the first color error being indicative of a color difference between the first video image and the second video image within the first overlap region;

determining, by the controller, a first color gain factor for the second camera based on the first color error;

generating a first adjusted video input stream for the second camera by applying the first color gain factor to reduce the first color error;

receiving, from the second camera an adjusted second video image including an adjusted second region of interest;

receiving, from a third camera positioned on the vehicle, a third video image corresponding to a third region of interest, the third camera positioned such that the third region of interest at least partially overlaps with the adjusted second region of interest;

determining, by the controller, a second overlap region, the second overlap region including a portion of the third video image where the third region of interest overlaps with the adjusted second region of interest;

determining, by the controller, a second color error for the third camera, the second color error being indicative of a second color difference between the third video image and the adjusted second video image within the second overlap region;

determining, by the controller, a second color gain factor for the third camera based on the second color error; and

generating a second adjusted second video input stream for the third camera by applying the second color gain factor to reduce the second color error,

further comprising adjusting, by the controller, the first color gain factor and the second color gain factor such that they are lesser than a maximum threshold and greater than a minimum threshold,

wherein the minimum threshold and the maximum threshold for one gain factor are based on the values of the other gain factors.

7. The method of claim 6, further comprising:

repeating the steps of determining a first color error and determining a first color gain factor for each of a group of colors consisting of red, green, and blue to produce a first red gain factor, a first green gain factor, and a first blue gain factor; and

generating a first adjusted video input stream for the second camera by applying the first red gain factor, the first green gain factor, and the first blue gain factor to reduce the first color error.

8. The method of claim 6, further comprising:

repeating the steps of determining a second color error and determining a second color gain factor for each of a group of colors consisting of red, green, and blue to produce a second red gain factor, a second green gain factor, and a second blue gain factor; and

generating a second adjusted video input stream for the third camera by applying the second red gain factor, the second green gain factor, and the second blue gain factor to reduce the second color error.

9. The method of claim 6, further comprising changing, by the controller, a plurality of YC_bC_r to RGB color space conversion coefficients for the first and second adjusted video streams.

10. The method of claim 6, further comprising applying a brightness gain to the first camera based on an average brightness.