System and process for color-balancing a series of oblique images

• US 10,580,169 B2
• Filed: 11/14/2018
• Issued: 03/03/2020
• Est. Priority Date: 10/12/2007
• Status: Active Grant

• Current Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Original Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Inventors: Schultz, Stephen L., Giuffrida, Frank D., Gray, Robert
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US16/191,232
• Publication Number:
  US 20190102915A1
• Time in Patent Office: 475 Days
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 1/6077 Colour balance, e.g. colour cast correction
• CPC Class Codes: H04N 1/6083 controlled by factors external to the apparatus

Abstract

Image processing systems and methods are disclosed, including an image processing system comprising a computer running image processing software causing the computer to divide an oblique aerial image into a plurality of sections having different color distributions, choose a first reference aerial image, having a consistent color distribution, for the first section, color-balance at least one pixel in the first section of the oblique aerial image, such that the first color distribution of the first section matches the consistent color distribution of the first reference aerial image, by performing color-balancing transformations for color bands for each of the at least one pixel in the first section.

Claims

1. An image processing system for color-balancing oblique images, comprising:

a computer running image processing software causing the computer to;

divide an oblique aerial image into a plurality of sections, wherein a first section of the plurality of sections has a first color distribution and a second section of the plurality of sections has a second color distribution, the first color distribution differing from the second color distribution, and wherein each of the first section and the second section has pixels, each pixel having a Red color band, a Green color band, and a Blue color band;

choose a first reference aerial image, having a consistent color distribution, for the first section by automatically matching at least a portion of geographic information of the first reference aerial image with at least a portion of geographic information of the first section;

color-balance at least one pixel in the first section of the oblique aerial image, such that the first color distribution of the first section matches the consistent color distribution of the first reference aerial image, by performing the following for each of the at least one pixel in the first section;

create a first color-balancing transformation for the Red color band;

create a second color-balancing transformation for the Green color band;

create a third color-balancing transformation for the Blue color band; and

combine the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation into a single pixel color value.

2. The system of claim 1, wherein the image processing software further causes the computer to:

choose a second reference aerial image, having a consistent color distribution, for the second section by automatically matching at least a portion of the geographic information of the second reference aerial image with at least a portion of the geographic information of the second section; and

color-balance at least one pixel in the second section of the oblique aerial image, such that the second color distribution of the second section matches the consistent color distribution of the second reference aerial image, by performing the following for each of the at least one pixel in the second section;

create a first color-balancing transformation for the Red color band;

create a second color-balancing transformation for the Green color band;

create a third color-balancing transformation for the Blue color band; and

combine the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation into a single pixel color value.

3. The system of claim 2, wherein the image processing software further causes the computer to store the color-balancing transformations for the first section and the second section.

4. The system of claim 2, wherein at least one of the first reference aerial image and the second reference aerial image is a nadir aerial image.

5. The system of claim 2, wherein at least one of the first reference aerial image and the second reference aerial image is another oblique aerial image.

6. The system of claim 2, wherein creating the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation for each of the at least one pixel in the first section and the second section, further comprises creating one or more histograms.

7. The system of claim 6, wherein creating one or more histograms comprises:

creating a first histogram for the color distribution of the first reference aerial image;

creating a second histogram for the color distribution of the first section of the aerial oblique image; and

creating a third histogram for one or more of the first, second, and third color-balancing transformations.

8. The system of claim 1, wherein the first color distribution of the first section of the aerial oblique image is caused by first specular reflections and the second color distribution of the second section of the aerial oblique image is caused by second specular reflections.

9. The system of claim 1, wherein a difference between the first color distribution of the first section of the aerial oblique image and the second color distribution of the second section of the aerial oblique image based on differing path length distances between a first location of a first point in the first section and a camera when the oblique aerial image was captured and a second location of a second point in the second section and the camera when the oblique aerial image was captured.

10. The system of claim 1, wherein combining the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation into a single pixel color value uses one or more of bi-linear interpolation, linear interpolation, cubics, and splines.

11. The method of claim 10, wherein creating the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation for each of the at least one pixel in the first section and the second section, further comprises creating one or more histograms.

12. The method of claim 11, wherein creating one or more histograms comprises:

creating a first histogram for the color distribution of the first reference aerial image;

creating a second histogram for the color distribution of the first section of the aerial oblique image; and

creating a third histogram for one or more of the first, second, and third color-balancing transformations.

13. An image processing method for color-balancing oblique images, comprising:

dividing, with a computer running image processing software, an oblique aerial image into a plurality of sections, wherein a first section of the plurality of sections has a first color distribution and a second section of the plurality of sections has a second color distribution, the first color distribution differing from the second color distribution, and wherein each of the first section and the second section has pixels, each pixel having a Red color band, a Green color band, and a Blue color band;

choosing, with the computer, a first reference aerial image, having a consistent color distribution, for the first section by automatically matching at least a portion of geographic information of the first reference aerial image with at least a portion of geographic information of the first section;

color-balancing, with the computer, at least one pixel in the first section of the oblique aerial image, such that the first color distribution of the first section matches the consistent color distribution of the first reference aerial image, by performing the following for each of the at least one pixel in the first section;

creating a first color-balancing transformation for the Red color band;

creating a second color-balancing transformation for the Green color band;

creating a third color-balancing transformation for the Blue color band; and

combining the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation into a single pixel color value.

14. The method of claim 13, further comprising:

choosing, with the computer, a second reference aerial image, having a consistent color distribution, for the second section by automatically matching at least a portion of the geographic information of the second reference aerial image with at least a portion of the geographic information of the second section; and

color-balancing, with the computer, at least one pixel in the second section of the oblique aerial image, such that the second color distribution of the second section matches the consistent color distribution of the second reference aerial image, by performing the following for each of the at least one pixel in the second section;

creating a first color-balancing transformation for the Red color band;

creating a second color-balancing transformation for the Green color band;

creating a third color-balancing transformation for the Blue color band; and

combining the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation into a single pixel color value.

15. The method of claim 14, further comprising storing the color-balancing transformations for the first section and the second section in a non-transitory computer medium.

16. The method of claim 14, wherein at least one of the first reference aerial image and the second reference aerial image is a nadir aerial image.

17. The method of claim 14, wherein at least one of the first reference aerial image and the second reference aerial image is another oblique aerial image.

18. The method of claim 13, wherein the first color distribution of the first section of the aerial oblique image is caused by first specular reflections and the second color distribution of the second section of the aerial oblique image is caused by second specular reflections.

19. The method of claim 13, wherein a difference between the first color distribution of the first section of the aerial oblique image and the second color distribution of the second section of the aerial oblique image based on differing path length distances between a first location of a first point in the first section and a camera when the oblique aerial image was captured and a second location of a second point in the second section and the camera when the oblique aerial image was captured.

20. The method of claim 13, wherein combining the first color-balancing transformation, the second color-balancing transformation, and the third color-balancing transformation into a single pixel color value uses one or more of bi-linear interpolation, linear interpolation, cubics, and splines.