Image processing method and device for accomplishing white balance regulation, computer-readable storage medium and computer device

• US 10,798,358 B2
• Filed: 06/05/2019
• Issued: 10/06/2020
• Est. Priority Date: 12/25/2017
• Status: Active Grant

• Current Assignee: Guangdong OPPO Mobile Telecommunications Corporation Limited
• Original Assignee: Guangdong OPPO Mobile Telecommunications Corporation Limited
• Inventors: Wang, Huichao
• Primary Examiner(s): Segura, Cynthia
• Application Number:
  US16/432,677
• Publication Number:
  US 20190289270A1
• Time in Patent Office: 489 Days
• Field of Search: None
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

An image processing method is provided. The image processing method includes: each frame of image in multiple continuous frames of images is processed to determine a number of light sources of each frame of image; it is determined whether a difference between a number of light sources of a kth frame of image and a number of light sources of a (k+1)th frame of image is equal to 0; and when the difference is unequal to 0, a color temperature of the (k+1)th frame of image is determined to be a color temperature of the kth frame of image and the (k+1)th frame of image is processed according to the color temperature of the (k+1)th frame of image. An image processing device, a computer-readable storage medium and a computer device are further provided.

Claims

1. An image processing method, comprising:

processing each frame of image in multiple continuous frames of images to determine a number of light sources of each frame of image;

determining whether a difference between a number of light sources of a kth frame of image and a number of light sources of a (k+1)th frame of image is equal to 0, wherein k is a positive integer; and

responsive to determining that the difference is unequal to 0, determining a color temperature of the (k+1)th frame of image to be a color temperature of the kth frame of image, and processing the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image;

wherein the operation of processing each frame of image in the multiple continuous frames of images to determine the number of the light sources of each frame of image further comprises;

determining a high-luminance region and a medium-luminance region according to a luminance distribution extending outwards from a center of a light source along a radial direction of the light source; and

determining a color of the light source by subtracting pixel averages of primary color channels of the medium-luminance region from pixel averages of primary color channels of the high-luminance region;

the method further comprises;

responsive to determining that the difference is equal to 0, performing white balance processing on an mth frame of image according to a color temperature of the mth frame of image, the mth frame of image comprising the kth frame of image or the (k+1)th frame of image, wherein m is be a positive integer and m≥

k.

2. The image processing method of claim 1, wherein the operation of determining, responsive to determining that the difference value is unequal to 0, the color temperature of the (k+1)th frame of image to be the color temperature of the kth frame of image and processing the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image comprises:

determining whether the number of the light sources of the kth frame of image is more than or equal to 1;

responsive to determining that the number of the light sources of the kth frame of image is less than 1, adopting a gray world method to perform white balance processing on the kth frame of image and the (k+1)th frame of image;

responsive to determining that the number of the light sources of the kth frame of image is equal to 1, determining the color temperature and number of the light sources of the (k+1)th frame of image according to the color temperature and number of the light sources of the kth frame of image and performing white balance processing on the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image; and

responsive to determining that the number of the light sources of the kth frame of image is more than 1, determining a primary light source according to at least one of scenario parameters, areas or luminance parameters of the light sources of the kth frame of image, determining the color temperature of the (k+1)th frame of image according to a color temperature of the primary light source, performing white balance processing on the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image, and determining the number of the light sources of the (k+1)th frame of image to be the number of the light sources of the kth frame of image, wherein the scenario parameters comprise image shooting time and signal strength of a Global Positioning System (GPS), and the luminance parameters comprise luminance of multiple light sources.

3. The image processing method of claim 1, wherein the operation of performing, responsive to determining that the difference value is equal to 0, white balance processing on the mth frame of image according to the color temperature of the mth frame of image, the mth frame of image comprising the kth frame of image or the (k+1)th frame of image, comprises:

determining whether the number of the light sources is more than or equal to 1;

responsive to determining that the number of the light sources is less than 1, adopting a gray world method to perform white balance processing on the mth frame of image;

responsive to determining that the number of the light sources is equal to 1, performing white balance processing on the mth frame of image according to a color temperature of the light source; and

responsive to determining that the number of the light sources is more than 1, determining a primary light source according to at least one of scenario parameters, areas or luminance parameters of the light sources and performing white balance processing on the mth frame of image according to a color temperature of the primary light source, wherein the scenario parameters comprise image shooting time and signal strength of a GPS, and the luminance parameters comprise luminance of multiple light sources.

4. An image processing device, comprising:

a processor;

a plurality of program modules, when, executed by the processor, cause the device to perform predefined functions, the plurality of program modules further comprising;

a first processing module, configured to process each frame of image in multiple continuous frames of images to determine a number of light sources of each frame of image;

a judgment module, configured to determine whether a difference between a number of light sources of a kth frame of image and a number of light sources of a (k+1)th frame of image is equal to 0, wherein k is a positive integer; and

a second processing module, configured to, responsive to determining that the difference is unequal to 0, determine a color temperature of the (k+1)th frame of image to be a color temperature of the kth frame of image and process the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image.

5. The image processing device of claim 4, wherein the first processing module comprises a plurality of program units, the plurality of program units comprising:

a division unit, configured to divide a preview image into multiple regions;

a first judgment unit, configured to, for each of the multiple regions, determine whether the region is a target region comprising a light source according to a histogram of the region;

a second judgment unit, configured to, responsive to determining that the region is the target region comprising light sources, determine whether there are multiple target regions adjacent to the region;

a splicing unit, configured to, responsive to determining that there are multiple target regions adjacent to the region, splice the multiple target regions into a light source;

a first determination unit, configured to, responsive to determining that there are no adjacent target regions adjacent to the region, determine the target regions as a light source; and

a counting unit, configured to count the light sources.

6. The image processing device of claim 4, wherein the first processing module comprises a plurality of program units, the plurality of program units comprising:

a second determination unit, configured to determine a high-luminance region and a medium-luminance region according to a luminance distribution extending outwards from a center of a light source along a radial direction of the light source; and

a third determination unit, configured to determine a color of the light source by subtracting pixel averages of primary color channels of the medium-luminance region from pixel averages of primary color channels of the high-luminance region.

7. The image processing device of claim 6, wherein the second processing module comprises a plurality of program units, the plurality of program units comprising:

a third judgment unit, configured to determine whether the number of the light sources of the kth frame of image is more than or equal to 1;

a first processing unit, configured to, responsive to determining that the number of the light sources of the kth frame of image is less than 1, adopt a gray world method to perform white balance processing on the kth frame of image and the (k+1)th frame of image;

a second processing unit, configured to, responsive to determining that the number of the light sources of the kth frame of image is equal to 1, determine the color temperature and number of the light sources of the (k+1)th frame of image according to the color temperature and number of the light sources of the kth frame of image, and perform white balance processing on the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image; and

a third processing unit, configured to, responsive to determining that the number of the light sources of the kth frame of image is more than 1, determine a primary light source according to at least one of scenario parameters, areas or luminance parameters of the light sources of the kth frame of image, determine the color temperature of the (k+1)th frame of image according to a color temperature of the primary light source, perform white balance processing on the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image, and determine the number of the light sources of the (k+1)th frame of image to be the number of the light sources of the kth frame of image, wherein the scenario parameters comprise image shooting time and signal strength of a Global Positioning System (GPS), and the luminance parameters comprise luminance of multiple light sources.

8. The image processing device of claim 6, wherein the plurality of program modules further comprises:

a third processing module, configured to, responsive to determining that the difference is equal to 0, perform white balance processing on an mth frame of image according to a color temperature of the mth frame of image, wherein the mth frame of image comprises the kth frame of image or the (k+1)th frame of image, wherein m is be a positive integer and m≥

k.

9. The image processing device of claim 8, wherein the third processing module comprises a plurality of program units, the plurality of program units comprising:

a fourth judgment unit, configured to determine whether the number of the light sources is more than or equal to 1 or not;

a fourth processing unit, configured to, responsive to determining that the number of the light sources is less than 1, adopt a gray world method to perform white balance processing on the mth frame image;

a fifth processing unit, configured to, responsive to determining that the number of the light sources is equal to 1, perform white balance processing on the mth frame image according to a color temperature of the light source; and

a sixth processing unit, configured to, responsive to determining that the number of the light sources is more than 1, determine a primary light source according to at least one of scenario parameters, areas or luminance parameters of the light sources and perform white balance processing on the mth frame image according to a color temperature of the primary light source, wherein the scenario parameters comprise image shooting time and signal strength of a GPS, and the luminance parameters comprise luminance of multiple light sources.

10. Anon-transitory computer-readable storage medium, comprising one or more computer-executable instructions that, when executed by one or more processors, cause the one or more processors to execute an image processing method, the method comprising:

processing each frame of image in multiple continuous frames of images to determine a number of light sources of each frame of image;

determining whether a difference between a number of light sources of a kth frame of image and a number of light sources of a (k+1)th frame of image is equal to 0, wherein k is a positive integer; and

responsive to determining that the difference is unequal to 0, determining a color temperature of the (k+1)th frame of image to be a color temperature of the kth frame of image, and processing the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image;

wherein the operation of processing each frame of image in the multiple continuous frames of images to determine the number of the light sources of each frame of image further comprises;

determining a high-luminance region and a medium-luminance region according to a luminance distribution extending outwards from a center of a light source along a radial direction of the light source; and

determining a color of the light source by subtracting pixel averages of primary color channels of the medium-luminance region from pixel averages of primary color channels of the high-luminance region;

the method further comprises;

responsive to determining that the difference is equal to 0, performing white balance processing on an mth frame of image according to a color temperature of the mth frame of image, the mth frame of image comprising the kth frame of image or the (k+1)th frame of image, wherein m is be a positive integer and m≥

k.

11. A computer device, comprising:

a memory and a processor, the memory storing a computer-executable instruction that, when executed by the processor, causes the processor to execute an image processing method, the method comprising;

processing each frame of image in multiple continuous frames of images to determine a number of light sources of each frame of image;

determining whether a difference between a number of light sources of a kth frame of image and a number of light sources of a (k+1)th frame of image is equal to 0, wherein k is a positive integer; and

responsive to determining that the difference is unequal to 0, determining a color temperature of the (k+1)th frame of image to be a color temperature of the kth frame of image, and processing the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image;

wherein the operation of processing each frame of image in the multiple continuous frames of images to determine the number of the light sources of each frame of image further comprises;

determining a high-luminance region and a medium-luminance region according to a luminance distribution extending outwards from a center of a light source along a radial direction of the light source; and

determining a color of the light source by subtracting pixel averages of primary color channels of the medium-luminance region from pixel averages of primary color channels of the high-luminance region;

the method further comprises;

responsive to determining that the difference is equal to 0, performing white balance processing on an mth frame of image according to a color temperature of the mth frame of image, the mth frame of image comprising the kth frame of image or the (k+1)th frame of image, wherein m is be a positive integer and m≥

k.

12. The computer device of claim 11, wherein the operation of determining, responsive to determining that the difference value is unequal to 0, the color temperature of the (k+1)th frame of image to be the color temperature of the kth frame of image and processing the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image comprises:

determining whether the number of the light sources of the kth frame of image is more than or equal to 1;

responsive to determining that the number of the light sources of the kth frame of image is less than 1, adopting a gray world method to perform white balance processing on the kth frame of image and the (k+1)th frame of image;

responsive to determining that the number of the light sources of the kth frame of image is equal to 1, determining the color temperature and number of the light sources of the (k+1)th frame of image according to the color temperature and number of the light sources of the kth frame of image and performing white balance processing on the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image; and

responsive to determining that the number of the light sources of the kth frame of image is more than 1, determining a primary light source according to at least one of scenario parameters, areas or luminance parameters of the light sources of the kth frame of image, determining the color temperature of the (k+1)th frame of image according to a color temperature of the primary light source, performing white balance processing on the (k+1)th frame of image according to the color temperature of the (k+1)th frame of image, and determining the number of the light sources of the (k+1)th frame of image to be the number of the light sources of the kth frame of image, wherein the scenario parameters comprise image shooting time and signal strength of a Global Positioning System (GPS), and the luminance parameters comprise luminance of multiple light sources.

13. The computer device of claim 11, wherein the operation of performing, responsive to determining that the difference value is equal to 0, white balance processing on the mth frame of image according to the color temperature of the mth frame of image, the mth frame of image comprising the kth frame of image or the (k+1)th frame of image, comprises:

determining whether the number of the light sources is more than or equal to 1;

responsive to determining that the number of the light sources is less than 1, adopting a gray world method to perform white balance processing on the mth frame of image;

responsive to determining that the number of the light sources is equal to 1, performing white balance processing on the mth frame of image according to a color temperature of the light source; and

responsive to determining that the number of the light sources is more than 1, determining a primary light source according to at least one of scenario parameters, areas or luminance parameters of the light sources and performing white balance processing on the mth frame of image according to a color temperature of the primary light source, wherein the scenario parameters comprise image shooting time and signal strength of a GPS, and the luminance parameters comprise luminance of multiple light sources.