Method and system for performing white balancing operations on captured images

• US 9,756,222 B2
• Filed: 06/26/2013
• Issued: 09/05/2017
• Est. Priority Date: 06/26/2013
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Cabral, Brian, Motta, Ricardo, Harwell, Mitchell
• Primary Examiner(s): Peterson, Christopher K
• Application Number:
  US13/927,565
• Publication Number:
  US 20150002693A1
• Time in Patent Office: 1,532 Days
• Field of Search: 3482231-2241
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

Embodiments of the present invention are operable to perform automatic white balancing operations on images captured by a camera system through the use of weights derived through crowdsourcing procedures. Embodiments of the present invention use crowdsourced weight data resident on the camera system in combination with sampled image data of a captured image to determine a likely illuminant source. When performing automatic white balancing operations on the captured image, embodiments of the present invention may also compute a confidence score which may present the user with a choice to either use the likely illuminant determined using the crowdsourced weights or the camera system'"'"'s default or normal automatic white balancing correction algorithm.

Claims

1. A method of performing white balancing for a captured image, said method comprising:

sampling image data from said captured image using a camera system producing sampled image data;

determining a likely illuminant for said captured image using a function of a set of weights resident on said camera system and said sampled image data, wherein said set of weights is derived using crowdsourcing procedures, wherein said determining said likely illuminant comprises calculating and using a summation of each weight of said set of weights combined with image data comprising coordinates and associated color data related to said sampled image data; and

computing a white balance correction for said captured image using said likely illuminant.

2. The method of performing white balancing as described in claim 1, wherein said determining further comprises computing a confidence score for said likely illuminant and using a default white balancing correction for said captured image if said confidence score is below a predetermined threshold.

3. The method of performing white balancing as described in claim 2, wherein said computing a confidence score further comprises prompting a user to select either said default illuminant white balancing correction or said likely illuminant after said confidence score is calculated.

4. The method of performing white balancing as described in claim 1, wherein said determining further comprises receiving an updated set of weights from a remote server system over a communication network.

5. The method of performing white balancing as described in claim 4, wherein said receiving further comprises transmitting a plurality of newly captured images gathered by said camera system to said remote server system over said communication network for generating said updated set of weights.

6. The method of performing white balancing as described in claim 4, wherein said receiving further comprises transmitting a plurality of user adjusted images captured by said camera system to said remote server system over said communication network for generating said updated set of weights.

7. The method of performing white balancing as described in claim 1, wherein said set of weights is derived from subjective user input gathered from a plurality of users concerning a plurality of different illuminants applied to a plurality of classified images.

8. The method of performing white balancing as described in claim 7, wherein said subjective user input comprises user selections of a most aesthetically pleasing trial image of a plurality of trial images each of different illumination sources.

9. The method of selecting a likely illuminant as described in claim 1, wherein said summation comprises a grouping of pixel coordinates and corresponding color values associated therewith and wherein further said determining a likely illuminant for said captured image further comprises comparing said grouping against a plurality of groupings of coordinates and associated color values stored in a data structure, wherein respective groupings of said data structure correspond to respective illuminant sources.

10. A system for selecting a likely illuminant for white balancing a captured image, said system comprising:

a sampling module of a camera system, wherein said sampling module is operable to sample image data from said captured image producing sampled image data;

a determination module operable to determine a likely illuminant using a function of a set of weights resident on said camera system and said sampled image data, wherein said set of weights is derived using crowdsourcing procedures, wherein said likely illuminant is determined based on a summation of each weight of said set of weights combined with image data comprising coordinates and associated color data related to said sampled image data;

a confidence calculation module operable to compute a confidence score for said likely illuminant, wherein said camera system is operable to use a default white balancing procedure if said confidence score is below a predetermined threshold;

a white balance computation module operable to compute a white balance correction data for said captured image using said likely illuminant provided said confidence score is above said threshold; and

a communication module operable to communicate data between a remote server and said camera system.

11. The system for selecting a likely illuminant as described in claim 10, wherein said confidence calculation module is further operable to prompt a user to select either said default white balancing procedure or said likely illuminant after said confidence score is calculated.

12. The system for selecting a likely illuminant as described in claim 10, wherein said determination module is further operable to receive an updated set of weights from said remote server system over a communication network.

13. The system for selecting a likely illuminant as described in claim 12, wherein said determination module is operable to transmit a plurality of captured images gathered by said camera system to said remote server system over said communication network for generating said updated set of weights.

14. The system for selecting a likely illuminant as described in claim 12, wherein said determination module is operable to transmit a plurality of user adjusted images captured by said camera system to said remote server system over said communication network for generating said updated set of weights.

15. The system for selecting a likely illuminant as described in claim 12, wherein said set of weights is derived from subjective user input gathered from a plurality of users concerning a plurality of different illuminants applied to a plurality of classified images.

16. The system for selecting a likely illuminant as described in claim 15, wherein said subjective user input comprises user selections of a most aesthetically pleasing trial image of a plurality of trial images each of different illumination sources.

17. The system for selecting a likely illuminant as described in claim 10, wherein said summation comprises a first group of pixel coordinates and corresponding color values associated therewith and wherein further said determination module is configured to determine a likely illuminant for said captured image by comparing said first group against a second group of pixel coordinates and corresponding color values associated therewith stored in a data structure, wherein said data structure stores a plurality of different groupings of pixel coordinates and corresponding color values associated with different illuminant sources.

18. A method of selecting a likely illuminant for white balancing a captured image, said method comprising:

generating sampled image data values from a captured image;

determining a likely illuminant of said captured image using a function of a set of weights resident on said camera system and said sampled image data values, wherein said set of weights is derived using crowdsourcing algorithms, wherein said determining said likely illuminant comprises calculating and using a summation of each weight of said set of weights combined with image data comprising coordinates and associated color data related to said sampled image data;

computing a confidence score for said likely illuminant, and computing a white balance correction for said captured image using said likely illuminant provided said confidence score is above a predetermined threshold.

19. The method of selecting a likely illuminant as described in claim 18, wherein said determining further comprises receiving an updated set of weights from a remote server system over a communication network.

20. The method of selecting a likely illuminant as described in claim 19, wherein said receiving further comprises transmitting a plurality of user adjusted images captured by said camera system to said remote server system over said communication network for generating said updated set of weights.

21. The method of selecting a likely illuminant as described in claim 19, wherein said set of weights is derived from subjective user input gathered from a plurality of users concerning a plurality of different illuminants applied to a plurality of classified images.

22. The method of selecting a likely illuminant as described in claim 21, wherein said subjective user input comprises user selections of a most aesthetically pleasing trial image of a plurality of trial images each of different illumination sources.

23. The method of selecting a likely illuminant as described in claim 18 further comprising using a default white balancing procedure to correct said captured image provided said confidence score is below said predetermined threshold.

24. The method of selecting a likely illuminant as described in claim 18, wherein said summation comprises a first group of pixel coordinates and corresponding color values associated therewith and wherein further said determination module is configured to determine a likely illuminant for said captured image by comparing said first group against a second group of pixel coordinates and corresponding color values associated therewith stored in a data structure, wherein said data structure stores a plurality of different groupings of pixel coordinates and corresponding color values associated with different illuminant sources.