Image sensing apparatus and color-correction matrix correcting method and look-up table establishing method

• US 9,219,897 B2
• Filed: 05/13/2014
• Issued: 12/22/2015
• Est. Priority Date: 03/19/2014
• Status: Active Grant

• Current Assignee: Novatek Microelectronics Corporation (Novatek Microelectronics Corp Limited)
• Original Assignee: Faraday Technology Corp.
• Inventors: Chuang, Hung-Chang
• Primary Examiner(s): Do, Anh
• Application Number:
  US14/276,974
• Publication Number:
  US 20150271459A1
• Time in Patent Office: 588 Days
• Field of Search: 382/167, 382/162, 382/163, 382/168, 382/103, 382/274, 382/280, 382/300, 348/E5.031, 348/E9.01, 348/222.1, 348/231.3, 348/208.14, 348/241, 348/242, 348/273, 358/525, 375/260, 375/267, 455/500, 702/86, 702/104
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/98 Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: H04N 1/58 Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction H04N1/62 takes precedence
• CPC Class Codes: H04N 1/60 Colour correction or control H04N1/54 takes precedence
• CPC Class Codes: H04N 1/6041 for controlling uniformity of color across image area
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85

Abstract

An image sensing apparatus, a color-correction matrix correcting method and a look-up table establishing method are provided. The image sensing apparatus calculates a block statistics value corresponding to a block of pixels in an image sensor array. Based on a look-up table, the image sensing apparatus determines a covariance value corresponding to a current gain value. According to the covariance value and the block statistics value, the image sensing apparatus corrects a color-correction matrix corresponding to the block of pixels. The image sensing apparatus can use an amended color-correction matrix to correct the color of the pixel, so as to reduce chroma noise or other noise.

Claims

1. A color-correction matrix correcting method adapted to an image sensing apparatus, the method comprising:

calculating, by a color-correction circuit, a block statistics value corresponding to a block of pixels in an image sensor array;

determining, by the color-correction circuit, a covariance value corresponding to a current gain value based on a look-up table; and

correcting, by the color-correction circuit, a color-correction matrix corresponding to the block of pixels according to the covariance value and the block statistics value.

2. The color-correction matrix correcting method according to claim 1, wherein the step of calculating the block statistics value comprises:

calculating a correlation matrix of the block of pixels and serving the correlation matrix as the block statistics value.

3. The color-correction matrix correcting method according to claim 2, wherein the step of calculating the correlation matrix comprises:

calculating the correlation matrix by

4. The color-correction matrix correcting method according to claim 1, wherein the step of determining the covariance value comprises:

finding at least one covariance matrix from the look-up table according to the current gain value; and

obtaining the covariance value according to the at least one covariance matrix.

5. The color-correction matrix correcting method according to claim 4, wherein the step of obtaining the covariance value comprises:

serving the at least one covariance matrix as the covariance value;

or

performing an interpolation operation by using the at least one covariance matrix to obtain the covariance value.

6. The color-correction matrix correcting method according to claim 4, wherein the step of obtaining the covariance value further comprises:

serving 0 as the covariance value when the current gain value is less than a threshold.

7. The color-correction matrix correcting method according to claim 1, wherein the look-up table comprises a plurality of covariance matrices, and the covariance matrices respectively correspond to different gain values.

8. The color-correction matrix correcting method according to claim 1, further comprising:

obtaining, by the color-correction circuit, a plurality of image frames by the image sensor array according to different sampling gain values during a correction period;

selecting, by the color-correction circuit, at least one sampling block from each of the image frames;

calculating, by the color-correction circuit, statistics of the at least one sampling block for each of the image frames; and

establishing, by the color-correction circuit, the look-up table according to the statistics corresponding to the different sampling gain values.

9. The color-correction matrix correcting method according to claim 1, further comprising:

obtaining, by the color-correction circuit, an image frame by the image sensor array according to a sampling gain value during a correction period;

selecting, by the color-correction circuit, at least one sampling block from the image frame;

calculating, by the color-correction circuit, statistics of the at least one sampling block for the image frame; and

establishing, by the color-correction circuit, the look-up table according to the statistics corresponding to the sampling gain value.

10. The color-correction matrix correcting method according to claim 9, wherein the step of selecting the at least one sampling block comprises:

selecting at least one block from a planar region of the image frame as the at least one sampling block;

or

selecting at least one block from a planar region having specific chroma of the image frame as the at least one sampling block.

11. The color-correction matrix correcting method according to claim 9, wherein the step of calculating the statistics comprises:

calculating

12. The color-correction matrix correcting method according to claim 11, wherein the parameter P_r is a mean or a middle value of red-color sensing values of different pixels in the j^th sampling block, wherein the parameter P_g is a mean or a middle value of green-color sensing values of different pixels in the j^th sampling block, and wherein the parameter P_b is a mean or a middle value of blue-color sensing values of different pixels in the j^th sampling block.

13. The color-correction matrix correcting method according to claim 9, wherein the step of establishing the look-up table comprises:

calculating a mean value or a weighted mean value of the statistics of the at least one sampling block; and

recording the mean value or the weighted mean value in the look-up table to serve the mean value or the weighted mean value as the covariance value corresponding to the sampling gain value.

14. The color-correction matrix correcting method according to claim 1, wherein the step of correcting the color-correction matrix comprises:

calculating C_new=C_original*(Cor−

CorNN<sup>−

1</sup>)^T*(Cor<sup>−

1</sup>)^T,

wherein C_new represents a corrected color-correction matrix corresponding to the block of pixels, C_origanal represents the color-correction matrix corresponding to the block of pixels, Cor represents the block statistics value corresponding to the block of pixels, CorNN represents the covariance value, Cor<sup>−

1</sup> represents an inverse matrix of Cor, and (Cor<sup>−

1</sup>)^T represents a transpose matrix of Cor<sup>−

1</sup>.

15. The color-correction matrix correcting method according to claim 14, wherein the color-correction matrix C_original is dependent on a light-source characteristic.

16. An image sensing apparatus, comprising:

an image sensing circuit, gaining an original image frame by using a current gain value and outputting a gained image frame;

an automatic white balance circuit, having an input terminal coupled to an output terminal of the image sensing circuit to receive the gained image frame, and outputting an adjusted image frame; and

a color-correction circuit, having an input terminal coupled to an output terminal of the automatic white balance circuit to receive the adjusted image frame, wherein the color-correction circuit calculates a block statistics value corresponding to a block of pixels in an image sensor array of the image sensing circuit according to the adjusted image frame, determines a covariance value corresponding to the current gain value based on a look-up table and corrects a color-correction matrix corresponding to the block of pixels according to the covariance value and the block statistics value.

17. The image sensing apparatus according to claim 16, wherein the color-correction circuit calculates a correlation matrix of the block of pixels and serving the correlation matrix as the block statistics value.

18. The image sensing apparatus according to claim 17, wherein the color-correction circuit calculates the correlation matrix by

19. The image sensing apparatus according to claim 16, wherein the color-correction circuit finds at least one covariance matrix from the look-up table according to the current gain value and obtains the covariance value according to the at least one covariance matrix.

20. The image sensing apparatus according to claim 19, wherein the color-correction circuit serves the at least one covariance matrix as the covariance value or performs interpolation by using the at least one covariance matrix to obtain the covariance value.

21. The image sensing apparatus according to claim 19, wherein the color-correction circuit serves 0 as the covariance value when the current gain value is less than a threshold.

22. The image sensing apparatus according to claim 16, wherein the look-up table comprises a plurality of covariance matrices, and the covariance matrices respectively correspond to different gain values.

23. The image sensing apparatus according to claim 16, wherein the color-correction circuit obtains a plurality of image frames by the image sensor array according to different sampling gain values during a correction period, selects at least one sampling block from each of the image frames, calculates statistics of the at least one sampling block for each of the image frames and establishes the look-up table according to the statistics corresponding to the different sampling gain values.

24. The image sensing apparatus according to claim 16, wherein the color-correction circuit obtains an image frame by the image sensor array according to a sampling gain value during a correction period, selects at least one sampling block from the image frame, calculates statistics of the at least one sampling block for the image frame and establishes the look-up table according to the statistics corresponding to the sampling gain value.

25. The image sensing apparatus according to claim 24, wherein the color-correction circuit selects at least one block from a planar region of the image frame as the at least one sampling block or selects at least one block from the planar region having specific chroma of the image frame as the at least one sampling block.

26. The image sensing apparatus according to claim 24, wherein the color-correction circuit calculates

27. The image sensing apparatus according to claim 26, wherein the parameter P_r is a mean or a middle value of red-color sensing values of different pixels in the j^th sampling block, the parameter P_g is a mean or a middle value of green-color sensing values of different pixels in the j^th sampling block, and the parameter P_b is a mean or a middle value of blue-color sensing values of different pixels in the j^th sampling block.

28. The image sensing apparatus according to claim 24, wherein the color-correction circuit calculates a mean value or a weighted mean value of the statistics of the at least one sampling block and records the mean value or the weighted mean value in the look-up table to serve the mean value or the weighted mean value as the covariance value corresponding to the sampling gain value.

29. The image sensing apparatus according to claim 16, wherein the color-correction circuit calculates C_new=C_originai*(Cor−

CorNN<sup>−

1</sup>)^T*(Cor<sup>−

1</sup>)^T,

wherein C_new represents a corrected color-correction matrix corresponding to the block of pixels, C_original represents the color-correction matrix corresponding to the block of pixels, Cor represents the block statistics value corresponding to the block of pixels, CorNN represents the covariance value, Cor<sup>−

1</sup> represents an inverse matrix of Cor, and (Cor<sup>−

1</sup>)^T represents a transpose matrix of Cor<sup>−

1</sup>.

30. The image sensing apparatus according to claim 29, wherein the color-correction matrix C_original is dependent on a light-source characteristic.

31. A look-up table establishing method adapted to an image sensing apparatus, the method comprising:

obtaining, by a color-correction circuit, an image frame by an image sensor array according to at least one sampling gain value during a correction period;

selecting, by the color-correction circuit, at least one sampling block from the image frame;

calculating, by the color-correction circuit, statistics of the at least one sampling block for the image frame; and

establishing, by the color-correction circuit, the look-up table according to the statistics corresponding to the sampling gain value.

32. The look-up table establishing method according to claim 31, wherein the look-up table comprises a plurality of covariance matrices respectively corresponding to different gain values.

33. The look-up table establishing method according to claim 31, wherein the step of selecting the at least one sampling block comprises:

selecting at least one block from a planar region of the image frame as the at least one sampling block;

or

selecting at least one block from a planar region having specific chroma of the image frame as the at least one sampling block.

34. The look-up table establishing method according to claim 31, wherein the step of calculating the statistics comprises:

calculating

35. The look-up table establishing method according to claim 34, wherein the parameter P_r is a mean or a middle value of red-color sensing values of different pixels in the j^th sampling block, the parameter P_g is a mean or a middle value of green-color sensing values of different pixels in the j^th sampling block, and the parameter P_b is a mean or a middle value of blue-color sensing values of different pixels in the j^th sampling block.

36. The look-up table establishing method according to claim 31, wherein the step of establishing the look-up table comprises:

calculating a mean value or a weighted mean value of the statistics of the at least one sampling block; and

recording the mean value or the weighted mean value in the look-up table to serve the mean value or the weighted mean value as the covariance value corresponding to the sampling gain values.