IMAGE NOISE CALIBRATION METHOD AND APPARATUS, IMAGE NOISE REDUCTION METHOD AND APPARATUS, AND IMAGE PROCESSING APPARATUS

• US 20200137334A1
• Filed: 12/23/2019
• Published: 04/30/2020
• Est. Priority Date: 11/22/2017
• Status: Abandoned Application

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: HU, Tao, LIU, Huaiyu, CAO, Zisheng
• Application Number:
  US16/725,711
• Publication Number:
  US 20200137334A1
• Time in Patent Office: Days
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response
• CPC Class Codes: H04N 25/672 between adjacent sensors or output registers for reading a single image

Abstract

An image noise calibration apparatus includes a processor. The processor acquires raw image data output by an image sensor. The image sensor includes an image sensitive unit array, and the raw image data is output by the image sensitive unit array in an optical black state. The processor further determines fixed pattern noise calibration data of the image sensor based on the raw image data output by the image sensitive unit array. The fixed pattern noise calibration data is used for noise reduction of the image sensitive unit array, and the number of the fixed pattern noise calibration data is less than the number of image sensitive units in the image sensor.

Claims

1. An image noise calibration apparatus, comprising a processor, wherein the processor executing a computer readable instruction set implements:

acquiring raw image data output by an image sensor, wherein the image sensor includes an image sensitive unit array, and the raw image data is output by the image sensitive unit array in an optical black state; and

determining fixed pattern noise (FPN) calibration data of the image sensor based on the raw image data output by the image sensitive unit array, wherein the FPN calibration data is used for noise reduction of the image sensitive unit array, and the number of the FPN calibration data is less than the number of image sensitive units in the image sensor.

2. The image noise calibration apparatus of claim 1, wherein determining the FPN calibration data of the image sensor based on the raw image data output by the image sensing unit array further includes:

for one column of image sensitive units in the image sensitive unit array, determining FPN calibration data of the column of image sensitive units based on raw image data output by the column of image sensitive units, wherein the FPN calibration data of the column of image sensitive units is used for noise reduction of the column of image sensitive units, and the number of FPN calibration data of the column of image sensitive units is less than the number of image sensitive units in the column of image sensitive units.

3. The image noise calibration apparatus of claim 2, wherein the image sensitive units in the column are configured to output image data of a class of M-channel, the FPN calibration data of the column of image sensitive units includes FPN calibration data of the class of M-channel, and the number of FPN calibration data corresponding to each class of channel is smaller than the number of image sensitive units corresponding to that class of channel.

4. The image noise calibration apparatus of claim 3, wherein the raw image data is image data of a Bayer domain.

5. The image noise calibration apparatus of claim 2, wherein the image sensitive units in the column are configured to output image data of a class of M-channel, and the number of FPN calibration data of the column of image sensitive units is M.

6. The image noise calibration apparatus of claim 5, wherein:

the raw image data is image data of an RGB Bayer domain; and

the class of M-channel includes a blue channel, a red channel, and two green channels.

7. The image noise calibration apparatus of claim 2, wherein the number of FPN calibration data of the column of image sensitive units is one.

8. The image noise calibration apparatus of claim 1, wherein determining the FPN calibration data of the image sensor based on the raw image data output by the image sensitive unit array further includes:

determining FPN data of each image sensitive unit based on the raw image data output by the image sensitive unit array; and

determining FPN data, among FPN data of each of the image sensitive units, that is larger than a threshold as the FPN calibration data of the image sensor.

9. The image noise calibration apparatus of claim 1, wherein the processor executing the computer readable instruction set further implements:

acquiring a dark current correction value of the image sensitive unit array; and

determining the FPN calibration data of the image sensor based on the raw image data output by the image sensing unit array further includes;

determining the FPN calibration data of the image sensor based on the raw image data output by the image sensitive unit array and the dark current correction value of the image sensitive unit array, wherein the FPN calibration data of the image sensor is a value with the dark current correction value being removed.

10. The image noise calibration apparatus according to any claim 1, wherein the quantity of the raw image data is at least one frame;

and

determining the FPN calibration data of the image sensor based on the raw image data output by the image sensing unit array further includes;

determining FPN calibration data for each frame based on each frame of raw image data output by the image sensitive unit array, and averaging the FPN calibration data of each frame to obtain the FPN calibration data of the image sensor;

or,

determining the FPN calibration data of the image sensor based on an average of the at least one frame of raw image data output by the image sensitive unit array.

11. The image noise calibration apparatus of claim 1, wherein the processor executing the computer readable instruction set further implements:

storing the FPN calibration data of the image sensor in a storage unit of a noise reduction module.

12. The image noise calibration apparatus of claim 11, wherein the noise reduction module is an image sensor, an image processor, or a noise reduction circuit connected to the image sensor.

13. The image noise calibration apparatus of claim 11, wherein the processor executing the computer readable instruction set further implements:

generating check data according to the FPN calibration data; and

storing the FPN calibration data and the check data in the storage unit of the noise reduction module.

14. The image noise calibration apparatus of claim 13, wherein the check data is generated using a cyclic redundancy check algorithm.

15. The image noise calibration apparatus of claim 13, wherein the processor executing the computer readable instruction set further implements:

verifying the FPN calibration data based on the check data.

16. The image noise calibration apparatus of claim 1, wherein, before the image sensor acquires an image, the processor executing the computer readable instruction set further implements:

determining that no valid FPN calibration data is stored in the image sensitive unit array.

17. The image noise calibration apparatus of claim 16, wherein determining that no valid FPN calibration data is stored in the image sensitive unit array further includes:

reading data in a storage unit, if the FPN calibration data does not exist or the FPN calibration data fails to be verified, it is determined that no valid FPN calibration data is stored in the image sensitive unit array.

18. An image noise reduction apparatus, comprising a noise reduction module, wherein the noise reduction module is configured to perform a noise reduction process on raw image data output by an image sensor, and the noise reduction process includes:

acquiring the raw image data output by the image sensor;

calculating compensation data of the raw image data according to pre-stored FPN calibration data of the image sensor; and

compensating the raw image data based on the compensation data.

19. The image noise reduction apparatus of claim 18, wherein calculating the compensation data of the raw image data according to the pre-stored FPN calibration data of the image sensor and the raw image data further includes:

obtaining a compensation level; and

calculating the compensation data of the raw image data based on the FPN calibration data and the compensation level.

20. The image noise reduction apparatus of claim 18, wherein the noise reduction process is performed when it is determined that exposure information corresponding to the raw image data satisfies a predetermined condition, and/or when it is determined that the FPN calibration data of the image sensor is successfully verified.