Image pipeline with dual demosaicing circuit for efficient image processing

• US 10,692,177 B2
• Filed: 08/10/2018
• Issued: 06/23/2020
• Est. Priority Date: 08/10/2018
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Wang, Muge, Pope, David R.
• Primary Examiner(s): Bhattacharya, Sam
• Application Number:
  US16/100,709
• Publication Number:
  US 20200051210A1
• Time in Patent Office: 683 Days
• Field of Search: 382162, 382167, 348 71, 3482221, 348246
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 3/4015 Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

Embodiments relate to a first demosaicing circuit and a second demosaicing circuit that can perform demosaicing of image data. The first demosaicing circuit processes received image data to generate a first demosaiced image for obtaining statistic information on the received image data. The second demosaicing circuit performs demosaicing of the received image data to generate a second demosaiced image. A processing circuit pipeline performs at least one of resampling, noise processing, color processing and output rescaling performed on the second demosaiced image based on the statistics information obtained from the first demosaiced image.

Claims

1. An apparatus for processing image signal data, comprising:

an image statistics circuit configured to generate statistics information on a received image data by processing a first demosaiced image, the image statistics circuit comprising a first demosaicing circuit configured to process the received image data to generate the first demosaiced image;

a second demosaicing circuit configured to perform demosaicing of the received image data to generate a second demosaiced image; and

a processing circuit pipeline configured to perform at least one of resampling, noise processing, color processing and output resealing on the second demosaiced image after receiving the statistics information generated by the image statistics circuit.

2. The apparatus of claim 1, wherein the first demosaicing circuit has a latency lower than a latency of the second demosaicing circuit.

3. The apparatus of claim 1, wherein the first demosaicing circuit is configured to generate the first demosaiced image by, for each output pixel location, obtaining an average of two green pixel values in different rows of the received image data adjacent to the output pixel location.

4. The apparatus of claim 1, wherein the first demosaicing circuit is configured to generate the first demosaiced image by, for each output pixel location, obtaining a weighted average of two red pixel values in a same row of received image data closest to the output pixel location.

5. The apparatus of claim 4, wherein one of the red pixel values is weighted three times compared to another of the red pixel values.

6. The apparatus of claim 1, wherein the first demosaicing circuit is configured to generate the first demosaiced image by, for each output pixel location, obtaining a weighted average of two blue pixel values in a same row of pixels of the received image data closest to the output pixel location.

7. The apparatus of claim 6, wherein one of the blue pixel values is weighted three times compared to another of the blue pixel values.

8. A method of demosaicing image data from an image sensor, comprising:

generating, by an image statistics circuit, statistics information on a received image data by processing a first demosaiced image, the image statistics circuit comprising a first demosaicing circuit configured to process the received image data to generate the first demosaiced image;

performing, by a second demosaicing circuit, demosaicing of the received image data to generate a second demosaiced image; and

performing, by a processing circuit pipeline, at least one of resampling, noise processing, color processing and output resealing on the second demosaiced image after receiving the statistics information generated by the image statistics circuit.

9. The method of claim 8, wherein the first demosaicing circuit has a latency lower than a latency of the second demosaicing circuit.

10. The method of claim 8, wherein the first demosaicing circuit generates the first demosaiced image by, for each output pixel location, obtaining an average of two green pixel values in different rows of the received image data adjacent to the output pixel location.

11. The method of claim 8, wherein the first demosaicing circuit generates the first demosaiced image by, for each output pixel location, obtaining a weighted average of two red pixel values in a same row of received image data closest to the output pixel location.

12. The method of claim 11, wherein one of the red pixel values is weighted three times compared to another of the red pixel values.

13. The method of claim 8, wherein the first demosaicing circuit generates the first demosaiced image by, for each output pixel location, obtaining a weighted average of two blue pixel values in a same row of pixels of the received image data closest to the output pixel location.

14. The method of claim 13, wherein one of the blue pixel values is weighted three times compared to another of the blue pixel values.

15. An electronic device, comprising:

an image sensor configured to generate image data;

an image statistics circuit configured to generate statistics information on a received image data by processing a first demosaiced image, the image statistics circuit comprising a first demosaicing circuit configured to process the received image data to generate the first demosaiced image;

a second demosaicing circuit configured to perform demosaicing of the received image data to generate a second demosaiced image; and

a processing circuit pipeline configured to perform at least one of resampling, noise processing, color processing and output resealing on the second demosaiced image after receiving the statistics information generated by the image statistics circuit.

16. The electronic device of claim 15, wherein the first demosaicing circuit has a latency lower than a latency of the second demosaicing circuit.

17. The electronic device of claim 15, wherein the first demosaicing circuit is configured to generate the first demosaiced image by, for each output pixel location, obtaining an average of two green pixel values in different rows of the received image data adjacent to the output pixel location.

18. The electronic device of claim 15, wherein the first demosaicing circuit is configured to generate the first demosaiced image by, for each output pixel location, obtaining a weighted average of two red pixel values in a same row of received image data closest to the output pixel location.

19. The electronic device of claim 15, wherein the first demosaicing circuit is configured to generate the first demosaiced image by, for each output pixel location, obtaining a weighted average of two blue pixel values in a same row of pixels of the received image data closest to the output pixel location.

20. The electronic device of claim 19, wherein one of the blue pixel values is weighted three times compared to another of the blue pixel values.