Applying chroma suppression to image data in a scaler of an image processing pipeline

• US 9,911,177 B2
• Filed: 08/31/2015
• Issued: 03/06/2018
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lim, Suk Hwan, Silverstein, D. Amnon
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US14/841,400
• Publication Number:
  US 20170061576A1
• Time in Patent Office: 918 Days
• Field of Search: None
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81
• CPC Class Codes: H04N 7/0117 involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 9/70 for colour killing

Abstract

An image processing pipeline may apply chroma suppression to image data at a scaler implemented in the image processing pipeline. Image data collected for an image may be received at a scaler that is encoded in a color space that includes a luminance component and chrominance components. When resampling the image data to generate a different size of the image, the scaler may attenuate the chrominance components of the image data according to the luminance component of the image data. The scaler may also perform dot error correction and convert the image data from one subsampling scheme to another.

Claims

1. An apparatus, comprising:

an image signal processor that comprises;

one or more front-end pipeline stages;

a scaler; and

one or more back-end pipeline stages that are subsequent to the one or more front-end pipeline stages;

the image signal processor, configured to;

receive, at the scaler, image data collected as part of an image frame, wherein the image data is encoded in a color space comprising a luminance component and a plurality of chrominance components;

resample, by the scaler, the image data in order to rescale the image frame to a different size, wherein to resample the image data the scaler is configured to;

apply chroma suppression to the chrominance components of the image data according to the luminance component of the image data; and

output the resampled image data to one of the one or more back-end pipeline stages.

2. The apparatus of claim 1, wherein the image signal processor is further configured to:

prior to the resample of the image data, correct, by the scaler, one or more dot errors in the image data.

3. The apparatus of claim 1, wherein to apply chroma suppression to the chrominance components of the image data, the image signal processor is configured to:

determine a value for a frequency range of the plurality of chrominance components of the image data;

determine a neutral value for the plurality of the chrominance components of the image data;

calculate a blending ratio for blending the value and the neutral value according to the luminance component of the image data; and

blend the value and the neutral value according to the blending ratio to generate an attenuated value of the chrominance components of the image data.

4. The apparatus of claim 1, wherein the image signal processor is further configured to:

receive, at the scaler, other image data collected as part of another image frame, wherein the other image data is encoded in the color space comprising the luminance component and the plurality of chrominance components; and

apply, by the scaler, chroma suppression to the chrominance components of the other image data according to the luminance component of the other image data to generate a chroma suppressed version of the other image data.

5. The apparatus of claim 4, wherein to resample the image data and to apply chroma suppression to the other image data, the image signal processor is configured to access stored image data in a same set of line buffers.

6. The apparatus of claim 4, wherein the other image data is received at the scaler in a 4:

4;

4 subsampling scheme, and wherein the image signal processor is further configured to;

prior to the application of the chroma suppression, convert the other image data into a 4;

2;

2 subsampling scheme; and

write the chroma suppressed version of the other image data to a memory.

7. The apparatus of claim 1, wherein the apparatus comprises a mobile computing device.

8. A method, comprising:

receiving, at a scaler implemented as part of an image signal processor, image data collected at an image sensor for an image, wherein the image data is encoded in a color space comprising a luminance component and a plurality of chrominance components;

resampling, by the scaler, the image data to scale the image to a different size, wherein the resampling comprises;

attenuating the plurality of chrominance components of the image data according to the luminance component of the image data in order to apply chrominance suppression to the image data; and

providing the resampled image data that is chrominance suppressed to a pipeline stage of the image signal processor for subsequent processing.

9. The method of claim 8, further comprising correcting one or more dot errors detected in the image data prior to resampling the image data.

10. The method of claim 9, wherein correcting the one or more dot errors detected in the image data comprises:

for a given pixel of the image data;

determining a dot detection value for the given pixel;

detecting a dot error at the given pixel, comprising;

determining that the dot detection value subtracted from a luminance value of the pixel exceeds a maximum pixel value of a neighborhood of pixels including the given pixel;

or

determining that the dot detection value added to the luminance value of the pixel exceeds a minimum pixel value of the neighborhood of pixels including the given pixel; and

in response to detecting the dot error, interpolating a corrected value for the luminance value with respect to a direction gradient of the neighborhood of pixels.

11. The method of claim 8, wherein attenuating the plurality of chrominance components comprises:

determining a value for a frequency range of the plurality of chrominance components of the image data;

determining a neutral value for the plurality of the chrominance components of the image data;

calculating a blending ratio for blending the value and the neutral value according to the luminance component of the image data; and

blending the value and the neutral value according to the blending ratio to generate an attenuated value of the chrominance components of the image data.

12. The method of claim 8, further comprising:

receiving, at the scaler, other image data collected at the image sensor for another image, wherein the other image data is encoded in the color space comprising the luminance component and the plurality of chrominance components; and

resampling, by the scaler, the image data to scale the image to a different size without attenuating the plurality of chrominance components.

13. The method of claim 8, wherein the plurality of chrominance components are encoded in an initial subsampling scheme and wherein the resampling further comprises reformatting the plurality of chrominance components into a different subsampling scheme.

14. The method of claim 8, wherein the image signal processor is implemented as part of a mobile computing device that also implements the image sensor.

15. A system, comprising:

a device configured to perform image processing, the device comprising;

a sensor interface configured to receive image data from an image sensor; and

an image processing pipeline configured to;

receive, at a scaler implemented as part of the image processing pipeline, image data collected at the image sensor for an image frame via the sensor interface, wherein the image data is encoded in a color space comprising a luminance component and a plurality of chrominance components;

resample, by the scaler, the image data to scale the image to a different size, wherein the resampling comprises;

vertically scale the image data according to the different size;

attenuate the plurality of chrominance components of the vertically scaled image data according to the luminance component of the image data to apply chrominance suppression to the image data; and

horizontally scale the image data including the attenuated chrominance components according to the different size; and

output the resampled image data as part of scaled version of the image frame.

16. The system of claim 15, wherein the image processing pipeline is further configured to:

prior to the resample of the image data, correct, by the scaler, one or more dot errors in the image data.

17. The system of claim 15, wherein to attenuate the plurality of chrominance components, the image processing pipeline is configured to:

determine a value for a frequency range of the plurality of chrominance components of the image data;

determine a neutral value for the plurality of the chrominance components of the image data;

calculate a blending ratio for blending the value and the neutral value according to the luminance component of the image data; and

blend the value and the neutral value according to the blending ratio to generate an attenuated value of the chrominance components of the image data.

18. The system of claim 15, wherein the image processing pipeline is further configured to:

receive, at the scaler, other image data collected at the image sensor for another image frame via the sensor interface, wherein the other image data is encoded in a color space comprising a luminance component and a plurality of chrominance components;

attenuate, by the scaler, the plurality of chrominance components of the other image data according to the luminance component of the other image data to apply chrominance suppression to the other image data; and

output the other image data as part of a chrominance suppressed version of the other image frame.

19. The system of claim 18, wherein to resample the image data and to attenuate the other image data the scaler is configured to access stored image data in a same set of line buffers.

20. The system of claim 15, wherein the system further comprises an electronic display and wherein the scaled version of the image frame is displayed via the electronic display as a preview image captured via the image sensor.