Method and apparatus to provide edge enhancements as part of a demosaicing process

• US 7,474,337 B1
• Filed: 10/24/2000
• Issued: 01/06/2009
• Est. Priority Date: 10/24/2000
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Cooper, Ted J.
• Primary Examiner(s): Ho; Tuan V
• Application Number:
  US09/696,436
• Time in Patent Office: 2,996 Days
• Field of Search: 348/273, 348/272, 348/280, 348/231.9, 348/625, 348/222.1, 348/254, 348/252, 382/167, 382/162, 382/266, 382/268, 382/269, 382/284, 358/517
• US Class Current: 348/222.1
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values

Abstract

A method and apparatus provide edge enhancements as part of a demosaicing process in which an image is captured. By providing edge enhancements as part of the demosaicing process, the edge enhancement process has access to unaltered spatial and chromatic information contained in the raw captured image data.

Claims

1. An image processing method comprising:

capturing a raw image; and

providing edge enhancements to increase edge detail of the captured raw image as part of a demosaicing process using a brightness map of the captured raw image, wherein the brightness map comprises luminance values extracted from the captured raw image, and the edge enhancement includes setting a color value of an edge pixel of a mask image to a color value of one of a plurality of neighboring pixel groups in a demosaiced raw image, the one neighboring pixel group having an average brightness value closest to a brightness value of the edge pixel.

2. The method of claim 1, further comprising:

performing post demosaicing processing on the captured raw image; and

outputting the processed image.

3. The method of claim 1, wherein the demosaicing process further includes:

detecting edges of the captured raw image using the brightness map;

creating the mask image from the edge detected brightness map; and

performing unsharp edge enhancement on the masked image.

4. The method of claim 3, wherein the demosaicing process further includes:

blending multiplicatively the unsharp edge enhanced image with the demosaiced raw image.

18. The method of claim 1, further comprising:

grouping a neighborhood of pixels around the edge pixel to create the plurality of neighboring pixels groups, wherein the grouping is based on the colors values of each of the neighborhood of pixels; and

computing an average brightness value for each of the plurality of neighboring pixels groups.

19. The method of claim 18, further comprising:

excluding from the plurality of neighboring pixel groups, pixels of the neighborhood of pixels that have neutral colors.

20. The method of claim 18, further comprising:

comparing the brightness value of the edge pixel with the average brightness value of each of the neighboring pixel groups.

5. An apparatus comprising:

an image capturing device to capture a raw image; and

a processor to provide edge enhancements to increase edge detail of the captured raw image as part of a demosaicing process using a brightness map of the captured raw image, wherein the brightness map comprises luminance values extracted from the captured raw image and the edge enhancement includes setting a color value of an edge pixel of a mask image to a color value of one of a plurality of neighboring pixel groups in a demosaiced raw image, the one neighboring pixel group having an average brightness value closest to a brightness value of the edge pixel.

6. The apparatus of claim 5, wherein the processor is to perform post demosaicing processing on the captured raw image and to output the processed image.

7. The apparatus of claim 5, wherein the processor is to detect edges of the captured raw image using the brightness map, to create the mask image from the edge detected brightness map, and to perform unsharp edge enhancement on the masked image.

8. The apparatus of claim 7, wherein the processor is to blend multiplicatively the unsharp edge enhanced image with the demosaiced raw map.

21. The apparatus of claim 5, wherein the processor is to group a neighborhood of pixels around the edge pixel to create the plurality of neighboring pixels groups, wherein the grouping is based on the colors values of each of the neighborhood of pixels and to compute an average brightness value for each of the plurality of neighboring pixels groups.

22. The apparatus of claim 21, wherein the processor is to exclude from the plurality of neighboring pixel groups, pixels of the neighborhood of pixels that have neutral colors.

23. The apparatus of claim 21, wherein the processor is to compare the brightness value of the edge pixel with the average brightness value of each of the neighboring pixel groups.

9. A machine-readable medium that provides instructions, which if executed by a processor, cause the processor to perform the operations comprising:

capturing a raw image; and

providing edge enhancements to increase edge detail of the captured raw image as part of a demosaicing process using a brightness map of the captured raw image, wherein the brightness map comprises luminance values extracted from the captured raw image and the edge enhancement includes setting a color value of an edge pixel of a mask image to a color value of one of a plurality of neighboring pixel groups in a demosaiced raw image, the one neighboring pixel group having an average brightness value closest to a brightness value of the edge pixel.

10. The machine-readable medium of claim 9, further providing instructions, which if executed by the processor, cause the processor to perform the operations comprising:

performing post demosaicing processing on the captured raw image; and

outputting the processed image.

11. The machine-readable medium of claim 9, further providing instructions, which if executed by the processor, cause the processor to perform the operations comprising:

detecting edges of the captured raw image using the brightness map;

creating the mask image from the edge detected brightness map; and

performing unsharp edge enhancement on the masked image.

12. The machine-readable medium of claim 11, further providing instructions, which if executed by the processor, cause the processor to perform the operations comprising:

blending multiplicatively the unsharp edge enhanced image with the demosaiced raw map.

24. The machine-readable medium of claim 9, further providing instructions, which if executed by the processor, cause the processor to perform the operations comprising:

grouping a neighborhood of pixels around the edge pixel to create the plurality of neighboring pixels groups, wherein the grouping is based on the colors values of each of the neighborhood of pixels; and

computing an average brightness value for each of the plurality of neighboring pixels groups.

25. The machine-readable medium of claim 24, further providing instructions, which if executed by the processor, cause the processor to perform the operations comprising:

excluding from the plurality of neighboring pixel groups, pixels of the neighborhood of pixels that have neutral colors.

26. The machine-readable medium of claim 24, further providing instructions, which if executed by the processor, cause the processor to perform the operations comprising:

comparing the brightness value of the edge pixel with the average brightness value of each of the neighboring pixel groups.

13. An image processing device comprising:

an image capturing unit to capture a raw image;

a memory device to store the captured raw image;

an output unit coupled to the memory device; and

a processor to provide edge enhancements to increase edge detail of the captured raw image in the memory device as part of a demosaicing process using a brightness map of the captured raw image, wherein the brightness map comprises luminance values extracted from the captured raw image, the brightness map is approximated from a bi-linear interpolation of the raw image, and the edge enhancement includes setting a color value of an edge pixel of a mask image to a color value of one of a plurality of neighboring pixel groups in a demosaiced raw image the one neighboring pixel group having an average brightness value closest to a brightness value of the edge pixel and to cause the enhanced image to be output is to the output unit.

14. The image processing device of claim 13, wherein the image capturing unit includes a charge-couple device (CCD) array, phototransistors, or photodiodes.

15. The image processing device of claim 13, wherein the output unit is a display device.

16. The image processing device of claim 15, wherein the processor is to perform post demosaicing processing on the captured raw image and to cause the image to be output to the display device.

17. The image processing device of claim 16, wherein the post demosaicing processing is a white balancing processing or a chromatic improvement processing.

27. The image processing device of claim 13, wherein the processor is to group a neighborhood of pixels around the edge pixel to create the plurality of neighboring pixels groups, wherein the grouping is based on the colors values of each of the neighborhood of pixels and to compute an average brightness value for each of the plurality of neighboring pixels groups.

28. The image processing device of claim 13, wherein the processor is to exclude from the plurality of neighboring pixel groups, pixels of the neighborhood of pixels that have neutral colors.

29. The image processing device of claim 13, wherein the processor is to compare the brightness value of the edge pixel with the average brightness value of each of the neighboring pixel groups.