System and method for processing images using predetermined tone reproduction curves

• US 7,893,975 B2
• Filed: 06/01/2007
• Issued: 02/22/2011
• Est. Priority Date: 10/13/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hayward, David, Marcu, Gabriel G., Zimmer, Mark
• Primary Examiner(s): WHIPKEY, JASON T
• Application Number:
  US11/756,918
• Publication Number:
  US 20080088858A1
• Time in Patent Office: 1,362 Days
• Field of Search: 348/188, 348/254, 348/674, 382/274, 358/519
• US Class Current: 348/254
• CPC Class Codes: H04N 1/407 Control or modification of tonal gradation or of extreme levels, e.g. background level
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration

Abstract

An automated RAW image processing method and system are disclosed. A RAW image and metadata related to the RAW image are obtained from a digital camera or other source. The RAW image and the related metadata are automatically processed using an Operating System service of a processing device to produce a resulting image in an absolute color space. When automatically processing, a predetermined tone reproduction curve is applied to the interpolate RAW image to produce the resulting image. The predetermined tone reproduction curve is derived from a plurality of reference images and is selected based on the metadata associated with the RAW image. The resulting image is then made available to an application program executing on the processing device through an application program interface with the Operating System service.

Claims

1. An automated RAW image processing method implementable by a processing device, the method comprising:

storing a plurality of predetermined tone curves, each of the predetermined tone curves derived by values of each pixel in a first reference image of a scene being paired to values of corresponding pixels in a second reference image of the scene, the first reference image obtained using a first camera, each of the predetermined tone curves generated based on the pairings and being associated with camera information for the first camera;

interpolating a RAW image to produce an interpolated image;

determining based on metadata associated with the RAW image which one or more of the predetermined tone curves to apply;

applying the one or more predetermined tone curves to the interpolated image to produce a resulting image; and

making the resulting image available to an application executing on the processing device.

2. The method of claim 1, wherein the processing device comprises a computer.

3. The method of claim 1, comprising initially receiving the RAW image and the associated metadata communicated from an imaging device.

4. The method of claim 3, wherein the imaging device comprises a digital camera.

5. The method of claim 1, wherein each of the plurality of predetermined tone curves is associated with a digital camera identifiable by the metadata associated with the RAW image.

6. The method of claim 1, wherein each of the one or more predetermined tone curves is derived by—

transforming the first and second reference images into grayscale images,

pairing grayscale values for the two reference images,

generating the predetermined tone curve based on the pairing, and

associating the predetermined tone curve with the camera information for the first camera.

7. The method of claim 1, wherein each of the one or more predetermined tone curves is derived by—

filtering the first reference image into a plurality of first color channel images, each first color channel image having pixel values of one color channel,

filtering the second reference image into a plurality of second color channel images, each second color channel image having pixel values of one color channel,

pairing color channel values for corresponding pixels of each of the first and second color channel images of the same color channel,

generating a plurality of color channel tone curves based on the pairings, and

associating the color channel tone curves with the camera information for the first camera.

8. The method of claim 1, wherein the act of interpolating the RAW image comprises demosaicing the RAW image into an interpolated image having a color space.

9. The method of claim 1, wherein the act of determining based on metadata associated with the RAW image which one of the one or more predetermined tone curves to apply comprises comparing information of the metadata identifying a digital camera to the camera information associated with the predetermined tone curves.

10. The method of claim 1, wherein the act of applying the predetermined tone curve to the interpolated image comprises:

determining tone values of each pixel of the interpolated image; and

adjusting the determined tone values of each pixel to match corresponding tone values from the predetermined tone curve.

11. The method of claim 1, wherein the act of making the resulting image available to an application executing on the processing device comprises interfacing an operating system service performing automated RAW image processing to the application using an application programming interface.

12. A non-transitory program storage device, readable by a programmable processing device, comprising instructions stored thereon for causing the programmable processing device to perform an automated RAW image processing method comprising:

storing a plurality of predetermined tone curves, each of the predetermined tone curves derived by values of each pixel in a first reference image of a scene being paired to values of corresponding pixels in a second reference image of the scene, the first reference image obtained using a first camera, each of the predetermined tone curves generated based on the pairings and being associated with camera information for the first camera;

interpolating a RAW image to produce an interpolated image;

determining based on metadata associated with the RAW image which one or more of the predetermined tone curves to apply;

applying the one or more predetermined tone curves to the interpolated image to produce a resulting image; and

making the resulting image available to an application executing on the processing device.

13. The program storage device of claim 12, wherein the processing device comprises a computer.

14. The program storage device of claim 12, comprising initially receiving the RAW image and the associated metadata communicated from an imaging device.

15. The program storage device of claim 14, wherein the imaging device comprises a digital camera.

16. The program storage device of claim 12, wherein each of the plurality of predetermined tone curves is associated with a digital camera identifiable by the metadata associated with the RAW image.

17. The program storage device of claim 12, wherein each of the one or more predetermined tone curves is derived by—

transforming the first and second reference images into grayscale images,

pairing grayscale values for the two reference images,

generating the predetermined tone curve based on the pairing, and

associating the predetermined tone curve with the camera information for the first camera.

18. The program storage device of claim 12, wherein each of the one or more predetermined tone curves is derived by—

filtering the first reference image into a plurality of first color channel images, each first color channel image having pixel values of one color channel,

filtering the second reference image into a plurality of second color channel images, each second color channel image having pixel values of one color channel,

pairing color channel values for corresponding pixels of each of the first and second color channel images of the same color channel,

generating a plurality of color channel tone curves based on the pairings, and

associating the color channel tone curves with the camera information for the first camera.

19. The program storage device of claim 12, wherein the act of interpolating the RAW image comprises demosaicing the RAW image into an interpolated image having a color space.

20. The program storage device of claim 12, wherein the act of determining based on metadata associated with the RAW image which one of the one or more predetermined tone curves to apply comprises comparing information of the metadata identifying a digital camera to the camera information associated with the predetermined tone curves.

21. The program storage device of claim 12, wherein the act of applying the predetermined tone curve to the interpolated image comprises

determining tone values of each pixel of the interpolated image;

and

adjusting the determined tone values of each pixel to match corresponding tone values from the predetermined tone curve.

22. The program storage device of claim 12, wherein the act of making the resulting image available to an application executing on the processing device comprises interfacing an operating system service performing automated RAW image processing to the application using an application programming interface.

23. An automated RAW image processing method implementable by a processing device, the method comprising:

obtaining a first reference image of a scene using a first camera;

obtaining a second reference image of the scene;

pairing values of each pixel in the first reference image to values of corresponding pixels in the second reference image;

generating one or more tone curves based on the pairing;

associating the one or more tone curves with camera information for the first camera; and

storing the one or more tone curves and the associated camera information on a processing device for use in processing a RAW image.

24. The method of claim 23, wherein the scene comprises a color chart having a plurality of color patches.

25. The method of claim 24, wherein the color chart comprises a gray ramp image.

26. The method of claim 23, wherein the act of obtaining the second reference image of the scene comprises using a second camera different from the first camera to obtain the second reference image of the scene.

27. The method of claim 23, wherein the act of obtaining the second reference image of the scene comprises using a different illuminant to obtain the second reference image of the scene than used to obtain the first reference image.

28. The method of claim 23, wherein the act of obtaining the second reference image of the scene comprises making at least one modification to the second reference image with an application that has not been made to the first reference image.

29. The method of claim 23, wherein the act of pairing values of each pixel in the first reference image to values of corresponding pixels in the second reference image comprises:

transforming the first and second reference images into first and second grayscale images,

pairing grayscale values for corresponding pixels in the first and second grayscale images,

generating a tone curve based on the pairing, and

associating the tone curve with the camera information for the first camera.

30. The method of claim 29, wherein the act of transforming comprises scaling each of the first and second reference images.

31. The method of claim 30, wherein the act of scaling comprises substantially reducing differences between the first and second reference images caused from demosaicing or noise.

32. The method of claim 29, wherein the act of transforming comprises:

determining a gain factor that matches a maximum brightness of one of the grayscale images to the other grayscale image; and

scaling grayscale values in the one grayscale image by the determined gain factor.

33. The method of claim 29, wherein the act of generating comprises using interpolation to produce a substantially continuous curve based on the paired grayscale values between the first and second grayscale images.

34. The method of claim 23, wherein the act of pairing values of each pixel in the first reference image to values of corresponding pixels in the second reference image comprises:

filtering the first reference image into a plurality of first color channel images, each first color channel image having pixel values of one color channel,

filtering the second reference image into a plurality of second color channel images, each second color channel image having pixel values of one color channel,

pairing color channel values for corresponding pixels of each of the first and second color channel images of the same color channel,

generating a plurality of color channel tone curves based on the pairings, and

associating the color channel tone curves with the camera information for the first camera.

35. The method of claim 23, further comprising using the one or more tone curves to automatically pre-process a RAW image.

36. The method of claim 35, wherein using the one or more tone curves comprises:

interpolating the RAW image into an interpolated image;

determining camera information from metadata associated with the RAW image;

obtaining the one or more tone curves associated with the determined camera information;

applying the one or more tone curves to one or more color channels of the interpolated image to produce a resulting image; and

making the resulting image available to an application operating on the processing device.