Transfer function and high dynamic range images

• US 7,626,614 B1
• Filed: 02/15/2005
• Issued: 12/01/2009
• Est. Priority Date: 02/15/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Marcu, Gabriel G.
• Primary Examiner(s): Tran; Nhan T
• Assistant Examiner(s): Diep; Trung
• Application Number:
  US11/058,770
• Time in Patent Office: 1,750 Days
• Field of Search: 348/298, 348/362, 348/222.1, 382/266, 358/228
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

Methods and apparatuses to derive at least one exposure function for high dynamic range imaging from images of different exposure durations. In one aspect, a method to generate a high dynamic range image from a plurality of images taken with a plurality of exposure durations respectively, includes: computing an exposure function from the plurality of images; and combining the plurality of images into a high dynamic range (HDR) image using the exposure function. In one example of an embodiment, the exposure function is evaluated through: evaluating first data representing a derivative of the exposure function from pixel values of the plurality of images; and integrating the first data to generate second data representing the exposure function.

Claims

1. A machine readable storage medium storing executable computer program instructions which when executed by a data processing system cause said system to perform a method for high dynamic range imaging, the method comprising:

evaluating first data representing a derivative of an exposure function from pixel values of a plurality of images taken with a plurality of exposure durations respectively, wherein at least some of the pixel values represent a region receiving substantially constant irradiance in the plurality of images; and

integrating the first data to generate second data representing the exposure function.

2. The medium of claim 1, further comprising:

combining the plurality of images into a high dynamic range (HDR) image using the exposure function.

3. The medium of claim 2, wherein for the images the exposure function indicates variation in exposure with respect to pixel value level produced at corresponding exposure.

4. The medium of claim 3, wherein said combining comprises:

averaging data representing irradiance evaluated according to the pixel values of the plurality of images and the second data.

5. The medium of claim 4, wherein the data representing irradiance is averaged using weight based on the pixel values.

6. The medium of claim 2, wherein said evaluating comprises:

computing the first data from data representing a derivative of logarithm of exposure duration with respect to pixel value for one or more pixels in a region receiving substantially constant irradiance in the plurality of images.

7. The medium of claim 6, wherein the one or more pixels comprise a contiguous block of pixels.

8. The medium of claim 7, wherein the block of pixels have pixel values within a tolerant threshold from an average of pixel values of the block of pixels.

9. The medium of claim 2, wherein said evaluating comprises:

computing a rate of change in logarithm of exposure duration over change in pixel value one or more pixels representing a region receiving substantially constant irradiance in the plurality of images.

10. The medium of claim 2, wherein said evaluating comprises:

evaluating a rate of exposure duration change over a product of exposure duration and pixel value change for the plurality of images.

11. The medium of claim 10, wherein the rate is evaluated for one or more pixels representing a region receiving substantially constant irradiance in the plurality of images.

12. The medium of claim 2, wherein the exposure function is evaluated from pixel values for one of:

a red channel;

a green channel; and

a blue channel.

13. The medium of claim 2, wherein the exposure function is evaluated from pixel values for a gray level.

14. A method of image processing, the method comprising:

evaluating first data representing a derivative of an exposure function from pixel values of a plurality of images of a scene taken with a plurality of exposure durations, wherein at least some of the pixel values represent a region receiving substantially constant irradiance in the plurality of images;

integrating the first data to generate second data representing the exposure function; and

generating an image, for display on a display device, of the scene from the plurality of images and the second data representing the exposure function.

15. The method of claim 14, wherein the image of the scene is generated from averaging data representing irradiance evaluated according to the pixel values of the plurality of images and the second data;

and wherein the data representing irradiance is averaged using weight based on the pixel values.

16. The method of claim 14, wherein said evaluating comprises:

computing the first data from data representing a derivative of logarithm of exposure duration with respect to pixel value for one or more pixels.

17. A data processing system for image processing, the data processing system comprising:

means for evaluating first data representing a derivative of an exposure function from pixel values of a plurality of images of a scene taken with a plurality of exposure durations, wherein at least some of the pixel values represent a region receiving substantially constant irradiance in the plurality of images; and

means for integrating the first data to generate second data representing the exposure function.

18. The data processing system of claim 17, wherein for the images the exposure function represents a logarithm of exposure as a function of pixel value level produced at corresponding exposure.

19. The data processing system of claim 17, further comprising:

means for generate an image of the scene from the plurality of images and the second data representing the exposure function through averaging data representing irradiance evaluated according to the pixel values of the plurality of images and the second data.

20. The data processing system of claim 19, wherein the data representing irradiance is averaged using weight based on the pixel values;

and the weight is larger for pixel values closer to saturated pixel values than for pixel values away from the saturated pixel values.

21. The data processing system of claim 17, wherein said means for evaluating comprises:

means for computing the first data from data representing a derivative of logarithm of exposure duration with respect to pixel value for one or more pixels in a region receiving substantially constant irradiance in the plurality of images.

22. The data processing system of claim 21, wherein said means for evaluating comprises:

means for selecting the one or more pixels based on pixel values in the plurality of images.

23. An image capturing device, comprising:

memory;

an image sensing device coupled to the memory to generate a plurality of images of a scene with a plurality of exposure durations; and

a processing circuit coupled to the image sensing device, the processing circuit to compute an exposure function from the plurality of images and combine the plurality of images into a high dynamic range (HDR) image using the exposure function,

wherein to compute the exposure function the processing circuit is to;

evaluate first data representing a derivative of the exposure function from pixel values of the plurality of images and the plurality of exposure durations, wherein at least some of the pixel values represent a region receiving substantially constant irradiance in the plurality of images; and

integrate the first data to generate second data representing the exposure function.

24. The device of claim 23, wherein the first data represents logarithm of exposure as a function of pixel value.

25. The device of claim 23, wherein for the images the exposure function indicates variation in exposure with respect to pixel value level produced at corresponding exposure;

and the HDR image of the scene is generated from averaging data representing irradiance evaluated according to the pixel values of the plurality of images and the second data.

26. The device of claim 25, wherein the data representing irradiance is averaged using weight based on the pixel values;

and wherein the weight for pixel values near saturated values is smaller than for pixel values away from the saturated values.

27. The device of claim 23, further comprising:

a housing; and

a lens coupled to the housing;

wherein the processing circuit and the image sensing device are enclosed within the housing.

28. A machine readable storage medium storing executable computer program instructions which when executed by a data processing system cause said system to perform a method for high dynamic range imaging, the method comprising:

receiving a plurality of images taken with a plurality of exposure durations respectively;

determining differences in exposure durations with respect to differences in pixel values of the plurality images to evaluate an exposure function; and

determining an irradiance map from the plurality of images using the exposure function,

wherein said determining differences in exposure durations with respect to differences in pixel values of the plurality images comprises;

determining a ratio between a difference in logarithm of two exposure durations and a difference in pixel values in two of the plurality of images taken with the two exposure durations respectively.

29. The medium of claim 28, wherein said determining differences in exposure durations with respect to differences in pixel values of the plurality images comprises:

determining a ratio between a difference in two exposure durations and a product of a combination of two pixel values and a difference in pixel values in the pixel values, the pixel values corresponding to a point of a scene in two of the plurality of images taken with the two exposure durations respectively.

30. A machine readable storage medium storing executable computer program instructions which when executed by a data processing system cause said system to perform a method for high dynamic range imaging, the method comprising:

receiving a plurality of images taken with a plurality of exposure durations respectively;

determining an exposure function from mathematically integrating a slope curve of the exposure function derived from the plurality of images and the plurality of exposure durations; and

recovering a high dynamic range (HDR) image from the plurality of images and the plurality of exposure duration using the exposure function.

31. The medium of claim 30, wherein the high dynamic range image comprises an irradiance map.

32. The medium of claim 30, wherein the high dynamic range image has a dynamic range larger than any of the plurality of images.

33. An image capturing device comprising:

a memory;

an image sensing device coupled to the memory to generate a plurality of images with a plurality of exposure durations; and

a processing circuit coupled to the image sensing device, the processing circuit configured to

determine an exposure function from a mathematical integration of a slope curve of the exposure function derived from the plurality of images and the plurality of exposure durations; and

recover a high dynamic range (HDR) image from the plurality of images using the exposure function.

34. The image capturing device of claim 33, wherein the plurality of exposure durations comprise a set of images having different exposure durations,

wherein the plurality of images each include a common scene, and

wherein the HDR image has a dynamic range larger than any of the plurality of images.