Digital camera with selectively increased dynamic range by control of parameters during image acquisition

• US 8,687,087 B2
• Filed: 06/01/2007
• Issued: 04/01/2014
• Est. Priority Date: 08/29/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Inventors: Pertsel, Shimon, Meitav, Ohad, Pozniansky, Eli, Galil, Erez
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US11/756,733
• Publication Number:
  US 20080055426A1
• Time in Patent Office: 2,496 Days
• Field of Search: 348/229.1, 348/230.1, 348/255, 348/256, 348/257
• US Class Current: 348/229.1
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 25/78 Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters

Abstract

A parameter of a digital camera or other digital image acquisition device is adjusted to maintain the resulting digital signal within a range carried by a digital processing path that carries a limited number of bits. The magnitude of the parameter is then also used to represent the image. Examples of the parameter include analog signal gain and exposure time. This is a cost effective way to increase the dynamic range of a digital camera, instead of increasing the width of its digital processing path. The digital data may be processed to obtain either tone mapped images, which are compatible with current cameras and other equipment, or to obtain images with a greater dynamic range for use with suitable displays and the like.

Claims

1. A method of processing electronic signals of at least one optical image that has a dynamic range of magnitudes outside of a predefined range of magnitudes, comprising:

acquiring preview image data having a reduced resolution that is less than a resolution of the at least one optical image, wherein the preview image data is acquired to set gain for acquiring image signals of the at least one optical image;

adjusting magnitudes of the image signals outside of the predefined range by quantities selected to allow measurement of the adjusted signals within the predefined range, the quantities selected based on preview image data acquired prior to the image signals,

representing the image signals by quantities representative of magnitudes of the image signals within the predefined range, and magnitudes of the adjusted signals within the predefined range along with their associated adjustment quantities,

calculating data for a tone mapped image of the at least one optical image, by a processor, from the quantities representative of the magnitudes of the image signals within the predefined range and the magnitudes of the adjusted signals within the predefined range and their associated adjustment quantities, and

outputting, by the processor, data of the tone mapped image via an output, wherein the data of the tone mapped image is further generated based on data generated by the analog-to-digital converter and data associated with photo exposure time of the image signals.

2. The method of claim 1, additionally comprising reconstructing at least one image from the quantities representative of the magnitudes of the image signals within the predefined range, and the magnitudes of the adjusted signals within the predefined range along with their associated adjustment quantities.

3. The method of claim 1, additionally comprising storing in a non-volatile memory the quantities representative of the magnitudes of the image signals within the predefined range, and the magnitudes of the adjusted signals within the predefined range and their associated adjustment quantities.

4. The method of any one of claims 1-3, wherein the associated adjustment quantities include a gain of the electronic signals.

5. The method of any one of claims 1-3, wherein the associated adjustment quantities include quantities related to a time of exposure.

6. The method of claim 1, additionally comprising displaying an image from the tone mapped image data.

7. The method of claim 1, additionally comprising reconstructing data of an image having a wide dynamic range by combining the tone mapped image data with the adjustment quantities associated with the adjusted signals.

8. The method of claim 7, additionally comprising displaying an image from the reconstructed data.

9. The method of claim 1, wherein the predefined range of magnitudes is defined by a limitation of a system on which the signal processing takes place.

10. The method of claim 1, additionally comprising, prior to adjusting magnitudes of the signals, determining whether the magnitudes of the signals are outside the predefined range, and, if so, thereafter determining their associated adjustment quantities.

11. The method of claim 10, wherein determining whether the magnitudes of the signals are outside the predefined range of magnitudes includes referencing stored signals of a prior image.

12. The method of claim 11, wherein the stored signals of a prior image include stored signals of preview images having a resolution that is less than a resolution of the magnitudes of the adjusted signals within the predefined range.

13. The method of claim 10, wherein determining whether the magnitudes of the signals are outside the predefined range of magnitudes includes referencing signals of one or more pixels within the same image.

14. The method of claim 10, wherein determining whether the magnitudes of the signals are outside the predefined range of magnitudes includes comparing the magnitudes of the signals with at least one reference magnitude.

15. The method of claim 1, wherein the signals that represent the magnitudes of individual pixels of the optical image and the adjustment quantities are associated with the individual pixels.

16. The method of claim 1, wherein the signals that represent the magnitudes of individual pixels of the optical image and the adjustment quantities are associated with blocks that individually include a plurality of individual pixels.

17. The method of claim 1, additionally comprising processing the images by utilizing at least one image processing algorithm with magnitudes of the image signals within the predefined range, and magnitudes of the adjusted signals within the predefined range, and thereafter combining the results.

18. The method of claim 17, additionally comprising thereafter employing the associated predefined range adjustment quantities with the combined results.

19. The method of claim 17, wherein combining the results of the processes includes employing the associated predefined range adjustment quantities.

20. The method of claim 17, wherein the at least one image processing algorithm includes a de-mosaicing algorithm.

21. The method of claim 17, wherein the at least one image processing algorithm includes a white balance algorithm.

22. The method of claim 17, wherein the at least one image processing algorithm includes a digital lighting algorithm.

23. The method of claim 17, wherein the at least one image processing algorithm includes a color mapping algorithm.

24. The method of claim 17, wherein the at least one image processing algorithm includes a noise reduction algorithm.

25. The method of claim 17, wherein the at least one image processing algorithm includes a highlight recovery algorithm.

26. The method of claim 17, wherein the at least one image processing algorithm includes a tone mapping algorithm.

27. The method of claim 17, wherein the at least one image processing algorithm includes a JPEG compression algorithm.

28. The method of claim 17, wherein the at least one image processing algorithm includes a HDR image reconstruction algorithm.

29. A method of acquiring data of at least one image, comprising:

acquiring preview image data having a reduced resolution that is less than a resolution of data for the at least one image, wherein the preview image data is acquired to set gain for acquiring image signals of the image;

directing the image onto a photosensor that generates an analog signal representing magnitudes of pixels within the image;

passing the photosensor analog signal through a controllable gain amplifier that provides an output of a level controlled analog signal;

applying the level controlled analog signal to an analog-to-digital converter having a defined input range window and which outputs digital signals representing magnitudes of the level controlled analog signal within the window;

determining whether the level controlled analog signal has a magnitude outside the window, and, if so, adjusting the gain of the variable gain amplifier to maintain the magnitude level controlled analog signal within the input range window of the analog-to-digital converter, the gain selected based on preview image data acquired prior to the image;

recording both the digital output signals of the analog-to-digital converter and associated values of gain of the controllable gain amplifier in a manner to be useable to recreate the image with a range in excess of that of the window;

calculating data for a tone mapped image of the at least one image, by a processor, from quantities representative of magnitudes of the level controlled analog signal within the input range window and magnitude of the adjusted signals within the input range window and their associated adjustment quantities; and

outputting, by the processor, data of the tone mapped image via an output, wherein the data of the tone mapped image is further generated based on data generated by an analog-to-digital converter and data associated with photo exposure time of the at least one image.

30. The method of claim 29, wherein determining whether the level controlled analog signal has a magnitude outside the window and adjusting the gain of the variable gain amplifier are both carried out for magnitudes of individual pixels of the photosensor analog signal.

31. The method of claim 29, wherein determining whether the level controlled analog signal has a magnitude outside the window and adjusting the gain of the variable gain amplifier are both carried out for magnitudes representing individual groups of pixels that include a plurality of adjacent pixels.

32. The method of claim 29, wherein determining whether the level controlled analog signal has a magnitude outside the window includes comparing the photosensor analog signal with at least one analog reference level.

33. The method of claim 29, wherein determining whether the level controlled analog signal has a magnitude outside the window includes comparing the output digital signals of the analog-to-digital converter with at least one digital reference level.

34. A method of processing electronic signals of at least one optical image that has a dynamic range of magnitudes outside of a predefined range of magnitudes, comprising:

acquiring preview image data having a reduced resolution that is less than a resolution of the at least one optical image, wherein the preview image data is acquired to set gain for acquiring image signals of the at least one optical image;

adjusting magnitudes of the image signals outside of the predefined range by quantities selected to maintain the adjusted signals within the predefined range during acquisition of them, the quantities selected based on preview image data acquired prior to the image signals;

processing the image signals by utilizing at least one image processing algorithm with magnitudes of the image signals within the predefined range, and with quantity adjusted magnitudes of the image signals outside of the predefined range, and thereafter combining the results;

calculating data for a tone mapped image of the at least one optical image, by a processor, from the quantities representative of the magnitudes of the image signals within the predefined range and the magnitudes of the adjusted signals within the predefined range and their associated adjustment quantities; and

outputting, by the processor, data of the tone mapped image via an output, wherein the data of the tone mapped image is further generated based on data generated by an analog-to-digital converter and data associated with photo exposure time of the image signals.

35. The method of claim 34, wherein the associated predefined range adjustment quantities are employed subsequent to the combining the results.

36. The method of claim 34, wherein the associated predefined range adjustment quantities are employed during the combining the results.

37. The method of claim 34, wherein the associated predefined range adjustment quantities include an image signal gain.

38. The method of claim 34, wherein the associated predefined range adjustment quantities include a quantity related to a time of exposure during acquisition of the image signals.

39. The method of claim 34, wherein the image processing includes use of a de-mosaicing algorithm.

40. The method of claim 34, wherein the image processing includes use of a white balance algorithm.

41. The method of claim 34, wherein the image processing includes use of a noise reduction algorithm.

42. The method of claim 34, wherein the image processing includes use of a color mapping algorithm.

43. The method of claim 34, wherein the image processing includes use of a digital lighting algorithm.

44. The method of claim 34, wherein the image processing includes use of a highlight recovery algorithm.

45. The method of claim 34, wherein the image processing includes use of a tone mapping algorithm.

46. The method of claim 34, wherein the image processing includes use of a JPEC compression algorithm.

47. The method of claim 34, wherein the image processing includes use of a HDR image reconstruction algorithm.

48. The method of claim 34, additionally comprising reconstructing at least one image from quantities representative of the magnitudes of the processed image signals within the predefined range, and the magnitudes of the processed adjusted signals within the predefined range along with their associated adjustment quantities.

49. The method of claim 34, additionally comprising storing in a non-volatile memory quantities representative of the magnitudes of the processed image signals within the predefined range, and the magnitudes of the processed adjusted signals within the predefined range and their associated adjustment quantities.

50. The method of claim 34, wherein the predefined range of magnitudes is defined by a limitation of a system on which the signal processing takes place.

51. The method of claim 34, wherein the signals that represent magnitudes based upon individual pixels of the optical image and the adjustment quantities are associated with the individual pixels.

52. The method of claim 34, wherein the signals that represent magnitudes based upon individual pixels of the optical image and the adjustment quantities are associated with blocks that individually include a plurality of individual pixels.

53. A method of acquiring and processing data of an optical image from a source, comprising:

acquiring preview image data having a reduced resolution that is less than a resolution of data for the optical image, wherein the preview image data is acquired to set gain for acquiring the data of the optical image from the source;

dividing a continuum of potential magnitudes from the source into a plurality of discrete ranges;

representing magnitudes from the source by data of values within individual ranges and associated data that identify the ranges in which the data of values lie, the associated data selected based on preview image data acquired prior to the data for the optical image from the source;

individually processing the data of values within the individual ranges without regard to the identity of the ranges in which they lie;

thereafter combining the processed data from multiple ranges;

calculating data for a tone mapped image of the optical image, by a processor, from quantities representative of the magnitudes of data for the optical image within an individual range and the magnitudes of the data of values within the individual range and associated adjustment quantities; and

outputting, by the processor, data of the tone mapped image via an output, wherein the data of the tone mapped image is further generated based on data generated by an analog-to-digital converter and data associated with photo exposure time of the data for the optical image.

54. The method of claim 53, wherein combining the processed data additionally includes combining the data with the processed data with the identity of the ranges in which the data lie.

55. The method of claim 53, wherein the data are acquired from a source including a photosensor having an output of magnitudes that represent a luminance of an optical image cast onto the photosensor.

56. The method of claim 53, wherein dividing the continuum of potential magnitudes from the source includes defining a plurality of discrete ranges of magnitudes that are the same, the discrete ranges of magnitudes being selected to be within the processing capabilities of a device that processes the data of values within the individual ranges.

57. A digital camera, comprising:

a photosensor having an electrical output proportional to a luminance of an optical image incident thereon;

an optical system that casts an optical image of an object scene onto the photosensor;

an analog-to-digital converter that generates digital data from an analog signal within a defined window of values at an input thereof;

an adjustable gain amplifier receiving the analog signal as an output of the photosensor and connected with the input of an analog-to-digital converter, the gain of the amplifier being controlled to maintain an input to the analog-to-digital converter within the window thereof, the gain selected based on preview image data acquired prior to acquiring the analog signal of the optical image of the object scene; and

a processor that utilizes both the data generated by the analog-to-digital converter and data of the gain of the amplifier to generate data to represent the luminance of the optical image cast onto the photosensor, wherein the processor executes instructions that enable actions, including;

acquiring preview image data having a reduced resolution that is less than a resolution of the analog signal of the optical image, wherein the preview image data is acquired to set gain for the amplifier that is acquiring the analog signal of the optical image;

calculating data for a tone mapped image from the quantities representative of magnitudes of the optical image'"'"'s analog signals within a predefined range for the window of values and the magnitudes of the adjusted analog signals within the predefined range and their associated adjustment quantities; and

outputting data of the tone mapped image via an output, wherein the data of the tone mapped image is further generated based on data generated by an analog-to-digital converter and data associated with photo exposure time of the optical image'"'"'s analog signals.

58. The camera of claim 57, additionally comprising an output through which the data of the image cast onto the photosensor are passed, including data of the gain of the amplifier.

59. The camera of claim 57, wherein the data of tone mapped images are passed through the data output without data of the gain of the amplifier.