Methods and systems for converting images from low dynamic to high dynamic range

• US 8,265,378 B2
• Filed: 12/24/2004
• Issued: 09/11/2012
• Est. Priority Date: 04/15/2004
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Whitehead, Lorne A., Seetzen, Helge, Ward, Gregory John
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US10/599,955
• Publication Number:
  US 20070269104A1
• Time in Patent Office: 2,818 Days
• Field of Search: 382/167, 382/162, 382/163, 382/166, 382/103, 358/520, 358/518, 358/504, 358/538, 358/540, 358/537, 358/1.9, 358/1.1
• US Class Current: 382/162
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement

Abstract

Aspects of the invention provide systems and methods for converting a digital image represented in a lower bit depth representation to a higher bit depth representation. A saturation region is identified, where a color model value of the pixels in the saturation region is above an upper saturation threshold or below a lower saturation threshold. The color model value for each pixel in the saturation region is then adjusted by a corresponding adjustment. The magnitude of the adjustment for each pixel is based on characteristics of the image data.

Claims

1. A method for converting image data of an image from a lower bit depth representation to a higher bit depth representation, the image having a saturation region wherein a color model value of each individual pixel in the saturation region is one of:

above an upper saturation threshold and below a lower saturation threshold, the method comprising;

identifying pixels in the saturation region; and

adjusting the color model value of one or more individual pixels in the saturation region by a corresponding adjustment, a magnitude of each adjustment dependent, at least in part, on a number of pixels between the corresponding individual pixel and an edge of the saturation region.

2. A method according to claim 1 comprising identifying pixels in the saturation region by scanning the color model values of pixels in the image data along a first scan axis to determine a start pixel at one edge of a saturation region and an end pixel at another edge of the saturation region.

3. A method according to claim 1 comprising determining the magnitude of each adjustment based, at least in part, on a function of the number of pixels between the corresponding pixel and the edge of the saturation region, wherein the function has an extremum in the saturation region.

4. A method according to claim 3 comprising identifying pixels in the saturation region by scanning the color model values of pixels in the image data along a first scan axis to determine a start pixel at one edge of a saturation region and an end pixel at another edge of the saturation region.

5. A method according to claim 4 wherein the extremum of the function is located substantially midway between the start pixel and the end pixel.

6. A method according to claim 5 wherein the function comprises a parabola having a vertex located substantially midway between the start pixel and the end pixel.

7. A method according to claim 4 wherein the magnitude of the function at the extremum depends, at least in part, on a number of pixels between the start pixel and the end pixel.

8. A method according to claim 4 wherein the magnitude of the function at the extremum is determined on the basis of at least one of:

a gradient between the color model values of the start pixel and at least one pixel preceding the start pixel and a gradient between the color model values of the end pixel and at least one pixel following the end pixel.

9. A method according to claim 5 wherein the function is one of linearly monotonically increasing and linearly monotonically decreasing from the start pixel toward a pixel located substantially midway between the start pixel and the end pixel and wherein the function is the other of linearly monotonically increasing and linearly monotonically decreasing from the pixel located substantially midway between the start pixel and the end pixel toward the end pixel.

10. A method according to claim 4 wherein a value of the function is unity for at least one of:

the start pixel and the end pixel.

11. A method according to claim 2 comprising determining the magnitude of each adjustment based, at least in part, on a number of pixels between the start pixel and the end pixel.

12. A method according to claim 2 comprising determining the magnitude of each adjustment on the basis of at least one of:

a gradient between the color model values of the start pixel and at least one pixel preceding the start pixel and a gradient between the color model values of the end pixel and at least one pixel following the end pixel.

13. A method according to claim 4 comprising determining a location of the extremum on the basis of at least one of:

a gradient between the color model values of the start pixel and at least one pixel preceding the start pixel and a gradient between the color model values of the end pixel and at least one pixel following the end pixel.

14. A method according to claim 1 comprising determining the magnitude of each adjustment based, at least in part, on a number of pixels in the saturation region.

15. A method according to claim 1 comprising determining the magnitude of each adjustment based, at least in part, on a gradient between the color model values of at least one pixel on the edge of the saturation region and at least one pixel outside of the saturation region.

16. A method according to claim 1 comprising determining the magnitude of each adjustment based, at least in part, on one or more temporally previous values of the color model value for the corresponding pixel.

17. A method according to claim 1 comprising determining the magnitude of each adjustment based, at least in part, on one or more other color model values for the corresponding pixel.

18. A method according to claim 1 comprising determining the magnitude of each adjustment based, at least in part, on:

detecting a lens flare pattern surrounding the saturation region; and

using a model of the lens flare pattern to predict color model values of the pixels in the saturation region.

19. A method according to claim 1 comprising initially adjusting the color model values of the pixels in the image data to form an intermediate higher bit depth representation of the image data and wherein adjusting the color model value of each of the individual pixels in the saturation region is performed on pixels of the intermediate higher bit depth representation.

20. A method according to claim 19 wherein initially adjusting the color model values of the pixels in the image data comprises at least one of:

scaling the color model values of the pixels in the image data; and

offsetting the color model values of the pixels in the image data.

21. A method according to claim 20 comprising scaling the color model values of the pixels in the image data and wherein scaling color model values of the pixels in the image data is uniform as between pixels in the image data.

22. A method according to claim 19 wherein identifying pixels in a saturation region is performed on pixels of the intermediate higher bit depth representation.

23. A method according to claim 2 wherein scanning color model values of pixels in the image data along a first scan axis comprises determining start and end pixels for one or more additional saturation regions within a line of pixels, wherein the color model value of each individual pixel in the one or more additional saturation regions is one of:

above the upper saturation threshold and below the lower saturation threshold.

24. A method according to claim 23 comprising adjusting the color model value of each of the individual pixels in the one or more additional saturation regions by a corresponding adjustment, a magnitude of each adjustment for each pixel in each of the one or more additional saturation regions dependent, at least in part, on a number of pixels between the corresponding pixel and at least one of:

the start pixel of the additional saturation region and the end pixel of the additional saturation region.

25. A method according to claim 2 comprising repeating scanning color model values for a plurality of lines of pixels of the image data along the first scan axis and adjusting the color model value of each of the individual pixels in each saturation region until the entire image has been scanned and adjusted along the first scan axis.

26. A method according to claim 25 comprising repeating scanning color model values for a plurality of lines of pixels of the image data along a second scan axis and adjusting the color model value of each of the individual pixels in each saturation region until the entire image has been scanned and adjusted along the second scan axis.

27. A method according to claim 26 comprising combining color model values of image data scanned and adjusted along the first scan axis with color model values of image data scanned and adjusted along the second scan axis to form the higher bit depth representation.

28. A method according to claim 27 wherein combining color model values comprises obtaining an average of color model values of image data scanned and adjusted along the first scan axis with color model values of image data scanned and adjusted along the second scan axis.

29. A method according to claim 27 wherein combining color model values comprises obtaining an average of color model values of image data scanned and adjusted along the first scan axis with color model values of image data scanned and adjusted along the second scan axis to obtain intermediate values and blurring groups of two or more adjacent pixels of the intermediate values to form the higher bit depth representation.

30. A method according to claim 1 comprising adjusting the color model value of each of the individual pixels in the saturation region by scaling the color model value of each of the individual pixels in the saturation region by a corresponding scaling factor.

31. A method according to claim 1 wherein the color model values comprise a mathematical combination of other color model values.

32. A method according to claim 1 comprising, after adjusting the color model values of the individual pixels in the saturation region, further adjusting the color model values of all of the pixels in the image data to maximize the range of the color model values in the higher bit depth representation.

33. A computer program product comprising a non-transitory medium carrying computer readable instructions which, when executed by a processor, cause the processor to execute a method for converting image data from a lower bit depth representation to a higher bit depth representation, the method comprising:

identifying pixels in a saturation region wherein a color model value of each individual pixel in the saturation region is one of;

above an upper saturation threshold and below a lower saturation threshold; and

adjusting the color model value of each of the individual pixels in the saturation region by a corresponding adjustment, a magnitude of each adjustment dependent, at least in part, on a number of pixels between the corresponding individual pixel and an edge of the saturation region.

34. A system for processing image data, the system comprising a processor for converting image data from a lower bit depth representation to a higher bit depth representation, the processor configured to:

identify pixels in a saturation region wherein a color model value of each individual pixel in the saturation region is one of;

above an upper saturation threshold and below a lower saturation threshold; and

adjust the color model value of each of the individual pixels in the saturation region by a corresponding adjustment, a magnitude of each adjustment dependent, at least in part, on a number of pixels between the corresponding individual pixel and an edge of the saturation region.

35. A system according to claim 34, the processor configured to identify pixels in the saturation region by scanning the color model values of pixels in the image data along a first scan axis to determine a start pixel at one edge of a saturation region and an end pixel at another edge of the saturation region.

36. A system according to claim 34, the processor configured to determine the magnitude of each adjustment based, at least in part, on a function of the number of pixels between the corresponding pixel and the edge of the saturation region, wherein the function has an extremum in the saturation region.

37. A system according to claim 36, the processor configured to identify pixels in the saturation region by scanning the color model values of pixels in the image data along a first scan axis to determine a start pixel at one edge of a saturation region and an end pixel at another edge of the saturation region.

38. A system according to claim 37 wherein the extremum of the function is located substantially midway between the start pixel and the end pixel.

39. A system according to claim 37 wherein the magnitude of the function at the extremum depends, at least in part, on a number of pixels between the start pixel and the end pixel.

40. A system according to claim 37 wherein the magnitude of the function at the extremum is determined on the basis of at least one of:

a gradient between the color model values of the start pixel and at least one pixel preceding the start pixel and a gradient between the color model values of the end pixel and at least one pixel following the end pixel.

41. A system according to claim 37 wherein a value of the function is unity for at least one of:

the start pixel and the end pixel.

42. A system according to claim 37, the processor configured to determine a location of the extremum on the basis of at least one of:

a gradient between the color model values of the start pixel and at least one pixel preceding the start pixel and a gradient between the color model values of the end pixel and at least one pixel following the end pixel.

43. A system according to claim 35, the processor configured to determine the magnitude of each adjustment based, at least in part, on a number of pixels between the start pixel and the end pixel.

44. A system according to claim 35, the processor configured to determine the magnitude of each adjustment on the basis of at least one of:

a gradient between the color model values of the start pixel and at least one pixel preceding the start pixel and a gradient between the color model values of the end pixel and at least one pixel following the end pixel.

45. A system according to claim 35, the processor configured to repeat scanning color model values for a plurality of lines of pixels of the image data along the first scan axis and to adjust the color model value of each of the individual pixels in each saturation region until the entire image has been scanned and adjusted along the first scan axis.

46. A system according to claim 45, the processor configured to repeat scanning color model values for a plurality of lines of pixels of the image data along a second scan axis and to adjust the color model value of each of the individual pixels in each saturation region until the entire image has been scanned and adjusted along the second scan axis.

47. A system according to claim 46, the processor configured to combine color model values of image data scanned and adjusted along the first scan axis with color model values of image data scanned and adjusted along the second scan axis to form the higher bit depth representation.

48. A system according to claim 34, the processor configured to determine the magnitude of each adjustment based, at least in part, on a number of pixels in the saturation region.

49. A system according to claim 34, the processor configured to determine the magnitude of each adjustment based, at least in part, on a gradient between the color model values of at least one pixel on the edge of the saturation region and at least one pixel outside of the saturation region.

50. A system according to claim 34, the processor configured to determine the magnitude of each adjustment based, at least in part, on one or more temporally previous values of the color model value for the corresponding pixel.

51. A system according to claim 34, the processor configured to determine the magnitude of each adjustment based, at least in part, on one or more other color model values for the corresponding pixel.

52. A system according to claim 34, the processor configured to determine the magnitude of each adjustment based, at least in part, on:

detecting a lens flare pattern surrounding the saturation region; and

using a model of the lens flare pattern to predict color model values of the pixels in the saturation region.

53. A system according to claim 34, the processor configured to adjust the color model value of each of the individual pixels in the saturation region by scaling the color model value of each of the individual pixels in the saturation region by a corresponding scaling factor.

54. A system according to claim 34 wherein the color model values comprise a mathematical combination of other color model values.

55. A system according to claim 34, the processor configured, after adjusting the color model values of the individual pixels in the saturation region, to further adjust the color model values of all of the pixels in the image data to maximize the range of the color model values in the higher bit depth representation.

56. A system for processing image data, the system comprising a signal processing unit configured to convert image data from a lower bit depth representation to a higher bit depth representation, the signal processing unit configured to:

identify pixels in a saturation region wherein a color model value of each individual pixel in each saturation region is one of;

above an upper saturation threshold and below a lower saturation threshold;

identify, using a two dimensional analysis of pixels of the image data, a pattern related to the saturation region;

adjust the color model value of each of the individual pixels in the saturation region by a corresponding adjustment, a magnitude of the adjustment to each individual pixel based, at least in part, on the identified pattern; and

convert the low bit depth representation of the image data to a higher bit depth representation, thereby accommodating the adjusted color model values in the higher dynamic range of the higher bit depth representation while preserving a color fidelity in regions of the image data outside the saturation region.

57. A system according to claim 56 wherein the identified pattern comprises a two-dimensional area of the saturation region.

58. A system according to claim 56 wherein the identified pattern comprises a gradient of pixels surrounding the saturation region.

59. A system according to claim 56 wherein the identified pattern comprises a presence of a lens flare feature.

60. A system according to claim 59 wherein the lens flare feature comprises at least one of:

a sun dog; and

a halo.

61. A system according to claim 56 wherein the adjustment to each individual pixel is based, at least in part, on a model of the identified pattern.

62. A system according to claim 56 wherein the image data comprises video data comprising multiple image frames and the identified pattern comprises a temporal pattern.

63. A system according to claim 62 wherein the temporal pattern comprises a number of frames for which a region has been saturated.

64. A system according to claim 56 wherein the identified pattern comprises multiple saturation regions separated by non-saturated regions.

65. A method for converting image data from a lower bit depth representation to a higher bit depth representation, the method comprising:

identifying pixels in a saturation region wherein a color model value of each individual pixel in each saturation region is one of;

above an upper saturation threshold and below a lower saturation threshold;

identifying, using a two dimensional analysis of pixels of the image data, a pattern related to the saturation region;

adjusting the color model value of each of the individual pixels in the saturation region by a corresponding adjustment, a magnitude of the adjustment to each individual pixel based, at least in part, on the identified pattern; and

converting the low bit depth representation of the image data to a higher bit depth representation, thereby accommodating the adjusted color model values in the higher dynamic range of the higher bit depth representation while preserving a color fidelity in regions of the image data outside the saturation region.

66. A method according to claim 65 wherein the identified pattern comprises a two-dimensional area of the saturation region.

67. A method according to claim 65 wherein the identified pattern comprises a gradient of pixels surrounding the saturation region.

68. A method according to claim 65 wherein the identified pattern comprises a presence of a lens flare feature.

69. A method according to claim 68 wherein the lens flare feature comprises at least one of:

a sun dog; and

a halo.

70. A method according to claim 65 wherein the adjustment to each individual pixel is based, at least in part, on a model of the identified pattern.

71. A method according to claim 65 wherein the image data comprises video data comprising multiple image frames and the identified pattern comprises a temporal pattern.

72. A method according to claim 71 wherein the temporal pattern comprises a number of frames for which a region has been saturated.

73. A method according to claim 65 wherein the identified pattern comprises multiple saturation regions separated by non-saturated regions.