Imaging device, imaging device image output method, and computer for assigning a dynamic range

• US 7,609,320 B2
• Filed: 09/18/2003
• Issued: 10/27/2009
• Est. Priority Date: 09/25/2002
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Okamura, Keisuke
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): HERNANDEZ, NELSON D
• Application Number:
  US10/528,968
• Publication Number:
  US 20060033823A1
• Time in Patent Office: 2,231 Days
• Field of Search: 348/222.1, 348/223.1, 348/362, 348/382, 348/361, 348/367, 348/689, 348/257, 348241-251, 348/389, 348/221.1
• US Class Current: 348/362
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

An image pickup apparatus for performing dynamic assignment of an output image in a dynamic range according to a subject. An image pickup device picks up an image of a subject, a signal processing section generates a composite image having a relatively wider dynamic range than at least either of the dynamic ranges of a long-time exposure image picked up with a relatively long exposure time by the image pickup device and a short-time exposure image picked up with a relatively short exposure time by the image pickup device, by synthesizing the long-time exposure image and the short-time exposure image A, control section compresses the composite image and dynamically varies the assignment proportion of a high luminance dynamic range to a low-middle luminance dynamic range in a dynamic range of an output image to be outputted as a video signal.

Claims

1. An image pickup apparatus comprising:

an image pickup device for picking up an image of a subject;

a signal processing section for generating a composite image, the composite image having a relatively wider dynamic range than at least either a dynamic range of a long-time exposure image picked up with a relatively long exposure time by said image pickup device or a dynamic range of a short-time exposure image picked up with a relatively short exposure time by said image pickup device, by synthesizing said long-time exposure image and said short-time exposure image; and

a control section for compressing said composite image and dynamically varying an assignment proportion of a high luminance dynamic range to a low-middle luminance dynamic range in a dynamic range of an output image to be outputted as a video signal,

wherein the signal processing section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and

wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same.

2. The image pickup apparatus according to claim 1,

wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least a luminance region which occupies said composite image.

3. The image pickup apparatus according to claim 1,

wherein said control section corrects said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range each time said composite image is generated.

4. The image pickup apparatus according to claim 2,

wherein said luminance region is at least either a high luminance region or a low-middle luminance region.

5. The image pickup apparatus according to claim 1,

wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least an average luminance signal level of a high luminance region which occupies said composite image.

6. The image pickup apparatus according to claim 1,

wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least an average luminance signal level of a low-middle luminance region which occupies said composite image.

7. The image pickup apparatus according to claim 1,

wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least a high luminance region which occupies said composite image.

8. The image pickup apparatus according to claim 1,

wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least a low middle luminance region which occupies said composite image.

9. The image pickup apparatus according to claim 1,

wherein said control section at least monotonically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range.

10. An image pickup apparatus comprising:

an image pickup device for picking up an image of a subject;

a signal processing section for generating a composite image, the composite image having a relatively wider dynamic range than at least either a dynamic range of a long-time exposure image picked up with a relatively long exposure time by said image pickup device or a dynamic range of a short-time exposure image picked up with a relatively short exposure time by said image pickup device, by synthesizing said long-time exposure image and said short-time exposure image; and

a control section for compressing said composite image and dynamically assigning the dynamic range of said composite image to the dynamic range of an output image to be outputted as a video signal,

wherein the signal processing section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and

wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same.

11. The image pickup apparatus according to claim 10,

wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image according to at least a luminance region which occupies said composite image.

12. The image pickup apparatus according to claim 10,

wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image each time said composite image is generated.

13. The image pickup apparatus according to claim 10,

wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image according to at least a high luminance region which occupies said composite image.

14. The image pickup apparatus according to claim 13,

wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image according to at least an average luminance signal level of said high luminance region which occupies said composite image.

15. The image pickup apparatus according to claim 10,

wherein said dynamic range is at least either a high luminance dynamic range or a low-middle luminance dynamic range.

16. The image pickup apparatus according to claim 15,

wherein said control section dynamically assigns assignment of at least either said high luminance dynamic range or said low-middle luminance dynamic range of said output image according to a high luminance region which occupies said composite image.

17. The image pickup apparatus according to claim 15,

wherein said control section dynamically assigns a section of said high luminance dynamic range of said output image to said low-middle luminance dynamic range according to at least a decrease of a high luminance region which occupies said composite image.

18. The image pickup apparatus according to claim 15,

wherein said control section dynamically assigns a section of said high luminance dynamic range of said output image to said low-middle luminance dynamic range according to at least a decrease of an average luminance signal level of a high luminance region which occupies said composite image.

19. The image pickup apparatus according to claim 15,

wherein said control section dynamically assigns a section of said low-middle luminance dynamic range of said output image to said high luminance dynamic range according to at least an increase of a high luminance region which occupies said composite image.

20. The image pickup apparatus according to claim 15,

wherein said control section dynamically assigns a section of said low-middle luminance dynamic range of said output image to said high luminance dynamic range according to at least an increase of an average luminance signal level of a high luminance region which occupies said composite image.

21. The image pickup apparatus according to claim 15,

wherein said control section at least monotonically varies an assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range of the said output image.

22. An image pickup apparatus comprising:

an image pickup device for picking up an image of a subject;

a detection section for detecting an image signal of a long-time exposure image picked up with a relatively long exposure time by said image pickup device, and an image signal of a short-time exposure image picked up with a relatively short exposure time by said image pickup device;

a synthesis section for generating a composite image from said long-time exposure image and said short-time exposure image on the basis of a switch luminance signal level determined from said image signals;

a control section for compressing said composite image according to a luminance region which occupies said composite image, and dynamically assigning a dynamic range of an output image to be outputted as a video signal; and

a compression section for compressing the dynamic range of said composite image on the basis of dynamic assignment of said dynamic range of said output image,

wherein the detection section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and

wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same.

23. The image pickup apparatus according to claim 22,

wherein said luminance region is at either a high luminance region or a low-middle luminance region.

24. The image pickup apparatus according to claim 22,

wherein said synthesis section acquires, from said short-time exposure image, pixels corresponding to at least a higher luminance signal level than said switch luminance signal level among pixels constructed in said composite image.

25. The image pickup apparatus according to claim 22

wherein said synthesis section acquires, from said long-time exposure image, pixels corresponding to at least a lower luminance signal level than said switch luminance signal level among pixels constructed in said composite image.

26. The image pickup apparatus according to claim 22,

wherein said dynamic range is at least either a high luminance dynamic range or a low-middle luminance dynamic range.

27. The image pickup apparatus according to claim 22,

wherein said control section determines a compression gain for compressing a luminance signal level of said composite image on the basis of at least an assignment proportion of a high luminance dynamic range of said output image to a low-middle luminance dynamic range thereof

28. The image pickup apparatus according to claim 22,

wherein said control section determines at least a high luminance compression gain for compressing a luminance signal level of a high luminance region in said composite image and a low-middle luminance compression gain for compressing a luminance signal level of a low-middle luminance region.

29. The image pickup apparatus according to claim 28,

wherein said control section further includes a compression gain calculation section for determining, for each luminance signal level of said composite image, at least either a final high luminance compression gain or a final low-middle luminance compression gain which are to be used by said compression section, on the basis of at least either said high luminance compression gain or said low-middle luminance compression gain.

30. The image pickup apparatus according to claim 22,

wherein said control section dynamically varies an assignment proportion between a high luminance dynamic range and a low-middle luminance dynamic range of said output image according to at least a high luminance region which occupies said composite image.

31. The image pickup apparatus according to claim 30,

wherein said control section at least monotonically varies the assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range of said output image.

32. An image pickup apparatus comprising:

an image pickup device for picking up an image of a subject;

a detection section for detecting an image signal of a long-time exposure image picked up with a relatively long exposure time by said image pickup device, and an image signal of a short-time exposure image picked up with a relatively short exposure time by said image pickup device;

a synthesis section for excluding said luminance signal level from a target to be synthesized, when at least said image signal corresponding to a luminance signal level of either said long-time exposure image or said short-time exposure image is absent, and synthesizing said long-time exposure image and said short-time exposure image on the basis of a switch luminance signal level;

a control section which dynamically assigns a dynamic range of an output image to be outputted as a video signal in which a composite image is compressed, according to a luminance region which occupies said composite image in which said long-time exposure image and said short-time exposure image are synthesized; and

a compression section for compressing the dynamic range of said composite image on the basis of dynamic assignment of said dynamic range of said output image,

wherein the detection section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and

wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same.

33. The image pickup apparatus according to claim 32,

wherein said synthesis section selects said luminance signal level lower than said switch luminance signal level in said long-time exposure image, as a target for said composite image.

34. The image pickup apparatus according to claim 32,

wherein said synthesis section selects said luminance signal level higher than said switch luminance signal level in said short-time exposure image, as a target for said composite image.

35. The image pickup apparatus according to claim 32,

wherein said luminance region is at least either a high luminance region or a low-middle luminance region.

36. The image pickup apparatus according to claim 32,

wherein said dynamic range is at least either a high luminance dynamic range or a low-middle luminance dynamic range.

37. The image pickup apparatus according to claim 32,

wherein said control section dynamically varies an assignment proportion of a high luminance dynamic range to a low-middle luminance dynamic range of said output image according to at least a high luminance region which occupies said composite image.