Generation of simulated long exposure images in response to multiple short exposures

• US 8,228,400 B2
• Filed: 04/17/2009
• Issued: 07/24/2012
• Est. Priority Date: 04/17/2009
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Liu, Ming-Chang, Robertson, Mark
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US12/426,101
• Publication Number:
  US 20100265357A1
• Time in Patent Office: 1,194 Days
• Field of Search: 348/239, 348/222.1, 348/362, 348/208.99, 348/208.4, 348/208.12, 348/208.13, 348/297, 382/107, 382/254
• US Class Current: 348/239
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

Simulating a long exposure-time image from a sequence of short exposure-time images captured at slightly different times. The sequence of images is combined in a temporal integration process to create a long exposure image that simulates the output from a still camera, steadied by a tripod, whose light-sensitive material has been exposed to the same scene from the time of the beginning of the first input image of the sequence to the last image of the input sequence. The method overcomes limitations of hand-held video and image recording devices, allowing the user to easily create effects normally associated with high-end digital still cameras under expert control.

Claims

1. An apparatus for simulating a long exposure-time image, comprising:

means for capturing a sequence of images which are digital images;

a computer with memory coupled to said computer; and

programming adapted for execution on said computer for;

receiving a sequence of normal exposure images at a normal exposure time setting from said means for capturing, in which image capture duration of said normal exposure time setting is less than a desired long exposure interval over which a simulated long exposure time image is to be obtained;

selecting a set of sequential images, from within said sequence of images, which span the desired long exposure interval;

performing global motion estimation and compensation wherein camera motion arising between each of said sequential images is eliminated by post processing;

identifying regions of independent motion from a background in the set of sequential images; and

performing artificial motion blurring on said regions of independent motion, prior to combining the set of sequential images through temporal integration, to create at least one long exposure image.

2. An apparatus as recited in claim 1, wherein said long exposure image spans a simulated exposure time of at least one-half (½

) second.

3. An apparatus as recited in claim 1, wherein stable long exposure images are created by said apparatus without necessitating use of a tripod, or other camera mount.

4. An apparatus as recited in claim 1, wherein said sequence of images comprise frames of a video sequence.

5. An apparatus as recited in claim 1, wherein said long exposure image comprises a digital still photograph.

6. An apparatus as recited in claim 1, further comprising programming adapted for execution on said computer for automatically generating long exposure streaking and blurring to create said artificial motion blurring.

7. An apparatus as recited in claim 6, further comprising:

programming adapted for execution on said computer for utilizing image sequence depth prior to target frame as a determiner of extent to which said long exposure streaking and blurring are generated.

8. An apparatus as recited in claim 1, further comprising programming adapted for execution on said computer for:

converting pixel values in said set of sequential images into linear light values prior to temporal integration when combining;

wherein said pixel values are converted by applying an inverse of a response function for a camera; and

wherein said camera response function comprises a mapping of light response of an imaging device of said camera.

9. An apparatus as recited in claim 1, further comprising programming adapted for execution on said computer for correcting for desired contrast and color balance after combining said sequence of images.

10. A camera configured for automatically creating still images having a simulated exposure time spanning a desired time period, comprising:

an electronic imaging element within a camera adapted for capturing a sequence of digital images;

a computer with memory coupled to said computer, said computer configured for controlling said electronic imaging element of said camera; and

programming adapted for execution on said computer for;

capturing, within said memory, a sequence of images spanning a desired time period, which temporally span a desired long exposure interval;

compensating for camera motion among said sequence of images;

converting pixel values in said sequence of set of sequential images into linear light values;

identifying regions of independent motion from a background in the sequence of images; and

performing artificial motion blurring on said regions of independent motion prior to combining the sequence of images through temporal integration to create one or more simulated long exposure images.

11. An apparatus as recited in claim 10, wherein stable long exposure images are created by said camera without necessitating use of a tripod, or other camera mount.

12. An apparatus as recited in claim 10, wherein said sequential images comprise frames of a video sequence.

13. An apparatus as recited in claim 10, wherein said long exposure images comprise long exposure digital still photographs.

14. An apparatus as recited in claim 10, further comprising programming adapted for execution on said computer for automatically generating long exposure streaking and blurring to create said artificial motion blurring.

15. An apparatus as recited in claim 10, further comprising programming adapted for execution on said computer for correcting for desired contrast and color balance after combining said sequence of images.

16. A method of creating a long exposure time still photograph from a captured sequence of short exposure images, comprising:

capturing a set of sequential images with a digital image capturing device, said set of sequential images spanning a desired time frame for a long exposure time still photograph;

compensating for motion of the digital image capturing device among said set of sequential images wherein camera motion arising between each of said sequence of digital images is eliminated by post processing;

identifying regions of independent motion from a background in the set of sequential images;

performing artificial motion blurring on said regions of independent motion prior to combining said set of sequential images through temporal integration to create at least one long exposure time still photograph having said desired time frame.

17. A method as recited in claim 16, further comprising:

converting pixel values in said set of sequential images into linear light values prior to performing temporal integration;

wherein said pixel values are converted by applying inverse of a response function for a camera; and

wherein said camera response function comprises a mapping of light response of an imaging device of said camera.

18. A method as recited in claim 16, wherein stable long exposure images are created by said method without necessitating mounting of the image capturing device on a tripod or other fixed mount.

19. A method as recited in claim 16, further comprising programming adapted for execution on said computer for automatically generating long exposure streaking and blurring to create said artificial motion blurring.

20. A method as recited in claim 16, further comprising programming adapted for execution on said computer for correcting for desired contrast and color balance after combining said sequence of images.