Image processing apparatus and image processing method and program

• US 8,797,433 B2
• Filed: 02/17/2012
• Issued: 08/05/2014
• Est. Priority Date: 02/24/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kaizu, Shun, Mitsunaga, Tomoo
• Primary Examiner(s): SPINKS, ANTOINETTE T
• Application Number:
  US13/399,222
• Publication Number:
  US 20120218426A1
• Time in Patent Office: 900 Days
• Field of Search: 348/296, 348/208.4, 348/241
• US Class Current: 348/296
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 25/533 by using differing integration times for different sensor regions
• CPC Class Codes: H04N 25/583 with different integration times
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array

Abstract

An image processing apparatus includes an intermediate image generating unit configured to input an image which has been shot with differing exposure times set by region, generates a plurality of exposure pattern images corresponding to differing exposure times based on the input image, and generates a plurality of timing images which are difference images of the plurality of exposure pattern images; and a distortion correction processing unit configured to generate a corrected image equivalent to an exposure processing image at a predetermined exposure time by synthesizing processing of the plurality of timing images.

Claims

1. An image processing apparatus comprising:

an intermediate image generating unit configured to

input an image which has been shot with differing exposure times set by region,

generate a plurality of exposure pattern images corresponding to differing exposure times based on said input image, and

generate a plurality of timing images which are difference images of said plurality of exposure pattern images; and

a distortion correction processing unit configured to generate a corrected image equivalent to an exposure processing image at a predetermined exposure time by synthesizing processing of said plurality of timing images.

2. The image processing apparatus according to claim 1, wherein said intermediate image generating unit is configured to

input an image having exposure time shifting in increments of regions shot by a focal-plane shutter operation,

generate a plurality of exposure pattern images corresponding to different exposure times based on the input image, and

generate a plurality of timing images which are difference images of said plurality of exposure pattern images.

3. The image processing apparatus according to claim 1, wherein said intermediate image generating unit generates a plurality of timing images having exposure times shorter than that of said input image;

and wherein said distortion correction processing unit synthesizes the plurality of timing images having exposure times that are shorter than that of said input image and generates said corrected image.

4. The image processing apparatus according to claim 1, wherein said distortion correction processing unit selects a timing image having an exposure time that is included in the exposure times of the corrected image that is to be generated, and executes synthesizing processing applying the selected timing image.

5. The image processing apparatus according to claim 1, wherein, for a timing image having all of the exposure times in the exposure times of the corrected image to be generated, at the time of computing pixel values of said corrected image, said distortion correction processing unit performs processing to reflect the pixel values of the timing image in all of the pixel values of the corrected image;

and wherein, for a timing image having an exposure time overlapping portion with a portion of the exposure times of the corrected image to be generated, said distortion correction processing unit performs processing to reflect the pixel values of the timing image in the pixel values of the corrected image according to the ratio of overlapping portions.

6. The image processing apparatus according to claim 1, wherein the timing image generated by said intermediate image generating unit is an image that has been set with exposure times that differ in pixel row increments;

and wherein said distortion correction processing unit selects a timing image having the exposure times included in the exposure times of the corrected image to be generated in pixel row increments, and executes synthesizing processing applying the selected timing image.

7. The image processing apparatus according to claim 1, wherein said intermediate image generating unit generates said timing image as continuously shot images with an exposure time shorter than that of said input image;

and

said intermediate image generating unit further including an output unit to output the timing image generated by said intermediate image generating unit as a high frame rate image.

8. The image processing apparatus according to claim 1, said intermediate image generating unit further including

a plurality of intermediate image generating units configured to execute processing in parallel as to a plurality of continuously shot images;

wherein said distortion correcting unit generates a corrected image equivalent to the exposure processing image having predetermined exposure times, with the synthesizing processing of the plurality of timing images generated by said plurality of intermediate image generating units.

9. The image processing apparatus according to claim 1, wherein a region, which is a control increment of exposure time of said input image, is one of a pixel block made up of a plurality of pixels, or a row, or a pixel.

10. The image processing apparatus according to claim 1, further comprising:

a motion detecting unit configured to execute motion detection in region increments of the input image;

wherein the output of said distortion correction processing unit is applied only for the region having motion detected, and an output image is generated.

11. The image processing apparatus according to claim 1, further comprising:

an imaging device; and

a control unit configured to execute exposure time control in region increments of said imaging device.

12. An image processing method executed by an image processing apparatus, said method comprising:

performing, by an intermediate image generating unit,

inputting of an image which has been shot with differing exposure times set by region,

generating of a plurality of exposure pattern images corresponding to differing exposure time based on the input image, and

generating of a plurality of timing images which are difference images of said plurality of exposure pattern images; and

generating, by a distortion correction processing unit, a corrected image equivalent to an exposure processing image at a predetermined exposure time by synthesizing processing of said plurality of timing images.

13. A non-transitory computer-readable medium comprising instructions to execute image processing with an image processing apparatus, the instructions comprising:

causing an intermediate image generating unit to

input an image which has been shot with differing exposure times set by region,

generate a plurality of exposure pattern images corresponding to differing exposure time based on the input image, and

generate a plurality of timing images which are difference images of said plurality of exposure pattern images; and

causing a distortion correction processing unit to generate a corrected image equivalent to an exposure processing image at a predetermined exposure time by synthesizing processing of said plurality of timing images.