Image processing device that merges a plurality of images together, image shooting device provided therewith, and image processing method in which a plurality of images are merged together

• US 8,154,634 B2
• Filed: 05/19/2009
• Issued: 04/10/2012
• Est. Priority Date: 05/19/2008
• Status: Active Grant

• Current Assignee: Xacti Corporation (Office Support KK)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Mori, Yukio, Fukumoto, Shimpei
• Primary Examiner(s): WHIPKEY, JASON T
• Application Number:
  US12/468,379
• Publication Number:
  US 20090284610A1
• Time in Patent Office: 1,057 Days
• Field of Search: 348/252, 348/253, 348/208.13, 382/255, 382/256, 382/258, 382/264, 382/275
• US Class Current: 348/252
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A merged image is generated by merging together a first image obtained by shooting with a reduced exposure time, a second image obtained by shooting with an increased exposure time, and a third image obtained by filtering out a high-frequency component from the first image. Here, a merging ratio at which the second and third images are merged together is determined by use of differential values obtained from the second and third images. Also, a merging ratio at which the first image and the second and third images (a fourth image) are merged together is determined by use of edge intensity values obtained from an image based on the first image.

Claims

1. An image processing device, comprising:

a merging portion that generates a merged image by merging together a first image obtained by shooting, a second image obtained by shooting with an exposure time that is longer than an exposure time with which the first image is shot, and a third image generated by filtering out a high-frequency component from the first image; and

a differential value calculation portion that calculates a differential value between each pair of corresponding pixels in the second and third images and

an edge intensity value calculation portion that extracts edges in an image based on the first image and calculates an edge intensity value of each pixel,

wherein

the merging portion determines a first merging ratio for each pixel according to the differential value, the first merging ratio being a merging ratio between the second and third images;

the merging portion determines a second merging ratio of each pixel according to the edge intensity value, the second merging ratio being a merging ratio between the first image and the second and third images; and

the merging portion generates the merged image by pixel-by-pixel merging processing based on the first and second merging ratios.

2. The image processing device according to claim 1,

wherein

the merging portion determines the first merging ratio such that a merging ratio of the second image is larger at a pixel where the differential value is smaller;

and

the merging portion determines the second merging ratio such that a merging ratio of the first image is larger at a pixel where the edge intensity is higher.

3. The image processing device according to claim 1,

wherein

the edge intensity value calculation portion calculates the edge intensity value according to the third image.

4. The image processing device according to claim 1,

wherein

the edge intensity value calculation portion calculates the edge intensity value according to the first image.

5. An image processing device, comprising:

a merging portion that generates a merged image by merging together a first image obtained by shooting, a second image obtained by shooting with an exposure time that is longer than an exposure time with which the first image is shot, and a third image generated by filtering out a high-frequency component from the first image; and

an enlargement portion that enlarges a second small image that is an image smaller than the second image to generate and output the second image,

wherein

the merging portion merges the first image, the second image outputted from the enlargement portion, and the third image together to generate a merged image.

6. An image processing device, comprising:

a merging portion that generates a merged image by merging together a first image obtained by shooting, a second image obtained by shooting with an exposure time that is longer than an exposure time with which the first image is shot, and a third image generated by filtering out a high-frequency component from the first image; and

an enlargement portion that enlarges a second small image that is an image smaller than the second image to generate and output the second image,

a reduction portion that reduces the first image to be substantially a size of the second small image to generate and output a first small image;

a displacement correction portion that compares the first and second small images, converts coordinates of pixels in the second small image, and achieves pixel-to-pixel correspondence between the second image outputted from the enlargement portion and the first image,

wherein

the merging portion merges the first image, the second image outputted from the enlargement portion, and the third image together to generate a merged image.