Image capturing apparatus and method therefor

• US 7,092,019 B1
• Filed: 05/30/2000
• Issued: 08/15/2006
• Est. Priority Date: 05/31/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ueda, Kazuhiko, Tsuchiya, Takashi, Ogata, Masami
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): AGGARWAL, YOGESH K
• Application Number:
  US09/580,166
• Time in Patent Office: 2,268 Days
• Field of Search: 348/362, 348/208.99, 348/208.1, 348/208.4, 348/208.6, 348/208.12, 382/294, 382/216, 382/217, 382/218
• US Class Current: 348/263
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

An image capturing apparatus and a method therefor are proposed for the production of a natural composite image corresponding to an object to be imaged. A misregistration amount (d) of each image (xS1) except for a reference image (xL1) is detected with respect to the reference image, misregistration of each image except for the reference image with respect to the reference image is corrected on the basis of the misregistration amount, and then the reference image and each misregistration corrected image (xS4) except for the reference image are composed, thereby making it possible to avoid a degraded image quality which would otherwise be produced when a plurality of images are composed without correction, and consequently produce a natural composite image corresponding to an object to be imaged.

Claims

1. An image capturing apparatus for capturing a plurality of images with different amounts of exposure to compose said images into a single composite image, said image capturing apparatus comprising:

integrating means for integrating pixel values for each pixel train in the horizontal direction to produce an array in the vertical direction, and for integrating pixel values for each pixel train in the vertical direction to produce a data array in the horizontal direction, for each of said plurality of images;

misregistration amount detecting means for detecting the amounts of misregistration of respective images except for a reference image using feature points of images, said reference image selected from said plurality of images, with respect to said reference image;

wherein the misregistration amount detecting means comprises a feature point extracting means for extracting feature points from said reference image and respective images except for said reference image, respectively;

wherein said feature point extracting means extracts feature points in said reference image and feature points in each of the images except for said reference image based on the respective data arrays in the vertical and horizontal directions;

misregistration correcting means for correcting the misregistration of the respective images except for said reference image with respect to said reference image based on said misregistration amounts; and

image composing means for composing said reference image and all of the respective misregistration corrected images except for said reference images.

2. The image capturing apparatus according to claim 1, wherein said misregistration amount detecting means comprises:

confidence level calculating means for calculating a confidence level for each of the images except for said reference image as said misregistration amount for each of a plurality of previously provided misregistration amount candidates, based on a positional relationship of the feature points in said reference image and the feature points in the respective images except for said reference image; and

maximum value detecting means for detecting said misregistration amount candidate which presents a maximum confidence level within said misregistration amount candidates as said misregistration amount in each of the images except for said reference image.

3. The image capturing apparatus according to claim 2, wherein

said confidence level calculating means corrects misregistration between a feature point in said reference image and a feature point in an image except for said reference image using said misregistration amount candidate, and counts the number of positions of said feature points which correspond to each other to calculate said confidence level, thereby calculating said confidence level in each of said plurality of misregistration amount candidates.

4. The image capturing apparatus according to claim 2, wherein

said feature point extracting means extracts plural types of feature points from said reference image and each of the images except for said reference image;

and

said confidence level calculating means calculates the confidence level for each of the images except for said reference images as said misregistration amount for each of said plurality of previously provided misregistration amount candidates based on a positional relationship of said plural types of feature points extracted from said reference image and said plural types of feature points extracted from each of the images except for said reference image.

5. The image device according to claim 2, wherein

said feature point is a pixel which forms an edge at which a pixel value presents an abrupt change.

6. The image capturing apparatus according to claim 2, wherein

said feature point extracting means extracts feature points in said reference image and feature points in each of the images except for said reference image based on luminance information included in said plurality of images.

7. An imaging method for capturing a plurality of images with different amounts of exposure to compose said images into a single composite image, said imaging method comprising the steps of:

integrating pixel values for each pixel train in the horizontal direction to produce an array in the vertical direction, and for integrating pixel values for each pixel train in the vertical direction to produce a data array in the horizontal direction, for each of said plurality of images;

detecting the amounts of misregistration of respective images except for a reference image, said reference image selected from said plurality of images, with respect to said reference image;

extracting feature points from said reference image and respective images except for said reference image, respectively, based on the respective data arrays in the vertical and horizontal directions;

correcting the misregistration of the respective images except for said reference image with respect to said reference image based on said misregistration amounts; and

composing said reference image and all of the respective misregistration corrected images except for said reference images.

8. The imaging method according to claim 7, comprising the steps of:

calculating a confidence level for each of the images except for said reference image as said misregistration amount for each of a plurality of previously provided misregistration amount candidates, based on a positional relationship of the feature points in said reference image and the feature points in the respective images except for said reference image; and

detecting said misregistration amount candidate which presents a maximum confidence level within said misregistration amount candidates as said misregistration amount in each of the images except for said reference image.

9. The imaging method according to claim 8, wherein

misregistration between a feature point in said reference image and a feature point in an image except for said reference image is corrected by using said misregistration amount candidate, and the number of positions of said feature points which correspond to each other is counted to calculate said confidence level, thereby said confidence level in each of said plurality of misregistration amount candidates is calculated.

10. The imaging method according to claim 8, comprising the steps of:

extracting plural types of feature points from said reference image and each of the images except for said reference image; and

calculating the confidence level for each of the images except for said reference image as said misregistration amount for each of said plurality of previously provided misregistration amount candidates based on a positional relationship of said plural types of feature points extracted from said reference image and said plural types of feature points extracted from each of the images except for said reference image.

11. The imaging method according to claim 8, wherein

said feature point is a pixel which forms an edge at which a pixel value presents an abrupt change.

12. The imaging method according to claim 8, comprising the step of:

extracting feature points in said reference image and feature points in each of the images except for said reference image based on luminance information included in said plurality of images.