System and method for generating a photograph

• US 8,145,004 B2
• Filed: 05/27/2008
• Issued: 03/27/2012
• Est. Priority Date: 05/27/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Inventors: Camp, Jr., William O., Marcinkiewicz, Walter M., Bowen, Toby J.
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US12/127,074
• Publication Number:
  US 20090297063A1
• Time in Patent Office: 1,400 Days
• Field of Search: None
• US Class Current: 382/254
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/195 the array comprising a two-dimensional array or a combination of two-dimensional arrays
• CPC Class Codes: H04N 1/19584 Combination of arrays
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0458 Additional arrangements for improving or optimising scanning resolution or quality
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/702 SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout

Abstract

A camera assembly for generating high resolution photograph includes a sensor assembly that has two regions of different effective resolution. Image data from multiple images that are captured with the sensor assembly are combined to generate output image data in the form of a high resolution picture. A first region of the sensor is used to capture image data for the images that are combined. A second region, that is of higher effective resolution than the first region, is used to capture image data for use in aligning the image data from the plural images prior to combination.

Claims

1. A camera assembly, comprising:

a sensor assembly that includes an image capture component and an image alignment component, the image alignment component located at a periphery of the image capture component, the image alignment component having a higher density of photo sensors per unit area and higher effective resolution than the image capture component; and

a controller that controls the generation of an output photograph, the controller configured to;

control the sensor assembly to capture at least two images, wherein there is a time delay between the capture of each image and each image comprises;

image data captured by the image capture component; and

alignment data captured by the image alignment component;

identify at least one alignment feature in the image data or the alignment data corresponding to a first of the images;

identify the at least one alignment feature in the alignment data corresponding to a second one of the images and determine an offset between the first and second images from corresponding locations of the alignment feature in the first and second images;

align the first and second images to remove offset corresponding to an integer multiple of a predetermined pixel pitch and leave an offset amount corresponding to a residual fraction of the pixel pitch; and

combine the image data for the first image with the image data for the second image.

2. The camera assembly of claim 1, wherein the resolution of the output photograph is greater than a native resolution of the image capture component.

3. The camera assembly of claim 1, wherein the image data is combined using interpolation.

4. The camera assembly of claim 1, wherein a pitch of photo sensors of the image capture component is a multiple of a pitch of photo sensors of the image alignment component.

5. The camera assembly of claim 1, wherein the predetermined pixel pitch is a pitch of photo sensors of the image alignment component.

6. The camera assembly of claim 1, wherein the image data from the image alignment component for the first image populates a corresponding portion of the output photograph.

7. The camera assembly of claim 1, further comprising a memory having a pixel resolution for the output photograph that is a function of a resolution of the image alignment component, the controller populating the memory with the combination of the image data from the captured images.

8. The camera assembly of claim 1, wherein the identified alignment feature from the first image is from the image data output by the image alignment component.

9. The camera assembly of claim 1, wherein the image alignment component is a Vernier sensing element located adjacent the image capture component.

10. The camera assembly of claim 9, wherein the predetermined pixel pitch is a pitch of photo sensors of the image capture component.

11. The camera assembly of claim 9, wherein the identified alignment feature from the first image is from the image data output by the image capture component.

12. The camera assembly of claim 1, wherein the image alignment component and the image capture component have different optical pathways to focus light from a scene respectively onto the image alignment component and the image capture component.

13. The camera assembly of claim 12, wherein the optical pathway for the image alignment component has a higher magnification than the optical pathway for the image capture component.

14. The camera assembly of claim 12, wherein the predetermined pixel pitch is a pitch of photo sensors of the image alignment component.

15. A method of generating a photograph, comprising:

controlling a sensor assembly to capture at least two images, wherein there is a time delay between the capture of each image, the sensor assembly having an image capture component and an image alignment component, the image alignment component located at a periphery of the image capture component, and the image alignment component having a higher density of photo sensors per unit area and higher effective resolution than the image capture component, wherein each image comprises;

image data captured by the image capture component; and

alignment data captured by the image alignment component;

identifying at least one alignment feature in the image data or the alignment data corresponding to a first of the images;

identifying the at least one alignment feature in the alignment data corresponding to a second one of the images and determining an offset between the first and second images from corresponding locations of the alignment feature in the first and second images;

aligning the first and second images to remove offset corresponding to an integer multiple of a predetermined pixel pitch and leave an offset amount corresponding to a residual fraction of the pixel pitch; and

combining the image data for the first image with the image data for the second image.

16. The method of claim 15, wherein the image alignment component is a Vernier sensing element located adjacent the image capture component.

17. The method of claim 15, wherein the image alignment component and the image capture component have different optical pathways to focus light from a scene respectively onto the image alignment component and the image capture component.