METHOD FOR FORMING AN IMPROVED IMAGE USING IMAGES WITH DIFFERENT RESOLUTIONS

• US 20100026839A1
• Filed: 08/01/2008
• Published: 02/04/2010
• Est. Priority Date: 08/01/2008
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Enge, Amy D., Border, John N.
• Granted Patent: US 8,130,278 B2
• Time in Patent Office: Days
• US Class Current: 348/231.200
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/84 for processing colour signals

Abstract

Multiple images of a scene are acquired over a contemporaneous period of time. Most of the multiple images are lower resolution images acquired with a lower resolution than the other of the multiple images. A corrected set of images is formed at least by correcting for motion present between at least some of the lower resolution images. In addition, a synthesized image is formed at least by merging (a) at least a portion of at least one of the images in the corrected set of images, and (b) at least a portion of at least one of the other of the multiple images. The synthesized image is stored in a processor-accessible memory system. The synthesized image exhibits improved image quality including reduced motion blur, a higher signal-to-noise ratio, and higher resolution over conventional techniques.

Claims

1. A method implemented at least in part by a data processing system, the method for forming an image, and the method comprising the steps of:

instructing acquisition of multiple images of a scene over a contemporaneous period of time, wherein most of the multiple images are lower resolution images acquired with a lower resolution than the other of the multiple images;

instructing formation of a corrected set of images at least by correcting for motion present between at least some of the lower resolution images;

instructing formation of a synthesized image at least by merging (a) at least a portion of at least one of the images in the corrected set of images, and (b) at least a portion of at least one of the other of the multiple images; and

instructing a storing of the synthesized image in a processor-accessible memory system.

2. The method of claim 1, wherein the step of instructing a formation of a corrected set of images further comprises instructing a merging of the lower resolution images that were motion-corrected into a single lower-resolution image, wherein the corrected set of images comprises only the single lower-resolution image.

3. The method of claim 2, wherein the step of instructing formation of a corrected set of images further comprises aligning features within at least some of the lower resolution images to offset such images from one another, and wherein the aligning occurs prior to the merging of the lower resolution images.

4. The method of claim 3, wherein the aligning of the lower resolution images is done to align features with one of the other of the multiple images.

5. The method of claim 1, wherein the other of the multiple images comprises only a single higher resolution image.

6. The method of claim 1, wherein the step of instructing acquisition further comprises instructing acquisition of the lower resolution images prior to acquisition of the other of the multiple images.

7. The method of claim 6, wherein the step of instructing acquisition further comprises instructing that the lower resolution images be saved in a rolling buffer.

8. The method of claim 6, further comprising the step of receiving a partially-depressed-capture-button signal, which causes the instructing of the acquisition of the lower resolution images, wherein the method further comprises the step of receiving a fully-depressed-capture-button signal after receiving the partially-depressed-capture-button signal, and wherein receipt of the fully-depressed-capture-button signal causes the instructing of the acquisition of the other of the multiple images.

9. The method of claim 1, wherein at least some of the lower resolution images are video images or preview images.

10. The method of claim 8,

wherein the lower resolution images are video images,

wherein the step of instructing acquisition further comprises instructing acquisition of audio data while acquiring the lower resolution images, and

wherein the method further comprises the step of instructing a storing of the audio data and the lower resolution images in the processor-accessible memory system in a manner that links the audio data and the lower resolution images to the synthesized image.

11. The method of claim 1, wherein the lower resolution images have substantially the same exposure time as the other of the multiple images.

12. The method of claim 1, wherein the lower resolution images have longer exposure times than the other of the multiple images.

13. The method of claim 1, wherein the step of instructing formation of the corrected set of images, the step of instructing formation of the synthesized image, and the step of instructing the storing of the synthesized image occur prior to noise-reduction-processing performed on any of the multiple images.

14. The method of claim 1, wherein the step of instructing formation of the corrected set of images, the step of instructing formation of the synthesized image, and the step of instructing the storing of the synthesized image occur after image-sensor-related-noise has been removed from each of the multiple images.

15. The method of claim 1, wherein the step of instructing formation of the corrected set of images, the step of instructing formation of the synthesized image, and the step of instructing the storing of the synthesized image occur after edges have been identified in each of the multiple images.

16. The method of claim 1, wherein the step of instructing formation of the corrected set of images, the step of instructing formation of the synthesized image, and the step of instructing the storing of the synthesized image occur after the multiple images have been fully color corrected and edge enhanced.

17. A processor-accessible memory system storing instructions configured to cause a data processing system to implement a method for forming an image, wherein the instructions comprise:

instructions for acquiring multiple images of a scene over a contemporaneous period of time, wherein most of the multiple images are lower resolution images acquired with a lower resolution than the other of the multiple images;

instructions for forming a corrected set of images at least by correcting for motion present between at least some of the lower resolution images;

instructions for forming a synthesized image at least by merging (a) at least a portion of at least one of the images in the corrected set of images, and (b) at least a portion of at least one of the other of the multiple images; and

instructions for storing the synthesized image in a processor-accessible memory system.

18. The memory system of claim 17, wherein the other of the multiple images comprises only a single higher-resolution image.

19. The memory system of claim 17, wherein the instructions for acquiring further comprise instructions for acquiring the lower resolution images prior to acquiring the other of the multiple images.

20. A system comprising:

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for forming an image, wherein the instructions comprise;

instructions for acquiring multiple images of a scene over a contemporaneous period of time, wherein most of the multiple images are lower resolution images acquired with a lower resolution than the other of the multiple images;

instructions for forming a corrected set of images at least by correcting for motion present between at least some of the lower resolution images;

instructions for forming a synthesized image at least by merging (a) at least a portion of at least one of the images in the corrected set of images, and (b) at least a portion of at least one of the other of the multiple images; and

instructions for storing the synthesized image in a processor-accessible memory system.

21. The system of claim 20, wherein the other of the multiple images comprises only a single higher-resolution image.

22. The system of claim 20, wherein the instructions for acquiring further comprise instructions for acquiring the lower resolution images prior to acquiring the other of the multiple images.